Overheads for Unit 4--Chapter 6 (Planning classroom tests and assessments)

OH 1
Major Points from Last Week

  1. Your goal is valid, reliable, useful assessment
  2. Which requires:
    1. Determining what is to be measured
    2. Defining it precisely
    3. Minimizing measurement of irrelevancies
  3. And is promoted by following good procedures

 

OH 2
Four Steps in Planning an Assessment

  1. Deciding its purpose
  2. Developing test specifications
  3. Selecting best item types
  4. Preparing items

 

OH 3
Step 1: Decide the Purpose

What location in instruction?

  1. pre-testing
  2. during instruction

  3. end of instruction

 

OH 4
Step 2: Develop Test Specifications

 

OH 5
Sample of a Content Domain (For this course)

  1. trends/controversies in assessment
  2. interdependence of teaching, learning, and assessment
  3. purposes and forms of classroom assessment
  4. planning a classroom assessment (item types, table of specs)
  5. item types (advantages and limitations)
  6. strategies for writing good items
  7. compiling and administering classroom assessments
  8. evaluating and improving classroom assessments
  9. grading and reporting systems
  10. uses of standardized tests
  11. interpreting standardized test scores

 

OH 6
Sample Table of Specifications (For chapters 6 and 7 of this course)

Sample SLOs (you would typically have more)

Bloom Levels

Remember

Understand

Apply

Analyze

Evaluate

Create

Identifies definition of key terms (e.g., validity)

X          

Identifies examples of threats to test reliability and validity

  X        

Selects best item type for given objectives

    X      

Compares the pros and cons of different kinds of tests for given purposes

      X    

Evaluates particular educational reforms (e.g., whether they will hurt or help instruction)

        X  

Create a unit test

          X

Total number of items

           

 

OH 7a
Spot the Poor Specific Learning Outcomes (For use with previous table of specifications)

Which entries are better or worse than others? Why? Improve the poor ones.

  1. Knowledge
    1. Knows correct definitions
    2. Able to list major limitations of different types of items
  2. Comprehension
    1. Selects correct item type for learning outcome
    2. Understands limitations of true-false items
    3. Distinguishes poor true-false items from good ones
  3. Application
    1. Applies construction guidelines to a new content area
    2. Creates a table of specifications
  4. Analysis
    1. Identifies flaws in poor items
    2. Lists general and specific learning outcomes
  5. Synthesis
    1. Lists general and specific content areas
    2. Provides weights for areas in table of specifications
  6. Evaluation
    1. Judges quality of procedure/product
    2. Justifies product
    3. Improves a product

 

OH 7b
Why are These Better Specific Learning Outcomes?

  1. Knowledge
    1. Selects correct definitions
    2. Lists major limitations of different item types
  2. Comprehension
    1. Selects proper procedures for assessment purpose
    2. Distinguishes poor procedures from good ones
    3. Distinguishes poor decisions/products from good ones
  3. Application
    1. Applies construction guidelines to a new content area
  4. Analysis
    1. Identifies flaws in procedure/product
    2. Lists major and specific content areas
    3. Lists general and specific learning outcomes
  5. Synthesis
    1. Creates a component of the test
    2. Provides weights for cells in table of specifications
  6. Evaluation
    1. Judges quality of procedure/product
    2. Justifies product
    3. Improves a product

 

OH 8
Step 3: Select the Best Types of Items/Tasks

What types to choose from? Many!

  1. objective--supply-type
    1. short answer
    2. completion
  2. objective--selection-type
    1. true-false
    2. matching
    3. multiple choice
  3. essays
    1. extended response
    2. restricted response
  4. performance-based
    1. extended response
    2. restricted response

Which type to use? The one that fits best!

  1. most directly measures learning outcome
  2. where not clear, use selection-type (more objective)
    1. multiple choice best (less guessing, fewer clues)
    2. matching only if items homogeneous
    3. true-false only if only two possibilities

 

OH 9
Strengths and Limitations of Objective vs. Essay/Performance

Objective Items

Essay/Performance Tasks

 

OH 10
Step 4: Prepare Items/Tasks

Strategies to Measure the Domain WellóReliably and Validly

  1. specifying more precise learning outcomes leads to better-fitting items
  2. use 2-way table to assure good sampling of complex skills
  3. use enough items for reliable measurement of each objective
  4. keep in mind how good assessment can improve (not just measure) learning

Strategies to Avoid Contamination

  1. eliminate barriers that lead good students to get the item wrong
  2. donít provide clues that help poor students get the item correct

 

OH 11
General Suggestions for Item Writing

  1. use table of specifications as guide
  2. write more items than needed
  3. write well in advance of testing date
  4. task to be performed is clear, unambiguous, unbiased, and calls forth the intended outcome
  5. use appropriate reading level (donít be testing for ancillary skills)
  6. write so that items provide no clues (minimize value of "test-taking skills")
    1. a/an
    2. avoid specific determiners (always, never, etc.)
    3. donít use more detailed, longer, or textbook language for correct answers
    4. donít have answers in an identifiable pattern
  7. write so that item provides no clues to other items
  8. seeming clues should lead away from the correct answer
  9. experts would agree on the answer
  10. if item revised, recheck its relevance