OH 1
Multiple Choice: Format and Uses

Format

• Stem
  • Direct question
  • Or incomplete statement
• Alternatives
  • Correct or best answer
  • Distracters (foils, decoys)

1. Knowledge outcomes
   • Terminology
   • Facts
   • Principles
   • Methods and procedures
2. Understanding and application
   • Identify application of facts/principles
   • Interpret cause-effect relations
   • Justify methods and procedures

OH 2
Multiple Choice: Advantages and Limitations

Advantages

1. Learning outcomes from simple to moderately complex can be measured
2. Highly structured and clear tasks are provided
3. Broad sample of achievement can be measured
4. Relatively free from response sets
5. Incorrect alternatives provide diagnostic information
6. Scoring is easy, objective, and reliable
7. Compared to short-answer, less likely to be vague and ambiguous
8. Compared to true-false, more reliable
9. Compared to matching, requires less homogeneous material

1. Time consuming to construct good items (but higher quality for time spent)
2. Often difficult to find plausible distracters
3. Ineffective for measuring some types of problem solving, ability to organize and express ideas (because require selection, not production)
4. Abstracted from factors in natural settings
5. Scores can be influenced by reading ability

OH 4
Multiple Choice: General Suggestions for Constructing Them

• Clearly stated problem
• Plausible distracters
• No clues

1. Present a single clearly formulated problem.
2. Use simple, clear language and no irrelevant material.
3. Put as much of the wording of the item as possible in its stem.
4. State in positive form, whenever possible.
5. Highlight any negative wording.

1. All are parallel and grammatically consistent with the stem.
2. Intended answer is correct or clearly best.
3. Avoid verbal clues (e.g., verbal associations).
4. Keep relative length from becoming a clue.
5. Distracters are plausible and attractive to the uninformed.
6. Minimize or avoid use of "none of the above" and "all of the above."
7. Vary position of correct answer in a random order.

1. Understanding items must have some, but not too much, novelty.
2. Don’t use multiple choice when other item types more appropriate.

OH 5
Ideas for Good Distracters

1. Draw from students’ most common errors
2. Use important sounding words (sometimes)
3. Include verbal associations with the stem
4. Use textbook language
5. Think of the most likely misunderstandings
6. Keep alternatives parallel in form and grammatically consistent with the stem
7. Keep alternatives homogeneous in content
8. Keep alternatives similar in length, vocabulary, complexity, structure

OH 6
Multiple Choice Items: Do you know when to use them?

When are multiple choice items more appropriate or less appropriate than the following item types?

• Short answer
• True-false
• Matching
• Performance-based tasks

OH 7
Illustrative Stems for Multiple Choice Knowledge Items
(Source: Gronlund, 1998, Assessment of Student Achievement, p. 57)

1. Knowledge of terminology
• What word means the same as __________?
• Which statement best defines the term_________?
• In this sentence, what is the meaning of the word________?
2. Knowledge of specific facts
• Where would you find ________?
• Who first discovered ________?
• What is the name of ________?
3. Knowledge of conventions
   • What is the correct form for ________?
   • Which statement indicates correct usage of ________?
   • Which of the following rules applies to ________?
4. Knowledge of trends and sequences
• Which of the following best describes the trend of ________?
• What is the most important cause of ________?
• Which of the following indicates the proper order of ________?
5. Knowledge of classifications and categories
   • What are the main types of ________?
   • What are the major classifications of ________?
   • What are the characteristics of ________?
6. Knowledge of criteria
   • Which of the following is a criterion for judging ________?
   • What is the most important criterion for selecting ________?
   • What criteria are used to classify ________?
7. Knowledge of methodology
   • What method is used for ________?
   • What is the best way to ________?
   • What would be the first step in making ________?
8. Knowledge of principles and generalizations
   • Which statement best expresses the principle of ________?
   • Which statement best summarizes the belief that ________?
   • Which of the following principles best explains ________?
9. Knowledge of theories and structures
   • Which statement is most consistent with the theory of ________?
   • Which of the following best describes the structure of ________?
   • What evidence best supports the theory of ________?

OH 8
Illustrative Stems for Multiple Choice Comprehension and Application Items
(Source: Gronlund, 1998, Assessment of Student Achievement, p. 58)

• Which of the following is an example of ________?
• What is the main thought expressed by ________?
• What are the main differences between ________?
• What are the common characteristics of ________?
• Which of the following is another form of ________?
• Which of the following best explains ________?
• Which of the following best summarizes ________?
• Which of the following best illustrates ________?
• What do you predict would happen if ________?
• What trend do you predict in ________?

• Which of the following methods is best for ________?
• What steps should be followed in applying ________?
• Which situation would require the use of ________?
• Which principle would be best for solving ________?
• What procedure is best for improving ________?
• What procedure is best for constructing ________?
• What procedure is best for correcting ________?
• Which of the following is the best plan for ________?
• Which of the following provides the proper sequence for ________?
• What is the most probable effect of ________?