As we reach the midpoint of the semester, where tests are as numerous to the students as the falling leaves, it is fitting to look back to October 1924 and the eloquent words of Grantland Rice…Outlined against a blue-gray October sky, the Four Horsemen rode again. In dramatic lore they are known as Famine, Pestilence, Destruction and Death. These are only aliases. Their real names are… Quizzes, Exams, Midterms, and Finals. Ok, maybe these Four Horsemen of Assessment are not as daunting as Stuhldreher, Miller, Crowley and Layden were on the football field that Saturday, but your students may feel equally overmatched, unnerved and bowled over during tests if your question design is not following the game plan for quality assessment. Students should be tested in a fair but challenging manner, which means instructors must take great care in composing multiple choice questions. Engaging student brains with well-designed questions will validly determine whether they have learned and retained the targeted skills and information to meet your course objectives.
Let’s review some general, but important, considerations when developing quality multiple choice items.
Stem and Options
After first drafting the stem (positively-worded) and correct answer (which together makes a complete and correct sentence), 3-5 distractor options should be created that are:
- Plausible enough to be chosen and possibly be argued as correct
- True statements that don’t answer the question
- Reflective of common student errors, assumptions or misconceptions
- Familiar, yet incorrect, words or phrases
- Reasonably likely to be chosen by those who don’t fully know the material
Visual, Verbal, Grammatical and Logical Cues
Review your answer, considering cues students may use to identify the answer, as these defeat the goal of the instructor to engage the type of thinking, learning, and feedback needed to by students to understand and connect the material to learning objectives. Cues may include:
- Grammar structure that is not consistent with the stem sentence.
- One distractor that is much longer than others (“too long to be wrong”)
- A single option that contains all the other options
- An option with a vague word or phrase like “usually,” “typically” or “may be”
- Two options with the same meaning
- One option in textbook/lecture language (correct), others in everyday language
Other Common Mistakes or Answer Identifiers
Finally, it is equally advisable to be aware of some common mistakes/issues when creating your multiple choice items. These elements can add confusion or vagueness to your item so avoid using:
- Negative wording of the stem (Which of the following is NOT…)
- Microscopically-fine distinctions between options (unless absolutely necessary)
- An option that is humorous, funny or cute
- Options that include All or None of the above, All, Always, or Never
- Stems that seek to find the exception in a list of correct options (All of the following are signs of XYZ except …)
- One option combines to other options (Both A & B)
——————————————–
References
Rice, Grantland “The Four Horsemen”. New York Herald Tribune, 18 October 1924. Retrieved from http://archives.nd.edu/research/texts/rice.htm October 11, 2019.
Hubert, Dan (2019). Creating Effective Multiple Choice Questions. University of Notre Dame. Kaneb Center for Teaching and Learning Workshop, September 25, 2019.
