Multiple choice tests are:
- the best assessment strategy regardless of the situation
- easy to write and easy to grade
- unable to accurately measure a student’s knowledge
- assessment tools that can be useful in many situations
Multiple choice assessments are one among many assessment tools that can be used to provide formative or summative feedback about a student’s ability to achieve a class’s learning goals. Like any tool, it has strengths and weaknesses, i.e. situations where it will be the right tool for a job and situations where it will be as a hammer to a screw. And to continue the metaphor, before using a tool in any given situation, it is important to ensure that it is the right tool and that it is not faulty.
Some of the strengths of multiple choice tests include the following
- Objective (limited ambiguity in answers)
- Grading tends to be easier
- Statistical analyses are typically more straightforward
- High reliability and validity when implemented well
While some of their weaknesses revolve around
- Excessive guessing can skew results
- Poorly written questions and responses
Another possible weakness is related to the answer space being bounded. It could be argued that a student must only pick the correct answer rather than generating it. This critique holds more weight when the questions or answers are not well-formed, but should be kept in mind when designing problems.
Good practices
- Use one learning objective per question (avoid trivia)
- If the question does not match up with a learning goal for the class, then it likely does not need to be tested on. Whereas a question containing multiple learning goals may be overly complicated and will likely be more approachable if split into separate questions.
- The question (stem) should be a complete statement
- A well-written question should allow a partial solution to be reached after reading the question but before reading the possible answers. A question/answer set where the question provides no content, ‘Which of the following is true’, should be reworded or restructured.
- Grammar should match between the stem and the possible responses
- If one or more of the responses (distractors) differs in person, gender, number, etc. then that answer is more likely to be dismissed by the student, regardless of the actual content.
- Offer a partial point incentive to limit guessing
- As an example, for a question where there are four possible answers, assuming no prior knowledge, a guess has a 1 in 4 chance of being correct. As such, offering partial credit of ¼ the total value of the question if it is left blank can help increase the reliability of the test. If someone has partial knowledge and is able to eliminate some number of answers, then they can still benefit from guessing, but for someone who has no knowledge, leaving the question blank is to their benefit and their lack of a guess will better reflect who is confident in their answers. See Ref. 1 for a more in-depth examination of this idea.
- Vary the number of answers
- This deals with the problem of guessing in a different manner and is more amenable for problems involving calculation. Instead of using three or four answers, instead provide five, six, or even ten possible choices. The more plausible answers provided, the less likely a guess is to be correct, increasing the reliability of the question. Again see Ref. 1 for a closer examination of this concept.
- Additionally, if the problem is a calculation, then the majority of the work is solving the problem and the number of answers does not greatly affect the time for the problem. Whereas the time required for a problem where each answer must be read and analyzed will scale with the number of answers
The above are some of the important ideas to keep in mind as you prepare a multiple choice test and if this has whetted your appetite please join us this Friday for the Kaneb’s workshop on “Writing multiple choice questions”.
Additionally, for a deeper exploration of the above topics please see the papers and good practice guides referenced below
References
Campbell, M. L. Multiple-Choice Exams and Guessing: Results from a One-Year Study of General Chemistry Tests Designed To Discourage Guessing Journal of Chemical Education 2015 ASAP
Towns, M. H. Guide to Developing High-Quality, Reliable, and Valid Multiple-Choice Assessments Journal of Chemical Education 2014 91 1426-1431
Other Useful Links
http://ctl.utexas.edu/teaching/assess-learning/question-types/multiple-choice
