Reasons to abandon multiple-choice questions

Long the staple of e-learning and a huge range of low and high stakes tests, the MCQ should be laid to rest or at least used sparingly. Their huge popularity has been due to several factors. First they are an artefact, a hangover from cardboard templates placed over squares to identify 'Xs". The tools for online learning also make them easy to write. This does not make it easy to write them well, in the same way that giving someone Word doesn't make them a novelist or poet.

But it has several flaws…

1. Probability

25% of getting it right on the standard four-option item, makes it less than taxing. True false, really a two option MCQ, is of course worse. I say 25% by research but Rodriguez (2005) shows that three options are optimal, then it goes to 33%.

2. Unreal

You rarely, if ever in the real world, have to choose things from short lists. This makes the test item somewhat odd and artificial, disassociated from reality.  They seem dissonant, as this is not the way our brains work (we don't cognitively select from four item lists in recall or automaticity). They are also weak on recall and therefore weak on the transfer of knowledge to the real world.

3. Distractors distract

It is too easy to remember the distractor, as opposed to the right answer. The fact that they are designed to distract makes them candidates for retention and so MCQs can become counterproductive. The research shows that four option MCQs often contain spurious and extraneous distractors that just add cognitive load and the chance of false recall.

4. Can be cheated

Pick longest item, second-guess the designer. Look for opposites and internal logic of distractor options. There are credible cheat-lists for multiple choice – Poundstone’s research shows that these approaches increase your chance of getting better scores. (20 cheats here)

5. Surface misleading

Take these two questions.

What is the Capital of Lithuania? Tallin, Vilnius, Riga. Minsk

What is the Capital of Lithuania? Berlin, Vilnius, Warsaw, Helsinki

Surface differences in options make these very different test items. And it is easy to introduce these surface differences, reducing the validity of the test items and test. MCQs often have huge variances depending on the options presented.

6. Difficult to write

I have written a ton of MCQs over 35 years – believe me they are seriously difficult to write. It is easy to select the noun from the text and come up with three other nouns. What is difficult it to test, is real understanding.

7. Little effort

This is the big one. As Roediger and McDaniel state in their book Make It Stick, choosing from a list requires little cognitive effort. You choose from a limited set of options, and do not use effortful recall (which is in itself increases retention).

Conclusion
Multiple-choice is not a terrible test item but it has had its day as the primary test item in online learning. We’re still designing test items in lock-step because the tools encourage us to do so, ignoring morepowerful open-response questions that require recall and the powerful act of writing/typing, which in itself, reinforces learning. New tools, such as WildFire, the AI-driven content creation service, focuses on open-response and effortful learning, for all of these seven reasons and more. The learner has to make more effort as that means deeper processing and higher retention.