Wednesday, November 01, 2017

7 reasons to abandon multiple-choice questions

Long the staple of e-learning and a huge range of low and high stakes tests, the MCQ should be laid to rest or at least used sparingly. It has several flaws…
1. Probability
25% of getting it right on the standard four-option item, makes it less than taxing. True false, really a two option MCQ, is of course worse.
2. Unreal
You rarely, if ever in the real world, have to choose things from short lists. This makes the test item somewhat odd and artificial, disassociated from reality.  They seem dissonant, as this is not the way our brains work (we don't cognitively select from four item lists). They are also weak on and therefore weak on the transfer of knowledge to the real world.
3. Distractors distract
It is too easy to remember the distractor, as opposed to the right answer. The fact that they are designed to distract makes them candidates for retention and so MCQs can become counterproductive.
4. Can be cheated
Pick longest item, second-guess the designer. Look for opposites and internal logic of distractor options. There are credible cheat-lists for multiple choice – Pounstone’s research shows that these approaches increase your chance of getting better scores. (20 cheats here)
5. Surface misleading
Take these two questions.
What is the Capital of Lithuania? Tallin, Vilnius, Riga. Minsk
What is the Capital of Lithuania? Berlin, Vilnius, Warsaw, Helsinki
Surface differences in options make these very different test items. And it is easy to introduce these surface differences, reducing the validity of the test items and test.
6. Difficult to write
I have written a ton of MCQs over 35 years – believe me they are seriously difficult to write. It is easy to select the noun from the text and come up with three other nouns. What is difficult it to test, is real understanding.
7. Little effort
This is the big one. As Roediger and McDaniel state in their book Make It Stick, choosing from a list requires little cognitive effort. You choose from a limited set of options, and do not use effortful recall (which is in itself increases retention).

Multiple-choice is not a terrible test item but it has had its day as the primary test item in online learning. We’re still designing test items in lock-step because the tools encourage us to do so, ignoring more powerful open-response questions that require recall and the powerful act of writing/typing, which in itself, reinforces learning. New tools, such as WildFire, the AI-driven content creation service, focuses on open-response and effortful learning, for all of these seven reasons and more. The learner has to make more effort as that means deeper processing and higher retention.

 Subscribe to RSS


Post a Comment

<< Home