All of the Above – 20 ways to cheat MULTIPLE CHOICE questions

Many multiple choice questions are poorly written. What better way to expose these errors than write a cheat-sheet for learners?

Of course, writing good test items is far more difficult than many imagine. Many make obvious mistakes. An interesting way to coming at this problem is to do some reverse engineering. If you think this doesn’t work, think again. Poundstone number crunched 100 tests with a total of 2456 questions to get some of these statistical biases.

Second-guessing the test designer

So here goes with my 20 ways to cheat Multiple Choice tests:

1.  Skip the hard questions, mark them with a cross, and go back to them. This means you’ll not lose marks for unanswered easy questions.

2.  Cover the options and try to answer. Prevents being misled by clever wrong options.

3.  If in doubt choose ‘B’, poor questions designers do not truly randomise the right options and have a bias towards ‘B’. Next best is ‘C’.

4.  If in doubt choose the ‘longest option’. Question designers often cannot make a right option any shorter, but have complete freedom with wrong options. This is quite common.

5.  If in doubt choose TRUE, in true/false questions, as they come easier to mind for designers.

6.  Reverse answers. Statistically, there is more T/F alteration in tests than in truly randomized sequences, as the brain struggles to randomise properly. So, if you’re sure you’ve got one right, reverse the next answer.

7.  Eliminate the outlier. Look for similarities in options and eliminate outliers e.g. 4p-q, 2p+q, 4p+q, 3p+q. Look for these internal patterns.

8.  If two options are opposites, one is likely to be correct. Designers tend to create the opposite of the right answer as their first distractor, so one of them is likely to be correct.

9.  Favour options with careful qualifiers, such as ‘sometimes, occasionally etc.’ as tested knowledge usually has more finite than absolute qualities.

10.  Be wary of options with absolute qualifiers, such as ‘always, never etc’. As these are often too definite to be correct.

11.  Choose a middle order option i.e. out of 100, 150. 200, 250, choose 150 or 200. Designers tend to have a bias, where right answers tend to be lower than the highest and higher than the lowest option.

12.  For questions that demand an ‘except’ or ‘not’, mark each option with a T for true and F for false against each option. And underline the word ‘not’ as it’s sometimes missed.

13.  ‘All of the above’ and ‘None of the above’ are both significantly likely to be correct. For it to be correct, the writer has to design options that were all correct, so, if you can’t spot any wrong answers, or see that two or more are correct, it increases the probability of ‘All of the above’ being correct. Similarly with ‘None of the above’.

14.  Typo or punctuation error, the option is likely to be wrong. Writers tend to proofread correct answers only.

15.  Look for grammatical agreement between the question and its options; ‘An.....’ and words starting with vowels or agreement between subject, object or verb.

16.  If you’re stuck, go with the ‘Least bad rule’. Eliminate least likely answers first.

17.  Look for clues about answers from other questions. Designers often, unintentionally, put clues, even answers, to questions in other questions.

18.  Ignore never heard of answers. If you’ve never heard of the answer, it’s likely to be made up and incorrect.

19.  Go with your first impression. The more you read, the more you tend to read into the wrong options.

20.  Always guess, unless there is a penalty. It’s a 1 in 4 chance, so don’t give it up.

Conclusion

This crib sheet can be used by students or question designers to improve their tests. 

Next post - how to write a great MC question...
And there's 500 more tips like these, all categorised, here.