All of the Above - how to cheatMultiple Choice questions, so it was heartening (or disheartening) to see some real evidence that backed up those claims. Poundstone chewed up stats on 100 tests from schools, colleges and other professional sources. He found the following weaknesses in serious professional certification exams, SATs and in many educational assessments that matter in terms of selection.
True-False test strategy
1. Go through the whole test getting those you know right first.
2. If the known answers before and after the one you don’t know are the same, choose the opposite, as there’s more alternation of TRUE and FALSE than in a truly random sequence, so to choose a different answer from the last one increases your chance of getting it right from 50% to 63%.
3. If the known answers before and after are different, choose TRUE as there’s more right TRUE (56%) answers than FALSE (44%).
Multiple Choice test strategy
1. Pick ‘none of the above’ or ‘all of the above’ as this gives you up to a 90% increase on random guessing.
2. Pick B on a four choice test, as this gives you an 11% increase, as human testers can’t randomise.
3. Avoid the previous choice gives you an 8% increase, again as human testers can’t randomise.
4. Choose the longest item. Even on the supposedly unbeatable SAT exams.
5. Eliminate outliers.
6. Always guess, as instinct can sometimes kick in to produce the right answer.
We live in an age of constant assessment, that is a shame. However, to live in an age of inefficient assessment is even worse, as smart, coached and tutored students can ‘game’ the system. For assessment designers it may be worth looking at these and other ‘game’ strategies to eliminate the weaknesses of your tests. First randomise the right answers. Humans just cannot randomise. Then look for the common biases. One final point. Randomisation is an algorithm. This is a case where algorithmic power of computers can eliminate human cognitive bias and weak competence.