Research
Rodriquez (2005) studied 27 research papers on multiple-choice questions and found that the optimal number of options is THREE not four. Vyas (2008) showed the same was true in medical education. But God is in the detail and their findings surfaced some interesting phenomena.
Four/five options increases the effort the learner has to make but this does not increase the reliability of the test
Four increases the effort needed to write questions and as distractors are the most difficult part of the test item to identify and write, they were often very weak.
Reduction in the number of options from four to three (surprisingly) increased the reliability of test scores.
Tests were shorter leaving more time for teaching.
Cheating MCQs
In my 20 tips on how to increase your score in multiple choice questions, I mention the fact that there's often one distractor that's obviously wrong, another reason for abandoning that often difficult to write fourth distractor.
Weaknesses in MCQs
Of course, multiple-choice is, in itself, weaker than open-input, which is why we can go one step further and have open response, either single words or short answers. Natural Language Processing allows AI not only to create such questions automatically but also provide accurate student scores. AI can also create both of these question types (also MCQs if desired) automatically, saving organisations time and money. This is surely the way forward in online learning. Beyond this is voiced input, again a step forward, and AI also allows this type of input in online learning. If you are interested in learning more about this, see WildFire.
Cheating MCQs
In my 20 tips on how to increase your score in multiple choice questions, I mention the fact that there's often one distractor that's obviously wrong, another reason for abandoning that often difficult to write fourth distractor.
Weaknesses in MCQs
Of course, multiple-choice is, in itself, weaker than open-input, which is why we can go one step further and have open response, either single words or short answers. Natural Language Processing allows AI not only to create such questions automatically but also provide accurate student scores. AI can also create both of these question types (also MCQs if desired) automatically, saving organisations time and money. This is surely the way forward in online learning. Beyond this is voiced input, again a step forward, and AI also allows this type of input in online learning. If you are interested in learning more about this, see WildFire.
Conclusion
So, you can safely reduce the number of MCQ options from five/four to THREE and not reduce the reliability of the tests. Indeed, there is evidence that it improves the reliability of tests. Not only that it saves organisations, teachers and learners time.
Bibliography
Rodriquez (2005) “Three Options Are Optimal for Multiple-Choice Items: A Meta-Analysis of 80 Years of Research”, Educational Measurement: Issues and Practice, Volume 24, Number 2, June 2005, pp. 3-13.
Vyas R (2008) Medical Education. The National medical Journal of India, Volume 21, No 3
No comments:
Post a Comment