10 reasons why 'Leaderboards suck in learning

Gamification has been around for years (over 30) but I fear it has recently descended into a superficial, Pavlovian box of tricks, overhyped by vendors keen to prove how ‘cool’ and ‘contemporary’ they are . In practice, much of it is a bit hokey, and insensitive to learning theory and, I fear, the whole point of the bigger game – learning.

Don’t get me wrong, I like gamification, in the sense of games’ techniques grounded in good learning theory. These include: levels of attainment, catastrophic failure and repeats, effortful learning, mini-sims and deliberate practice. What I witness, however, is the superficial buzz of the arcade; leaderboards, points, board games, sound effects, noise and animated piffle.

Gamification only works when the cognitive load, that gamification always entails, results in greater learning outcomes. It you go for approaches that are superficially buzzy, yet work against learning, in that they draw the learner away from actual learning, realism, context and transfer, you will have failed.  You may even have demotivated, rather than motivated most of your learners.

This is especially true of the one gamification technique I do not like, and do not recommend - the leaderboard. The word betrays itself, embedding as it does the the word ‘Leader’, but there are many other problems:

1. Decoupled from standards

Leaderboards are inappropriate for personal development. Learning is not a race, not the survival of the fittest. Any league table that ranks people into a linear list decouples the learning from real measures of competence. You can list all you want but that list is not anchored in competences just because it is ranked. Let’s say you have 100 learners in five leagues of 20. The entire 100 may be failing but you can still sort them into a table.

2. Leaders problem

One of the problems with leaderboards is something I experienced on a project for a major bank some years ago, where we set up product quizzes on kiosks (like fruit machines) around the country. The leaderboard was quickly dominated by a few fanatics, one site in particular, who would do anything, including cheat, to stay at the top. Leaderboard encourage people to cheat. These ‘leaders’ became a problem. We had to take it down. This is something I’ve seen elsewhere.

3. Zero sum game

In general, social challenges, where individual learners are exposed to group comparisons and scores, can demotivate those who lag behind. The downside is obvious, seeing people move ahead of you makes you think you’re not good enough. For every person that moves up a place there’s someone who moves down so the motivational spur may, in total, be zero. For every leader there’s a laggard. In fact, it may be worse than this, as even those above the median point may feel disappointed. In fact the leaderboard technique tends to ignore the majority in favour of the few at the top.

4. Self determination theory

Far better to let the learner use themselves as the benchmark, set self-targets and self-goals, then encourage progress in that way. This avoids the danger of disaffecting entire groups who do not see their learning in terms of games and learning. Gamification is often best when it is almost invisible and embedded in the learning strategy. This ‘self determination’ theory is well researched and relies more on encouraging intrinsic, as opposed to extrinsic motivation.

5. Short-term gain but no long-term gain

Over time the above effects become exacerbated. You get a short-term rush and gain, followed by a longer tail of disappointment, then disillusionment.

6. Good and bad failure

Visibility of ranked failure may also decrease that learner’s willingness to learn through good failure. It’s like cholesterol, there’s good failure and bad failure. Bad failure is illusory ranking, good failure is that which you learn from. Ranked lists make learners more risk averse.

7. Tests not behaviours

It is rare that assessment within a learning programme or experience matches the actual required behaviours in the real world. The tests are often simplistic, too text based often simple multiple-choice. You can do well on leaderboards by focusing relentlessly on the types of answers required in text-based tests. See my list of ways to cheat on MCQs.

8. Lower retention

When the score or position in a league table become the goal, there is less chance that the content will actually be remembered and recalled. Deeper processing and reflection is less unlikely when your goal is a score and not competence. It’s a case of in one ear, get the score, then out the other.

9. Conflict

Leaderboards may induce conflict rather than co-operation. When winning or moving ahead of your colleagues becomes the goal, collaborative or team effort becomes secondary. So this individualistic approach may destroy any semblance of collaborative or team behaviour that one would normally want to encourage in workplace learning.

10. Transfer

Transfer of skills to the real word is important and leaderboards set up an alternative, distinct and distractive world for the learner – the leaderboard. This diminishes the congruence between what is learned and the real target in the real world, that is the point of transfer.

Conclusion

All in all, leaderboards may seem like a good idea but for one upside, competitive motivation, but they have many identifiable downsides, statistically, motivationally and in terms of learning theory. The workplace is not a sporting venue, not a race for the top. Some areas may be driven by such competitive urges, arguably sales, but one should not universalise this phenomenon into learning. The learning game is not a penny arcade game, it’s about cognitive change – that’s hard and should not be trivialised. I must reiterate that I am NOT against gamification, only its superficial promises. For more see my Don't play Pavlov with gamification - 10 DOs and DON'Ts.

AI assessment apocalypse – a 5-step solution

Some describe the assessment issue in higher education as apocalyptic, destroying the very fabric of higher education. This is exaggerated. This piece offers an alternative,  to turning learning and assessment into a toxic cat-and-mouse game, where there are many more mice, and the mice are winning, It is about better and more authentic assessment.

There is no silver bullet, as this is a multivariant problem involving student motivations, teaching, institutional practices and technology. Those who simply shout ‘bring back in-person exams’ are ignoring the causes and not offering adequate solutions. The solution involves several steps.

Step 1: Stop the blame

The first step is to admit the serious nature and scale of the problem but also accept it is not the students’ fault. There is a bias in teaching, research and assessment, towards the teacher and researcher. They are the means to an end, not the end in itself. The profession all too ready to blame students and dismiss AI, when these problems were explicit before AI hit the scene. It is hopelessly utopian to expect learners not to use AI. 

Rather than accuse, the solution has to be involve not tempting them with shortcuts that resut from an accusatory environment and poor assessment. Redesign the system to allow more time for teaching and eliminate the temptation that may lead to a toxic environment of accusations, false positives and expulsions; a life-changing disaster for any young person. 

It may also be a life-changing disaster for the faculty member or administrator who ends up making a false accusation. This has already happened with a Minister for Higher Education and leaders of major educational institutions, accused of plagiarism themsleves, and removed.

Most students do not want to cheat. However, when the pressure is overwhelming, from the perception of peers (everyone is doing it, I’d be a fool not to) and parents, tteaching not as good as it could be, assessments poorly designed; students will take available shortcuts. Step 1 is to recognise that cheating is normal and that in high stakes exams, people will take high stakes risks, so don’t blame the students.

We also need to cool down on ths idea that using idea is simply congitive surrender, destroying the leaner's ability to learn. There is a fundamental flaw in most debate about cognitive surrender to AI. The argument that we should be keeping learning difficult, is very different from the idea of useful, deliberate difficulty. As I said earlier, dull lectures, poor teaching, obscure content and accessibility is a big problem that requires more focus on teaching.

Students use AI because they find it useful in learning, to find things out, expand on concepts, unravel things they find difficult, test themselves, produce flashcards for revision and practice. To ban AI would be to throw the baby out with the bathwater, and the bath.

Step 2: No silver bullet

Cheating has always been rampant in education. It was there before AI, with a range of techniques and technologies. I wrote about this in my book ‘Learning Technologies’. There was a whole section on cheating technology from Confucian silk cheat sheets to repurposed calculators, false arms and even surgical implants. Cheating has been an intrinsic feature of educational assessment. As long as there are exams, people will try to take shortcuts.

Let’s take a cold, hard look at in-person essay-based exams. For generations, smart students have looked at past papers, worked out the probability of topics appearing in their exam, pre-writing essays, then memorising them for regurgitation in the exam. The assumption was that we were testing critical thinking. Yet no one who has ever written anything using critical thinking would claim that a piece of writing, written in pen from top left to bottom right of page, without redrafting, reordering and rewriting, even approximated critical thinking. Critical thinking is an internal dialogue where you think, reconsider, revise, seriously reorder and rewrite, as you proceed. This is as far away from regurgitating essays in exams as you can get.

Even in formative essay assessments, students would readily beg, steal and borrow essays from each other, get help from their graduate parents or pay essay mills. These mills were huge enterprises with tens of thousands employed in Nairobi, China and elsewhere, where the well-educated poor provided essays and dissertations for the rich. It was generally ignored by the system (no real moral outrage, as with AI) even though everyone knew it was endemic, especially among students who were studying in their second language. Why? This became a lucrative source of income, the real reason for sliding the problem under the carpet. AI suddenly became one big essay mill, free or cheap, and everyone had access to its services. The revenues of the known cheat companies plummeted.

This is why the current emergency over assessment is really the surfacing of an old and existing problem. We can pretend it is all about AI but AI has merely surfaced a deep, existing problem. It is not fundamentally an AI problem, it is a system and human nehavu=ioural problem. A lot of cheating is the artefact of of the existing teaching and assessment processes and design. It is the same problem that pushes parents to help students with their assignments, hire tutors and pay for exam prep.

Step 2 is to recognise that in-person exams may help, and are not to be scoffed at, but they are not the whole solution, not the silver bullet.

Step 3: Create a Hub

Policies are, at best sticking plasters, at worst they exacerbate the problem. They are certainly not a solution to a large and evolving problem, as they suck up collective effort, are often ignored, then just sit there unrevised and unloved until out-of-date in relation to the advances and uses of AI. 

A policy is one thing, strategy another. Rather than rant and rail around academic integrity, or blaming students, one must understand the problem, then come up withe ‘workable’ solutions. 

There is a lot of hand-wringing and ethical hubris centred around words like integrity, responsible, ethical, trust and so on. This form of abstraction comes easy to academics and administrators but it does not tackle the problems head-on. This is a practical problem that needs workable, pragmatic design solutions.

AI was the fastedst adopted technology in the history of our species and has continued to get better, as it learns. The solution, therefore, needs to involve a process, not a single policy or event. That process needs to be owned and maintained by the institution or cluster of institutions, even nationally. At first, this needs to be a one-stop-shop for advice, tools and services on assessment. This can be part of a wider hub for the use of AI in general, by all in the institution; administrators, researchers, teachers and students. A technology that is globally universal, used by almost all students to learn, warrants this level of attention. Create that hub and keep it up to date.

Step 4: Multimodal assessment

In many subjects, if you depend on just writing as proof of learning, you do not have an AI problem, you have a learning design and assessment problem. When writing is treated as the sole proof of learning for everything, AI exposure isn’t the flaw. the flaw is in the assessment design.

In the real world, all jobs involve the doing of things, dealing with people, using tools, practical tasks. 80% of jobs in the world are deskless, and those that are deskless are being automated by AI. If you do want to switch towards skills that just focus on expression through text, then other forms of teaching and assessment are necessary.

We are now on the other side of  the Gutenberg Parenthesis, where more is available in multiple media formats, from which one can teach and learn. teachers actually speak and listen, we now listen to audiobooks and podcasts, recorded lectures, videos and audio dialogue using AI.

Multimodal assessment is the optimisation of assessment by moving beyond text, now made possible, as AI has become truly multimodal. Models have integrated all media types and can ingest text, audio, images and video, as well as output these media from your text. This offers you the opportunity to free assessment from the tyranny of pure text.

One useful shift, in a world where listening and speaking to others is likely to be more useful than simply writing, is to record oral assessments and use AI to grade them. The argument against oral exams is that they take too much time, but transcription and automatic grading can speed up the process. You also eliminate the stress and problems of worrying about who used AI. Hartmann's (2025) reoriented an upper-level humanities course around oral exams and tracked the time to show that oral exams can verify student understanding directly and, importantly, that they may not take more instructor time than essay.“ Instructor time investment proved comparable to traditional paper grading” with oral exams taking 13 hours , compared to 15 for grading papers. The main point made by the paper was that oral assessment can be integrated back into your existing courses. They may even force your students to do the work, through the idea that they will be properly assessed. 

Video based assessment of performance is also becoming possible as AI can recognise what the person is doing in an uploaded video. This swings effort away from detecting AI, towards designing assessments that assess student performance. All outputs from learners can be ingested and interpreted by AI; text, audio, images and video. 

Digial portfolios are also an option for gathering evidence across the course. What is often missed is that is shows the student thinking, the ability to recall the foundational knowledge that allows thought to flourish and build a case. It shows skills, not in just writing, but a fuller form of expression and doing. This is not to say it is right for all subjects and skills but portfolios and oral exams should be part of the teaching and assessment toolkit.

A multimodal approach also offers a solution to other problems: accessibility and dyslexia. Dyslexics love AI, as it has for years offered a text to speech option. Others find that rewriting and summarising through AI translates the content into something they can understand more readily than often abstruse academic language. In other words, AI is often used by learners to simply ‘access’ and understand content. So widen out your assessment options and think beyond just text.

Step 5: Automate assessment

Automating assessment has become possible in many cases. This is not to say that all assessment should be automated, only ‘optimally’ automated. There can still be expert validation and quality checking. This has the additional advantage of freeing up busy teaching time for actual teaching.

The simple generation and marking of quizzes in formative assessment can clearly be automated. Students do this routinely. They instinctively know the ‘test effect’ works, so build their own quizzes and flashcards with spaced-practice, using AI. Many now use specialist tools like NotebookLM and ChatGPT’s education features, to help them improve the productivity of their own learning. Learning is also being integrated into tools like Google Translate, so that you can practice learning a language through role play and immersion. There is a strong argument for automating much formative assessment via platforms which gives data back to teachers about individual student performance.

Summative assessments have to have strong input by faculty but the questions, rubrics and marking can often be automated by AI. Automating marking is the single most effective way to free up time for teaching, research and other activities. This also includes the marking and feedback of essays.

We can also automate much more detailed feedback. As Dylan Wiliam has been saying for years, far too much assessment has no forward-looking pedagogy. It is seen by students as an end-point, when it should be feeding forward. AI can do this. Have experts in the loop by all means but look at ways to automate the bulk of the work. This as a field that is advancing rapidly, as AI capability progresses.

Step 6: Test Centres

The Opposite of Cheating by Tricia Bertrand Gallant presents a different set of perspectives. She and fellow author David Rettinger, flip the argument and start, not from the institutional, but student perspective.

She argues, based on flipping the debate towards students’ needs, that testing should be the responsibility of separate and shared ‘Test Centres’. These would provide assessment expertise and the ability to design and manage the delivery of assessments. This removes the pressure on faculty, who, on the whole, do not have the necessary expertise on assessment or its delivery. These centres would look at automating as much as possible, while being careful about verification and standards. It is clear that as AI improves, and it is at a blistering pace, so the automation of assessment and marking will become easier, better and cheaper.

Assessment is a rapidly evolving problem that needs this rapid and adaptive response. This is not a final solution as such, but a new approach to a growing problem that focuses expertise, while relieving the system of the unbearable pressure of producing and policing assessment. Teachers are not cops.

The question is whether this should be a single, clustered or national initiative. Huge savings would be possible if it were organised by the sector nationally. This is unlikely, as there is no real legal or political mechanisms for such a strategic approach. Tertiary education institutions are not known for their sharing, so even clusters are unlikely. That does not invalidate the strengt hof the idea.

These test centres could be physical but more sensibly virtual. This would allow testing at any time, on any subject. It is bizarre that one can only get tested on one day of the year, resits often not available for months, even a year later. Imagine of this were true of driving? A Test Centre should allow testing on demand. 

Conclusion

Oddly, the AI prwessure is forcing tertiary education to rethink, and reassess its own assessment. This is long overdue. It is acting as a catalyst for reshaping the role of the teacher and learner in relation to technology, accepting that AI is here to stay. The alternative is to boil like the proverbial frog, fail to respond with anything other than a policy document, constantly accusing students and/or institution for failing to properly assess learners. This is the road to ruin and regret, not the road to success.

Bibliography

Hartmann, C. (2025). Oral exams for a generative AI world: Managing concerns and logistics for undergraduate humanities instruction. College Teaching.

All of the Above – 20 ways to cheat MULTIPLE CHOICE questions

Many multiple choice questions are poorly written. What better way to expose these errors than write a cheat-sheet for learners?

Of course, writing good test items is far more difficult than many imagine. Many make obvious mistakes. An interesting way to coming at this problem is to do some reverse engineering. If you think this doesn’t work, think again. Poundstone number crunched 100 tests with a total of 2456 questions to get some of these statistical biases.

Second-guessing the test designer

So here goes with my 20 ways to cheat Multiple Choice tests:

1.  Skip the hard questions, mark them with a cross, and go back to them. This means you’ll not lose marks for unanswered easy questions.

2.  Cover the options and try to answer. Prevents being misled by clever wrong options.

3.  If in doubt choose ‘B’, poor questions designers do not truly randomise the right options and have a bias towards ‘B’. Next best is ‘C’.

4.  If in doubt choose the ‘longest option’. Question designers often cannot make a right option any shorter, but have complete freedom with wrong options. This is quite common.

5.  If in doubt choose TRUE, in true/false questions, as they come easier to mind for designers.

6.  Reverse answers. Statistically, there is more T/F alteration in tests than in truly randomized sequences, as the brain struggles to randomise properly. So, if you’re sure you’ve got one right, reverse the next answer.

7.  Eliminate the outlier. Look for similarities in options and eliminate outliers e.g. 4p-q, 2p+q, 4p+q, 3p+q. Look for these internal patterns.

8.  If two options are opposites, one is likely to be correct. Designers tend to create the opposite of the right answer as their first distractor, so one of them is likely to be correct.

9.  Favour options with careful qualifiers, such as ‘sometimes, occasionally etc.’ as tested knowledge usually has more finite than absolute qualities.

10.  Be wary of options with absolute qualifiers, such as ‘always, never etc’. As these are often too definite to be correct.

11.  Choose a middle order option i.e. out of 100, 150. 200, 250, choose 150 or 200. Designers tend to have a bias, where right answers tend to be lower than the highest and higher than the lowest option.

12.  For questions that demand an ‘except’ or ‘not’, mark each option with a T for true and F for false against each option. And underline the word ‘not’ as it’s sometimes missed.

13.  ‘All of the above’ and ‘None of the above’ are both significantly likely to be correct. For it to be correct, the writer has to design options that were all correct, so, if you can’t spot any wrong answers, or see that two or more are correct, it increases the probability of ‘All of the above’ being correct. Similarly with ‘None of the above’.

14.  Typo or punctuation error, the option is likely to be wrong. Writers tend to proofread correct answers only.

15.  Look for grammatical agreement between the question and its options; ‘An.....’ and words starting with vowels or agreement between subject, object or verb.

16.  If you’re stuck, go with the ‘Least bad rule’. Eliminate least likely answers first.

17.  Look for clues about answers from other questions. Designers often, unintentionally, put clues, even answers, to questions in other questions.

18.  Ignore never heard of answers. If you’ve never heard of the answer, it’s likely to be made up and incorrect.

19.  Go with your first impression. The more you read, the more you tend to read into the wrong options.

20.  Always guess, unless there is a penalty. It’s a 1 in 4 chance, so don’t give it up.

Conclusion

This crib sheet can be used by students or question designers to improve their tests. 

Next post - how to write a great MC question...
And there's 500 more tips like these, all categorised, here.

10 DOs & DON’Ts in MOBILE LEARNING - Game of Phones

Mobile learning is confusing. In theory it sounds great, in practice it’s often misattributed hype. Different devices have different patterns of use. The fact that you make ‘responsive’ e-learning simply means that it can be delivered on different devices NOT that it will be used on different devices. M-learning is, therefore, often more fiction than fact.

1. DON’T expect people to play tennis in a cupboard

People don’t do long e-learning courses on mobile phones. It’s a device for short, episodic activity, not long, deep, reflective learning experiences. Large e-learning courses on mobiles are a waste of time. It’s like playing tennis in a cupboard.

2. DO use for informal learning

Formal stuff is hard on mobiles, so focus on informal, such as fast facts, flashcards, quick quizzes, comms and support for students, social media and so on. People use mobiles informally, so deliver informal learning. 
3. DO think about social media
Mobiles have become the primary means for accessing social media. Twitter has become a great form of CPD for all sorts of professionals. Indeed, all forms of social media have some relevance in learning. So thing about this aspect of informal learning.

4. DO use for projects

Long used for gathering material and evidence for assessment in vocational learning. Mobiles are great at gathering data, whether notes, images, audio or video. It gives impetus to learning by doing.

5. DO use for performance support

OK, you’re stuck and only have a mobile to hand. That’s when you need the right answer to a question or solution to a problem. This is short, sharp and useful. Learning at the point of need.

6. DO use for spaced practice

The most ignored piece of theory in the psychology of learning is the Ebbinghaus forgetting curve – we forget most of what people suppose we learn – fact. The solution is spaced practice and mobiles are powerful, portable and personal devices in our pockets, so we can deliver spaced practice by delivering cues from any learning experience across a defined time.
7. DO use for assisted learning
Do use for useful apps that assist learning. A good exampel is the AI app, Photomaths. Teachers hate it, learners love it. You point your camera at the maths problem and it gives you the answer. Not only that, it gives you the breakdown of the steps between the question and answer. That's useful. 

8. DO think about context
One of the advantages of mobile is the fact that it is personal, powerul and portable. Think about recommending its use in context, In the store in retail, in the ward in medicine and so on. With the Internet of Things we may even see Beacon driven learning, where learning is triggered by beacons in the workplace.
9. DON’T expect people to pay

Yes, non-wifi use costs money. Don’t expect learners to pay unless they have agreed to this approach. This sounds obvious but it’s a fact that is too often forgotten in designing m-learning.

10. DON’T cheat on mobile metrics

First, there’s confusion about what ‘mobile’ devices are, a confusion that is confusing the hell out of everyone. When I say “he’s using a mobile”, I don’t mean he’s using a ‘tablet’. Otherwise, I’d say “he’s using a tablet”.  Yet people are reporting mobile use as phones plus tablets. If the answer is, tablets are taken around by people and used as second screens, then those two criteria also apply to my laptop. This is sleight of hand. Don’t cheat on mobile metrics

Conclusion

In the same way that tablets have been hyped in schools, as they are limited in terms of complex learning tasks such as long-form writing, coding, tools etc., mobiles are hyped in formal learning. Don’t treat all devices as delivery devices. Different devices have different learning attributes.