MOOCs: course completion is wrong measure

In a fascinating survey taken at the start of the University of Derby’s ‘Dementia’ MOOC, using Canvas, where 775 learners were asked whether they expected to fully engage with the course, 477 said yes but 258 stated that they did NOT INTEND TO COMPLETE. This showed that people come to MOOCs with different intentions. In fact, around 35% of both groups completed, a much higher level of completion that the vast majority of MOOCs. They bucked the trend.

Now much is made of dropout rates in MOOCs, yet the debate is usually flawed. It is a category mistake to describe people who stop at some point in a MOOC as ‘dropouts’. This is the language of institutions. People drop out of institutions -  ‘University dropouts,’ not open, free and online experiences. I’m just amazed that 40 million have dropped in.

So let’s go back to that ‘Dementia’ MOOC, where 26.29% of enrolees never actually did anything in the course. These are the window-shoppers and false starters. False starters are common in the consumer learning market. For example, the majority of those who buy language courses, never complete much more than a small portion of the course. And in MOOCs, many simply have a look, often just curious, others want a brief taster, just an introduction to the subject, or just some familiarity with the topic, and further in, many find the level inappropriate or, because they are NOT 18 year old undergraduates, find that life (job, kids etc.) make them too busy to continue. For these reasons, many, myself included, have long argued that course completion is NOT the way to judge a MOOC (Clark D. 2013, Ho A. et al, 2014; Hayes, 2015).

Course completion may make sense when you have paid up front for your University course and made a huge investment in terms of money, effort, moving to a new location and so on. In open, free and online courses there is no such commitment, risks and investments. The team at Derby argue for a different approach to the measurement of the impact of MOOCs, based not on completion but meaningful learning. This recognises that the diverse audience want and get different things from a MOOC and that this has to be recognised. MOOCs are not single long-haul flights, they are more like train journeys where some people want to get to the end of the line but most people get on and off along the way.

Audience age

Here’s two sets of data from the Derby Dementia MOOC and the six Coursera MOOCs delivered by the University of Edinburgh. It is clear that MOOCs attract a much older audience than the average campus student.

This is important, as older learners are far less likely to want pieces of paper and certification or bother that much about not completing the full diet of content.

Audience mix

We are also seeing a drift away from the initial graduate only audience. There is still a skew towards graduates but this is because these are the early adopters and almost the only group who know that MOOCs exist. Only now, do we see some serious marketing, targeted at different audiences and this is starting to have effect. Indeed, the majority of participants (55%) in the Dementia MOOC are not University graduates.

Audience motivation

Now here’s an interesting thing.  A point often forgotten in MOOCs -  learner motivation.

This compares well with the Edinburgh data.

The bottom line is that people who do MOOCs really want to learn. They are not largely motivated by pieces of paper or even completion.

Conclusion

As MOOC audiences are different from traditional HE students and as their audiences change in terms of age, background and motivation, the more likely MOOCs will have to respond to these new audiences and not mimic University semester courses. The team at Derby have already suggested an alternative set of metrics for measuring the success of a MOOC. They’re right. It’s time to move beyond the boring. Repetitive questions we hear every time the word MOOC is mentioned – dropout, graduates only…..

Bibliography

Hadi, S. Gagen P. New model for measuring MOOCs completion rates. Presentation at European MOOCs Stakeholder Summit.
You can enrol for the University of Derby 'Dementia' MOOC here.
And more MOOC stuff here.