Tuesday, May 21, 2013

Report on 6 MOOCs turns up 10 surprises

Great report from the University of Edinburgh on their six 2013 Coursera MOOCs. The report has good data, tries to separate out active learners from window shoppers and not short on surprises. It’s a rich resource and a follow up report is promised. Well done Edinburgh – this is in the true spirit of HE – open, transparent and looking to innovate and improve.
Six courses
Introduction to Philosophy: 98,129
Critical Thinking: 75,884
E-learning & Digital Cultures: 42,844
Astrobiology: 39,556
AI Planning: 29,894
Equine Nutrition: 23,332
Ten surprises
Rather than summarise the report, I’ve plucked out the Top Ten surprises, that point towards the future development of MOOCs:
1. Large no of enrolments (309,628)
2. Age spread
3. Huge subject-sensitive gender range (13-87%)
4. Low no. students/ in teaching & education (36.8%)
5. Learners from 176 countries (61% outside US/UK)
6. Close to zero from China
7. Main driver – learning, low interest in certification
8. Sparse use of Forums
9. Big range on SoA across courses (4-44%)
10. Expectations – met more or completely (77%)
1. Large no of enrolments (309,628)
Good numbers but the report wisely points towards a large number of ‘window shoppers’. This is a consequence of being early and they expect numbers to fall with a change towards more serious and sustainable ‘learners’ in the future. However, it points towards massive, unmet demand for for MOOCs.
2. Age spread
Given the level of the courses, it is clear that a wide range of ages want MOOCs. The standard ’18 year-old undergraduate’ profile is blown out of the water with MOOCs. Only 1 in 5 fit this 18-24 model. 

3. Huge subject-sensitive gender range (13-87%
One of the great ‘elephant in the room’ issues in HE is the gender imbalance. Different courses have incredible imbalances. This is also reflected in MOOCs. It would be interesting to collate gender data against preferred use of Forums and social media.

4. Less than third students or people in teaching and education. Does this show that MOOC demand does not fit the traditional ‘undergraduate model’? The data here is skewed by the ‘E-learning and Digital Cultures’ course where 51% were in teaching and learning. When you strip this out, MOOCs are certainly open (in spirit) and therefore attract diverse audiences.

5. Learners from 176 countries (61% outside US/UK)
39% from US and UK, but that’s where it was publicised. If anything the real surprise is that the other 61% is from the rest of the world. The broad global pull for MOOCs is clear.

6. Close to zero across courses from China. This casts doubt about MOOCs attracting that lucrative foreign student market.
7. Main driver – learning, low interest in certification
This is a lesson that many MOOC commentators are learning, that MOOCs reflect, not demand for certification but demand for ‘learning’ with only around a third interested in certification or career. . That’s not to say that certification is not important, it’s just less important than educators think. Curiosity about online education and MOOCs, however, is the temporary pollutant in the data.

8. Low Forum participation. This confirms my view that Forums are useful but overegged. On technical courses, I’ve experienced very low use of Forums and discussions, mostly around alleged mistakes by the academics in definitions and the maths. Even in highly ‘discursive’ courses, like Philosophy and the ‘curated’ E-learning course the numbers are relatively small. One of the most interesting data sets is that on the use of forums and social media. “The respondents of the Exit survey were more independent than social learners, with high self-reported time spent on videos and quizzes and less on online social activities.” It is assumed, by social constructivists, that people are desperate for this type of interaction and social learning experience, but many don’t seem to participate.

9. Big range on SoA across courses (4-44%)
Surprised that the E-learning course was so low but, having taken this course, I think it raises some interesting questions about quality and structure. I, and others, found the content a little weak and, although it’s a subject I’m passionate about, it didn’t do it for me.

10. Expectations – met more or completely (77%)
These figures are extremely promising with 77% feeling very good about their experience and 98% seeing MOOCs as having to some extent, exceeded or completely met expectations. Given that this was the first experiment with MOOCs, I’m impressed.

Impressive report, full of fascinating facts and figures. If I were looking at MOOCs, I’d pour over this data carefully. That, combined with the useful information on resources expended by the University, is an invaluable business planning tool. In my next post, I’ll look at the way Edinburgh planned and coped with governance on this initiative - equally fascinating.
Download University of Edinburgh report here.


Dave Ferguson said...

I've participated in, though never completed, a couple of MOOCs, including one connectivist-style (Siemens/Downes) and one offered through MIT. In terms of forum participation, I wonder if one of the reasons for low participation isn't simply the law of diminishing returns.

Here's what I mean: whether in a traditional threaded format, or in Google Plus, it seems to me that beyond 20 or 25 active participants, the conversations range too broadly -- there's a lower signal-to-noise ratio. In addition, latecomers hop in and ask questions that have been asked before, rather than reviewing what's already been said or doing a little searching first.

Granted, the forums I've seen have pretty poor search features--enter a phrase and get a bunch of hits without much context.

I mean these comments as a description and not an indictment. Particularly with the MIT MOOC, I applied my prior experience and made more deliberate choices about what subgroups I'd join and how I'd interact with people.

In my case, that tracks with the remark about preferring independent over online-social: exchanges within a small group, and one-to-one conversations with members of that small group, were more to my taste than the at-large stuff.

Still, I did post some comments or suggestions to the group as a whole, both because I thought many might find these helpful, and to do the kind of sharing I think is worthwhile in a MOOC.

Donald Clark said...

Dave - all of this chimes with me, especially the law of diminishing returns, poor search. There needs to be more sophistication in the Forum architecture. My own view is that social media does this better, where you can form strong/weak ties or observe a Twitter stream and have one-to-one comments on posts. Local meetups also useful. There have been several Coursera meetups in my own town, Brighton. Forums seem a little old school. Interesting reflections, as always.

Unknown said...

I find it enlightening that enrollment in these MOOCs was not primarily for certification purposes, but for learning. I wish educational leaders at the state levels would realize this fact when requiring certification hours and mandating educators to teach "to the test" and ultimately use students' scores for teacher evaluation purposes. Instead, we should be promoting a passion and a love for learning among students. Students should be able to learn the way they learn best - and schools should not have to dread the "forty days of testing" during a single school year.

Anonymous said...

Teachers won't take them (even if they learn something) until they count as a credit towards their pay scale.

Edie said...

This is the first of your posts that I've read... I'd find it very helpful if you would spell out acronyms. What is HE? SoA?

Donald Clark said...

Statement of Achievement

Helen said...

Thanks Donald, some surprising (and not so surprising) data.

I've done one xMOOC through Coursera and will be doing another later this year. I've also done 2 cMOOCs (fully), one in progress and 2 I'm lurking in actively (if there's such a thing as actively lurking - reading the posts, responding but not actually doing any of the suggested work).

I'm also a MOOC drop out for two Critical Thinking and Reasoning (hope this isn't saying anything about me) and eLearning and Digital Cultures (I had to stand on a chair to catch the reading material flying over my head - but the videos were great), and another one involuntarily (the debacle of the Fundamentals of Online Learning MOOC of April 2013).

I agree with Dave's comments and from a personal development point of view, they've been great for establishing some networks that continue through Twitter as well as picking up some great tools. Funnily enough, I never use their discussion forums because I find them too unwieldy and get lost in them.

I know they're a contentious subject at the moment with academic institutions but for some free learning, access to new networks - they are well worth it for a self directed learner like myself. I think I get more enjoyment from finding people who are doing the same MOOC and talking about it - there's a new social connection....

Mike said...

I think that it is interesting when people find that the discussion forum are unfulfilling or simply unwieldy. There seems to be an issue with providing the kind of "learning community" that exists in conventional class situations. In my case. some of the deepest discussions I have had pursuant to course work have occured in out-of-class informal group settings. Creating an exact equivalent of this milieu is an inherent difficulty in the MOOC concept.

Donald Clark said...

This has always also been a problem on campus. On MOOCs a number of alternatives have arisen as an alternative to unwieldy forums:
Google Hangouts
I have much wider and deeper access to groups than I ever had at University. Witness this useful dialogue.

Parke Muth said...

I have recently posted a piece on my experience with the MOOC highlighted in The New Yorker last week. Professor Nagy was kind enough to respond positively to my remarks about how MOOCs have the potential to return us to the origins of community and shared knowledge.


Unknown said...

Donald I think the forums are very usefull. Only a few people make posts, but many read them and help to resolve the quizes and assingnments. Is a good tool to more advances students and TAs share their knowledge specially in solving problems. Note that in a MOOC there are no practical classes and if someone can´t solve a problem himself there is nobody to ask.

e-purser said...

Interesting post Donald, thanks. I wouldn't generalise too much from it, but my own take on MOOC forums has been that compared to other options for discussion, they kind of suck... but they'd be ok if there was nothing better around of course. A few of us wrote up our experience of using social media and inter-connected blogs just after we did one of the Edinburgh Moocs (which was incidentally pretty much full of teachers actually, being a course about what digital culture means for education) - if you're interested, here's how we summarised experiences.: http://elearningeuropa.info/en/article/Realising-the-Potential-of-Peer-to-Peer-Learning%3A-Taming-a-MOOC-with-Social-Media and then specifically about blogging: http://elearningeuropa.info/en/article/Quad-blogging%3A-Promoting-Peer-to--Peer-Learning-in-a-MOOC

Leslie B. said...

I'm in a MOOC to find out what the experience is like. In mine, the professor is not an expert in field -- if he were teaching in my region, the accreditation people would not let him give this course as he does not have 18 graduate hours (he has BA, MA, and PhD but in other fields).

Result: the videos, quizzes, and discussion questions are useless, but these are required. If the discussion questions were good ones, the fora would be very interesting. The fora in this course that *are* interesting are the non-required ones. They are interesting because in them, people are discussing the problems with the course, how to manage them, how to get something out of the materials despite the situation. I have learned from these fora. Enterprising people have also created Facebook pages for the course and there, they are having useful discussions and sharing useful material.

Ideaon INC said...

The 10 surprises report is nice and thanks for sharing.