Report on 6 MOOCs turns up 10 surprises

Great report from the University of Edinburgh on their six 2013 Coursera MOOCs. The report has good data, tries to separate out active learners from window shoppers and not short on surprises. It’s a rich resource and a follow up report is promised. Well done Edinburgh – this is in the true spirit of HE – open, transparent and looking to innovate and improve.

Six courses

Introduction to Philosophy: 98,129

Critical Thinking: 75,884

E-learning & Digital Cultures: 42,844

Astrobiology: 39,556

AI Planning: 29,894

Equine Nutrition: 23,332

Ten surprises

Rather than summarise the report, I’ve plucked out the Top Ten surprises, that point towards the future development of MOOCs:

1. Large no of enrolments (309,628)

2. Age spread

3. Huge subject-sensitive gender range (13-87%)

4. Low no. students/ in teaching & education (36.8%)

5. Learners from 176 countries (61% outside US/UK)

6. Close to zero from China

7. Main driver – learning, low interest in certification

8. Sparse use of Forums

9. Big range on SoA across courses (4-44%)

10. Expectations – met more or completely (77%)

1. Large no of enrolments (309,628)

Good numbers but the report wisely points towards a large number of ‘window shoppers’. This is a consequence of being early and they expect numbers to fall with a change towards more serious and sustainable ‘learners’ in the future. However, it points towards massive, unmet demand for for MOOCs.

2. Age spread

Given the level of the courses, it is clear that a wide range of ages want MOOCs. The standard ’18 year-old undergraduate’ profile is blown out of the water with MOOCs. Only 1 in 5 fit this 18-24 model. 

3. Huge subject-sensitive gender range (13-87%

One of the great ‘elephant in the room’ issues in HE is the gender imbalance. Different courses have incredible imbalances. This is also reflected in MOOCs. It would be interesting to collate gender data against preferred use of Forums and social media.

4. Less than third students or people in teaching and education. Does this show that MOOC demand does not fit the traditional ‘undergraduate model’? The data here is skewed by the ‘E-learning and Digital Cultures’ course where 51% were in teaching and learning. When you strip this out, MOOCs are certainly open (in spirit) and therefore attract diverse audiences.

5. Learners from 176 countries (61% outside US/UK)

39% from US and UK, but that’s where it was publicised. If anything the real surprise is that the other 61% is from the rest of the world. The broad global pull for MOOCs is clear.

6. Close to zero across courses from China. This casts doubt about MOOCs attracting that lucrative foreign student market.

7. Main driver – learning, low interest in certification

This is a lesson that many MOOC commentators are learning, that MOOCs reflect, not demand for certification but demand for ‘learning’ with only around a third interested in certification or career. . That’s not to say that certification is not important, it’s just less important than educators think. Curiosity about online education and MOOCs, however, is the temporary pollutant in the data.

8. Low Forum participation. This confirms my view that Forums are useful but overegged. On technical courses, I’ve experienced very low use of Forums and discussions, mostly around alleged mistakes by the academics in definitions and the maths. Even in highly ‘discursive’ courses, like Philosophy and the ‘curated’ E-learning course the numbers are relatively small. One of the most interesting data sets is that on the use of forums and social media. “The respondents of the Exit survey were more independent than social learners, with high self-reported time spent on videos and quizzes and less on online social activities.” It is assumed, by social constructivists, that people are desperate for this type of interaction and social learning experience, but many don’t seem to participate.

9. Big range on SoA across courses (4-44%)

Surprised that the E-learning course was so low but, having taken this course, I think it raises some interesting questions about quality and structure. I, and others, found the content a little weak and, although it’s a subject I’m passionate about, it didn’t do it for me.

10. Expectations – met more or completely (77%)

These figures are extremely promising with 77% feeling very good about their experience and 98% seeing MOOCs as having to some extent, exceeded or completely met expectations. Given that this was the first experiment with MOOCs, I’m impressed.

Conclusion

Impressive report, full of fascinating facts and figures. If I were looking at MOOCs, I’d pour over this data carefully. That, combined with the useful information on resources expended by the University, is an invaluable business planning tool. In my next post, I’ll look at the way Edinburgh planned and coped with governance on this initiative - equally fascinating.
Download University of Edinburgh report here.

Futurelearn MOOC: 10 Qestions: When, who, what, cost, funding, courses, look, pedagogy?

Paid top dollar to attend a MOOC conference by UK Universities in which Martin Bean (Vice Chancellor of Open University) and Simon Nelson (CEO of Futurelearn) were to give talks. Although the conference was about MOOCs and online learning, most of Simon’s talk was about the BBC. That was fine but not entirely relevant. However, his other public interview on Futurelearn are not short of ambition:

“In three years’ time we hope to be offering a level of online learning that we can’t dream about at the moment” says Simon, “It may sound ridiculous in ambition, but one of my team said to me that in five or 10 years, rather than hanging out on Facebook of an evening, people will feel they can hang around in the Futurelearn product.” I rather like this crazy level of optimism but it needs a sense check. So I asked him some questions….

1. When’s it coming?

The aim, Simon says, is “to have products on the web by the middle of 2013, and a full consumer launch sometime in the autumn”. So we can expect something soon. The question is whether this is too little, too late. He made a great deal of the ‘virtue in coming second’ but we had no choice in the matter. Also, as far from being second, Futurelearn is not even in the first ten – Coursera, Udacity, edX, Udemy, NovoED, Openlearning, Google, OpenupStudy, Open2learn, Iversity, Desire2learn…the list goes on. They may, by launch, not even be in the top twenty. Nevertheless, there may still be wisdom in taking your time and getting it right, a point well made by Simon.

2. Who are the partners?

“British Council, British Museum and British Library.” You clearly need to have ‘British’ in your name to be a partner. British Council could be useful in foreign marketing, and the other two as sources of assets, but none have any business or entrepreneurial spirit. There’s been lots of ‘digital’ initiatives with these partners in the past, lots of ‘digitise this’ and ‘digitise that’ projects, but nothing that’s succeeded in the learning space. None of the players have built a business from scratch. That’s a worry.

3. What about the past failures in this space?

I mentioned UKeU and BBC Jam but could have included the OU’s disastrous expedition to the US, the NHSU and others. To be fair, Simon probably has no knowledge of any of this. He has no experience in education or online learning, which could be a blessing (or a curse). The reason I mentioned these, was that we must not repeat the mistakes of the past. These included the wrong leadership (lacked business and start-up experience), wrong partners (old school), bad technology decisions (idiosyncratic), late delivery and poor, sometimes over-produced, content (wrong developers). One thing I hope will not happen is a repeat of the BBC Jam experience, where experienced, online learning providers were ignored in favour of the so-called ‘digital’ talent of the day. I don't think this should dominate the debate as we need to move on but we do need to acknowledge the researched reasons for their collapse.

4. Does Futurelearn have enough ‘entrepreneurial kick’?

“Time will tell”. Honest answer but this is my greatest fear. None of the partners have ANY track record in real entrepreneurial enterprises. In fact, they are all publically funded organisations. This worries me. Why didn’t we open this up to the companies and experts in the UK who know about innovative pedagogy and MOOCs? We had the knowledge, expertise and software to do this, not from a standing start but from a position of strength. What we needed was not a single throw of the dice but a spread bet to develop the market. Futurelearn has switched the market off in the UK for at least a year. That hasn’t deterred some UK companies. One I know is already selling successfully direct to the US. One other large Maths MOOC is being funded by a charity (watch this space). But monoculture in an emerging market is not often a good idea. Then again, if the product is world beating and UK Universities get behind it, it could just work. I just wonder how they got past OJEU rules?

5. How much is the funding?

“You’ll have to ask my Chairman (Martin Bean).” I know how much funding went into Udacity ($21m), Coursera ($22m), edX ($60m). Why the secrecy here?

6. Where has the money come from?

“Open University, other than that I can’t say. You’ll have to ask my Chairman.” In the absence of an answer, my guess is that Willets has done a deal in terms of Government money, channelled through the Open University and that guaranteed service revenues will have to come from the 21 participating Universities. This makes sense but why the secrecy? This is the ‘Future’ of learning, not the Freemasons – and it’s public money. Again, it’s not such a big deal but if I were a potential customer, I’d want to know how the supplier was capitalised.

7. As the Open University is a publically funded institution, should there be more transparency?

“You’ll have to ask my Chairman.” This was getting awkward. You’ll have got the idea by now that this is BBC speak for ‘no comment’.

8. What courses?

Courses will take 6-10 weeks. Southampton wants to offer Web Science and Oceanography in its first set of courses but there’s little in the public domain on this. My guess is that they’ll obviously avoid competition through duplication and play to the strengths of each institution. There’s some wisdom in having a spread of courses available.

9. What does it look like?

“I showed a couple of screen shots and will put it up on Slideshare.” Thougt he wouldn’t and he didn’t (come on Simon). To be fair, he did show a couple of screens; simple, white background, scrolling with centred text at the top, lots of block text and a picture of the Moon at the top. They looked awful, but this is just prototype stuff, so let’s wait on the real deal. I’d have thought however, that as we’re weeks away from some releases, there’d be some good ‘taster’ content.

10. What’s the pedagogic structure?

Simon hinted at their MOOCs being made up of “Units and Learning Blocks”. A Unit is 2-6 hours of learning time with a clear end goal and assessment. Within each Unit will be a number of Learning Blocks. A block seems to be [video, text, discussion, test]. Nothing radical there but that’s OK – a clear, simple structure is fine. OER content will be used but only when evaluated as relevant and of sufficient quality. I may be wrong here, but reading between the lines he does seem to be moving to a flexible approach where you can choose different approaches. This would be interesting as it would move us beyond both the linear lecture approach of Udacity, Coursera and edX approach without descending into the fragmented mess of extreme social constructivism.

Conclusion

In the US, there’s a healthy ecosystem of entrepreneurial Universities such as Stanford, MIT and Harvard, along with focused not-for-profit organisations such as the Gates Foundation and others, along with knowledgeable investment capital. They came up with the goods. This ecosystem, I believe, gives it an entrepreneurial edge which we ignore at our peril. Nevertheless, Futurelearn may come up with a different set of goods. Simon seemed like a competent guy and Martin Bean’s a good leader, and no fool. The danger is dishing up the usual British solution that lacks edge and commercial push. We need this to work. I just hope to God it’s not another BBC Jam – all fur coat and no knickers.

MOOC on Human-Computer Interaction: videos have 7 fails in HCI

 

Enjoying Stanford’s Coursera MOOC on Human-Computer Interaction but (oh the irony) the screen design and pedagogy of the many videos, is awful. Don’t get me wrong, it’s easy to use, has good content and I’m getting what I thought I’d get - a reasonable course. It is very video heavy, which is OK; let’s face it most HE courses are lecture heavy - at least they’re not an hour long, I can watch these when I want, repeat them and, in Coursera fashion, I get a bit of formative assessment during the videos, something students rarely get in real institutions. But it could have been so much better.

1. Small screen, low retention.

Now I don’t mean to be picky but having a tiny talking head (see above), literally less than 4.5% the size of the screen on the bottom right, is a BAD idea. Nass & Reeves (ironically from Stanford) did a great experiment with video at different screen sizes, showing that the smaller the screen size the lower the retention. Read the research guys.

2. Too much talking head.

It’s dull, the delivery is often poor and poorly edited (i.e. padded out). It’s like watching a very long news item read on one news story, time after time. Even worse is the fact that it’s a medium shot, showing the whole upper body. Go on a video course guys, you need to see the white of their eyes. Use close ups. To be fair there’s more images, graphs and screens with audio only as the course progresses. Think about ‘attention’ guys.

3. Cognitive dissonance

The too much talking head error is compounded by presenting text headings and blocks of text in a huge font over the rest of the screen.Mayer & Clark’s research showed that you don’t show text and video at same time, as you have to hop back and forth from visual (reading - semantic) to audio channels. Even worse, Scott the presenter reads the text but it’s not the text that appears on the screen. Also, the framing of the video, with text cut in half behind the presenter is cognitive noise we can do without. Watch some TV guys.

4. Paucity of images

What’s odd is the fact that when schemas or techniques or procedures are being described there’s often no images shown. This is like PowerPoint without any pictures, just big headings. In many places the point, event or procedure would have been better served by cutting away to what was being described or relevant images. It gets better in some places. Very strange.

5. Presentation style

As it’s often a little dull, I found my attention tended to drift. I can read faster than the presenter speaks, and when in the first video he started looking down and reading points one by one, the video producer in me rebelled. I get impatient with slow, amateurish delivery, which is why I like the edX and Coursera x1.5 speed feature. In fact, so much of this sounds like the reading of written material that it could have been text. Read something on relevant media mix guys. I like Scott Klemmer, but he’s no presenter, and after a while his excessive hand gestures and delivery style start to grate. This can be a problem for the single academic courses - it's like watching a ten week news programme with only one presenter.

6. Poor editing

The in-video questions are poorly edited in, so you often get a snippet of a sentence from the next sequence. Small point but it makes the production seem a little amateurish. Edit it properly guys. Again Nass & Reeves showed that these unexpected and awkward pauses and edits lower retention.

7. Poor question design

In-video questions are made progressively easier and meaningless. This is learning design at its worst.  The same question is posed, with the same options, up to four times. So when you’ve answered 1 out of 4, the next question is 1 out of 3, the next 1 out of 2 and the last a meaningless 1/1. Even worse, is the cardinal sin of two options actually being correct but only one accepted. All of this is bizarre and lazy. Read something on test items guys.

Not all Coursera MOOCs are so poor on video. The University of Edinburgh MOOC on E-learning and Digital Cultures (one of 6 MOOCs attracting 306,000 starters), which ran at the start of the year, didn’t do talking heads, relying on curated video. This caused some consternation with students who expected lectures. This course had much more of a looser structure with discussion, Google Hangouts and social media, none of it moderated by academics. Interestingly, I liked this Edinburgh course less, as I thought it was weak on focus, depth and content. There’s a balance to be struck here and much to do on improving the pedagogy and design of MOOCs. I don't buy the cMOOC/xMOOC thing - it's a simplistic dualism. There a whole variety of pedagogies that lie between straight instruction and social constructivism (I like neither in their pure form).

Conclusion

What Coursera should have done is do what this course recommends, apply the usability test strategies that Krug, Norman and Nielsen recommend. Get in the experts and do ‘voiced trials'. I have spent nearly 30 years designing and producing online learning and would never have got a client to pay for these courses. To be fair, compared to the benchmark of dull one hour long lectures, it’s an improvement and it’s a start. This is a constructive critique of the videos and I must remind you that the course is rich in assignments and practical work. Let’s hope they get some HCI professionals in to make it a little more usable and ‘learner and learning’ friendly. It’s not as if people haven’t done this before.

Education Debate: Bun fight with Baker, Benn, rosen & Gibb

A hostile and baying audience, of largely teachers, Nick Gibb (ex Minister for Schools), Kenneth Baker (Thatcherite turned vocational learning evangelist), Melissa Benn (critic of Govism) and Michael Rosen (Gove’s bête noir). My friend Mathew Clayton organised the debate and we spoke about the need to get the juices flowing, by finding adversaries, he did, and it worked.

Nick Gibb wants facts and more facts

Nick Gibb was first out of the blue corner, with the usual statistics around failure – large numbers of children not achieving 5 reasonable GCSEs and low levels of numeracy and literacy. He was proud that he had freed schools from the stranglehold of the Local Authorities and then brought in that weasel word ‘competition’ which he thinks will sort it all out. He felt he had sharpened OFSTED, raised the bar on teacher entry qualifications and slimmed down the primary curriculum, as well as reintroducing a more knowledge-based curriculum. Oh and he loves Latin. To be fair I did agree with his final point about the teaching profession, when left to its own devices, managed to wreck literacy for two decades or more through the use of ‘look and say’ literacy teaching. I witnessed this ‘whole word’ madness in the primary school my own kids attended in the late nineties. The move to phonics is good. However, he made a serious gaff, that Rosen picked up on later….

Melissa Benn wants reform without rancour

She objected to the ‘tone’ of the reform and would prefer ‘reform without rancour’. Fair enough, but this is England, where parents would eat other parents livers to get their kids into the 'right' school and where the teaching profession has never been short of delivering bouts of its own rancour. That was obviously alive and kicking at this event. She did, however, have a good point about ‘demoralisation’. It’s one thing to criticise, another to kick an entire profession in the teeth. Where she came into her own was on the evidence. Here, she thought, the policies were fraudulent. Academies are not better, indeed often worse. Curriculum changes idiosyncratic and regressive, and free schools downright dangerous. She feared a return to the Secondary modern versus Grammar schools and her final recommendation was the Finnish system, a low test, comprehensive system that produces world leading results.

Kenneth Baker wants vocational provision

Baker has turned out to be a  bit of a rebel in his dotage. He sounded more like a Trade Union leader than Thatcherite. Pupils will stay on until 17 this year and 18 next year. We haven’t grasped the consequences of this, he claims. We need to reset the break point to 14, not 16 and introduce vocational paths, if we are to succeed as a country. He’s right – school leaving exams at 16 make no sense. Read his book. It’s not half bad. In it he recommends four types of colleges, schools, and academies:

1. Liberal Arts Colleges for academic studies;

2. University Technical Colleges (UTCs);

3. Career Colleges for practical, vocational subjects;

4. Sports, Creative and Performing Arts Colleges.

He thinks this will create a coherent range of routes leading to university, apprenticeships and employment. The problem is parity. As long as we refuse to acknowledge parity of academic and vocational qualifications, these will fail. However, to give him his due, at least he has some ideas around vocational learning. He was attacked by the audience for daring to mention ‘empoyability’. But he’s right. Education is not ALL about employability but education and teaching have long ignored its importance. Germany had copied our system after the war and flourished. Blair, he thinks, put a spanner in the works by killing off the Tomlinson recommendations – again he’s right. Interestingly, he was also against the creation of any new faith schools – good man.

Michael Rosen wants pedagogy not demagogy

Michael put the boot into Nick Gibb, when Nick recommended the new SPAG tests at 11, something introduced in the face of all the evidence that shows that teaching grammar is a waste of time, he read back a hopelessly, ungrammatical sentence Gibb had uttered just a few minutes earlier. Rosen’s point was that language changes. Oh how we laughed! He rightly ridiculed Gove, who thinks he’s an omniscient expert on everything. Imagine a Secretary of State for health telling doctors how to diagnose and treat patients, that's waht Gove does in education. This would be fine, but as Gibb said earlier, teachers were teaching 'whole word' literacy not long ago, a technique akin to voodoo. Medicine is based on science. Education is not well evidenced –witness learning styles, Mozart effect, L/R brain theory, Brain Gym, whole word literacy and so on. All we hear about is teaching, he claimed, and little on learning. Then he stuck the knife in on selective evidence, backed up with some knowledgeable interventions from the audience.

Conclusion

I didn’t wholly agree with any of them but agreed with some of what all of them said. In fact, the most impressive speaker was Kenneth Baker, as he, at least, had a clear idea about shaping the future. He was also keen to focus, not on the top 25% but the rest. Baker was not, as Michal Rosen put it, “depressingly utilitarian” but we do need to ask whether these sort of choices should be made at 14 or whether we widen out the options later. Gibb was backward looking and depressing but right about the failure of teaching and education to really deliver on literacy, when it went off at a tangent with ‘whole word’ teaching but he was wholly misguided with his guff on Latin and SPAG tests. Melissa Benn was right to uncover the selectivity of the evidence by Gove but didn’t really seem to have much of a vision beyond copying Finland. I happen to agree with this but in class-ridden, conservative-parent Britain it’s hopelessly utopian. Michael Rosen was right to focus on learning but lacks vision. Bit of a British bun fight but that was what was needed to make us all reflect. The problem is that UK education game has too many vested interests - independent schools, faith schools, universities, unions, parents and social classes to ever sort any of this out. A national perspective around the future for our young people isn't even on the table.

MOOCs: old narratives v new narrative - open, scalable, diverse & relevant

Narratives

There’s been a lots of different reactions to MOOCs and a few fixed narratives have emerged:

1. ‘US Valley’ narrative around Khan, Stanford, not-for-profits, investors, Coursera, Udacity, NovoED and so on.

2. ‘Canadian connectivist’ narrative that MOOCs originated with Siemens & Downes and have been usurped by the ‘US valley’ folks.

3. ‘Out of OER’ narrative, where MOOCs are seen as building upon the Open Educational Resource culture.

4. ‘Traditional backlash’ narrative, that MOOCs dangerously undermine the traditional values and funding of Universities. 

5. ‘Silver bullet’ narrative where MOOCs are seen as the future saviour for higher education.

In my view, none are wholly true, yet all have a degree of truth. What we have to do is stop seeing all of these as mutually exclusive and look to the future not the past. This is a phenomenon or movement that, whatever its origins has the momentum that none of the past initiatives seemed to gather. It’s a time to drive forward with debate and discussion, not constantly checking the wing mirror.

New MOOC narrative

My own position is that we need a future-looking narrative that lies beyond all of these. Here’s a thought. MOOCs will not replace or even undermine Universities. In fact, they are likely to make our Universities even more important as the future keepers of cultural capital. No one wants to see our University system fail or crumble. Then again, many want to see aspects of the closed ‘ivory tower’ reshaped into something a little flatter, more open and accessible. There are genuine worries about insularity, quality of teaching, cost, access and relevance. If we can reposition academe as more open, transparent and relevant, that could be to the benefit of us all. There are seven components to this narrative:

1. Open

Being more open, through MOOCs, will engage and re-engage potential school leavers, parents, alumni, adult learners and the majority of people worldwide who may see it as a realistic aspiration. Just as important are those who,frankly, have no chance of ever seeing the inside of a University. The data from MOOCs already show a huge appetite from an untapped audience around the world for knowledge and learning. I suspect that academics, research and reputations of Universities would be enhanced of that knowledge were seen as more open and accessible

2. Scalability

Higher Education does face the problem of increasing costs. In most other areas of human endeavour, increased volume leads to decreased costs. Along comes a solution that promises to ease that problem. Sure the business models have yet to be refined, but they will. Sure there may be less teacher-student face-to-face contact but this is the ‘trade-off’, namely that a MOOC may have less student/professor contact but some of that may be worth sacrificing for openness and access. Sebastian Thrun was teaching 200 students at Stanford, on his MOOC it was 169,000. That would have taken 800 years at his old teaching rate. Even with the 26,000 that completed, it’s 130 years. The benefits of scale and literally ‘massive’.

3. Diversity

The philosophy Professors at San Jose, who recently wrote an ‘open letter’, complained that MOOCs undermine the ‘diversity’  of the student mix. How they came to that conclusion beggars belief. MOOCs are massively diverse in terms of age, nationality, ethnic origins and background. This is precisely is a consequence of them being Massive, Open and Online. This is an important point in learning, as critical thinking may well be enhanced by having a larger, more diverse set of globally-based, learners engaged on courses. It shifts us out of our cultural groupthink and brings in a wider range of experience, example and perspectives.

4. Academic status

Rather than the occasional academic making an appearance through a TV series on art of history, we could see a renaissance of interest in knowledge and learning if they engaged more directly and openly with society. A good example in the UK are Classical scholars, such as Mary Beard and Robin Lane-Fox, who have headed up TV series on Roman history. With MOOCs, many more talented academics will have a chance to reach out to audiences beyond their own yearly intake of students.

5. Relevance

This may also realign university subjects and activities more closely with the needs of their communities, economies and student needs. I live in a relatively small town, Brighton, with two large Universities, yet there is precious little engagement between them and the local population. The vast majority would be hard pressed to name the Vice-chancellors or even a single academic at either institution. As a local employer , who employed many students from both Universities, it worried me over many years how disinterested they were in even minor curriculum tweaks or the fate of students beyond graduation date. Engagement with the local community through the arts, debates, public lectures and reuse of low-occupancy buildings and sports facilities would make Universities more loved.

 6. Giving

Rather than the educational colonialism of setting up shop in the developing world with new-build campuses, the developed world could funnel educational aid through MOOCs. This would have greater impact through scale and lower costs. The evidence from MOCCs so far is that huge numbers of people are accessing them from countries where HE is not affordable or even remotely accessible for the majority of citizens. I’d like to see some foreign aid budgets go to MOOCs, especially further down the educational ladder into schools.

7. Reframe away from ’18 year-old undergrad

When something new, and let’s even use that word ‘disruptive’, hits a sector, debate erupts, especially on social media and blogs. This is all good as it helps us think through the many issues that emerge, some predictable, some not so predictable. But one thing has happened that surprises me in the debate is the framing of this new phenomenon (MOOCs) into the old, restrictive model of the 18 year old undergraduate course.

If you believe that the purpose of a MOOC is to mimic the standard undergraduate course, you will be disappointed as many of the participants in MOOCs are not young undergraduates. You will also see drop-out, rather than drop-in, a category mistake that sees anything other than passing the final exam as failure (a BIG mistake). There is also a false assumption that face-to-face teaching is a necessary condition for learning. It is not. We learn most of what we learn, not from direct teaching but informally from all sorts of sources and interactions. This is not to say that teaching is unimportant. In practice, on MOOCs, human contact takes all sorts of forms, from teacher to student, student to student, content to student, peer assessment, physical meetups among students, forums, social media. This is a rich blend of human interaction and, in connectivist MOOCs, it is this very feature that, their connectivist founders claim, makes them work so well. There are demands for more rigour in summative assessment, despite the fact that many learners may not want summative assessment at all and others lighter forms of assessment. MOOCs are taken for all sorts of reasons by all sorts of people from all sorts of places. For many it’s not a paper-chase. Squeezing the debate back into the ‘do I get a credit for this course – if not it’s a waste of time’ is wrong-headed.

God’s in the detail

Sure, there’s the old world that has to adjust to new ideas but we can’t hang on to old practices just because they’ve been around for a long time - we’d never have got rid of slavery! On the other hand we must be careful not to totally abandon old practice and look for readjustments, for example, the recording or inclusion of active learning within lectures. We can surely borrow from the work that’s been done on OER, connectivist MOOCs, adaptive learning and so on. MOOCs are not the preserve of one group, country or group of elite Universities.

To move forward we have to look at the different species of MOOCs, new target audiences, different economic models and the pedagogic detail.  There’s more to MOOCs than just cMOOCs and xMOOCS - the taxonomy is much richer and wider. Vast new audiences are also emerging. New players in new combinations are trying new ways of making education cheaper. In particular, VOOCs (Vocational Open Onine Learning Courses) are hitting the market through Udemy and on teh other platforms. On pedagogy, we have different forms of recorded lectures (much progress been made here), peer assessment (very promising), forums, groups, adaptive learning, social media, physical meetups arranged by students (this is interesting), summative assessment (lots of options here) and so on. Kites are being flown and no doubt some will go into free-fall, others hover and yet other soar. 

MOOCs: Kick ass on final assessment

MOOCs make everyone reflect, discuss and experiment with pedagogy in way that is far more agile than the slow and ponderous ‘research’ route. Let’s face it, HE accreditation is odd. You get a two numbers with a dot between them. What use is that? We need far more innovation on what we assess, when we assess and how we assess. MOOCs are starting to give us real answers.

So what models have emerged?

1. No certification

First up, MOOCs are NOT, fundamentally, about summative assessment. It is clear than huge numbers of learners don’t give a toss about accreditation. For them, and I’m one of them, it’s not a paper chase but a learning experience. Many will choose to learn without wanting to sit a final exam or get any form of certification. Don’t assume that everyone is gagging for a certificate from the University of ‘somewhere’ – they’re not. To be honest, as someone who spent years delivering massive learning projects to employers, few of them care a jot about certificates. We need to separate the MOOC movement from the idea of summative assessment being a necessary condition for success. Some free MOOCs offer no certification at all, seeing it as a pure learning experience. Carnegie Mellon have a whole rack of such courses on language learning, science and maths.

For many, however, certification will be desirable. This may be important for students who want to use these courses for progression, jobs, even personal motivation and satisfaction. Certification also matters as a revenue model for the platform providers and Universities. This is where they hope to make money.

2. Certificate of completion

Certification is for completion, the norm in Coursera, simply recognises that the student has stuck with the course, got through all of the formative assessments and assignments and, well, completed the course. This is fine for those who simply want some recognition at the end, without a need for official accreditation.

3. Certificate of mastery

Some edX courses from Harvard and MIT have Certificates of Mastery. They come with a grade but not an official credit. EdX offer a certificate of mastery issued at the discretion of edX and the University that offered the course. These certificates have been free but they plan to charge a modest fee in the future. In an interestingly footnote, edX hold certificates for learners from Cuba, Iran, Syria and Sudan in line with US embargoes!

4. Certificates of distinction

Different levels of accomplishments are being offered by many MOOC providers. With Udacity, this is the core model, with the following different grades; completion, distinction, high distinction, highest distinction. This is not far off the 3^rd, 2.2, 2.1dn 1^st model. Udacity also offer a "testing kit" to any institution for a low fee if they are interested in providing proctored exams on our courses.

5. University credits

On selected courses for San Jose State University (transferable within the California State University system), where credits are available, you pay $150 and this buys you the course, course support, direct comms with instructors/staff and online proctored exams with credited transcript. There are different grades; completion, distinction, high distinction, highest distinction and a service where resumes are sent to prospective employers.

How and when are these exams managed?

Proctored online

Huge efforts are being made to allow learners to sit summative exams online. It’s a complex but not insurmountable problem. Identity, cheating, security and other issues have to be addressed. Iris, fingerprint and voice recognition are just some of the digital identity methods used. Motion sensing and camera identification are also used. Progress is being made. Note that almost all exam methods are subject to cheating. Even proctored offline paper exams do suffer from distributed leaks, teacher and student cheating. One of the advantages of online testing is that questions can be drawn from randomised banks or different numbers laced into test items, and answer options randomised, to prevent the straight copying cheating that exists in physical, paper exams.

Udacity and Coursera both offer online proctored exams at home (a cost of $60–$90) through ProctorU. With ProctorU, you make an appointment, log in to the website and speak to a live proctor who talks you through the process via webcam. You can select a date, time and you are ready to go. At the appointed time, the proctor gets control of your screen and IDs you by requesting photo ID. The proctor will snap photos of you and ask you personal questions, using public databases. They will also make the student do a 360 degree scan of the room with the webcam and ask to see the monitor and its surroundings on the webcam, mirror or CD, left and right. During the exam, the proctor watches the student’s body and eye movement through the webcam.

Proctored test centres

Udacity and edX both offer proctored exams at Pearson VUE test centres. There’s lots of angst around Pearson’s involvement in proctored exams, through Pearson VUE, but why? They have invested in test centres and can deliver this stuff to large numbers of people at low cost. This is how we pass our driving test. We pay for a course to learn the theory and practice (increasingly learning the theory online), then book a test. National networks of centres allow students and adult learners to sit exams at place and time of their own convenience. This frees learners from the tyranny of time and place. 

Pearson VUE has test centres in every US state and over 4400 test centres in 160 countries. These centres have surveillance and biometric systems, in particular a digital fingerprinting system, used by the FBI, that has an almost zero rejection rate.

Innovations

This flurry of activity in MOOCs has produced summative assessment that takes us forward in our thinking:

1. Has different degrees of certification based on demand

2. Caters for different types of learner

3. Offers anytime assessment

4. Offers anywhere exams at home

5. Offers network of test centre exams

6. Sees education funded by volume certification

7. Can be cheaper

8. Pushes Innovation in online testing, like essay marking

9. Makes us see that certification is not always desirable

When people say, there’s nothing new in MOOCs, think again and look at the detail. When we do, there’s some radical changes taking place, not least in exams and certification. The main benefit is in loosening up the whole process and not regarding certification as some sort of one-off, end-of-year, binary pass or fail activity. We can expect more experimentation and innovation, and more is good.

One final note, and this is radical. Why can’t we separate accreditation and testing from the institutions that deliver the learning? It avoids the obvious conflict of interest. Why can’t we have a free Google like service for accreditation?  Wouldn’t that be great for learners?

MOOCs: taxonomy of 8 types of MOOC
MOOCs: Who’s using MOOCs? 10 different target audiences
MOOCs: a breath of fresh air, albeit the same air

MOOCs: more action in 1 year than last 1000 years