MOOCs: Kick ass on final assessment

MOOCs make everyone reflect, discuss and experiment with pedagogy in way that is far more agile than the slow and ponderous ‘research’ route. Let’s face it, HE accreditation is odd. You get a two numbers with a dot between them. What use is that? We need far more innovation on what we assess, when we assess and how we assess. MOOCs are starting to give us real answers.

So what models have emerged?

1. No certification

First up, MOOCs are NOT, fundamentally, about summative assessment. It is clear than huge numbers of learners don’t give a toss about accreditation. For them, and I’m one of them, it’s not a paper chase but a learning experience. Many will choose to learn without wanting to sit a final exam or get any form of certification. Don’t assume that everyone is gagging for a certificate from the University of ‘somewhere’ – they’re not. To be honest, as someone who spent years delivering massive learning projects to employers, few of them care a jot about certificates. We need to separate the MOOC movement from the idea of summative assessment being a necessary condition for success. Some free MOOCs offer no certification at all, seeing it as a pure learning experience. Carnegie Mellon have a whole rack of such courses on language learning, science and maths.

For many, however, certification will be desirable. This may be important for students who want to use these courses for progression, jobs, even personal motivation and satisfaction. Certification also matters as a revenue model for the platform providers and Universities. This is where they hope to make money.

2. Certificate of completion

Certification is for completion, the norm in Coursera, simply recognises that the student has stuck with the course, got through all of the formative assessments and assignments and, well, completed the course. This is fine for those who simply want some recognition at the end, without a need for official accreditation.

3. Certificate of mastery

Some edX courses from Harvard and MIT have Certificates of Mastery. They come with a grade but not an official credit. EdX offer a certificate of mastery issued at the discretion of edX and the University that offered the course. These certificates have been free but they plan to charge a modest fee in the future. In an interestingly footnote, edX hold certificates for learners from Cuba, Iran, Syria and Sudan in line with US embargoes!

4. Certificates of distinction

Different levels of accomplishments are being offered by many MOOC providers. With Udacity, this is the core model, with the following different grades; completion, distinction, high distinction, highest distinction. This is not far off the 3^rd, 2.2, 2.1dn 1^st model. Udacity also offer a "testing kit" to any institution for a low fee if they are interested in providing proctored exams on our courses.

5. University credits

On selected courses for San Jose State University (transferable within the California State University system), where credits are available, you pay $150 and this buys you the course, course support, direct comms with instructors/staff and online proctored exams with credited transcript. There are different grades; completion, distinction, high distinction, highest distinction and a service where resumes are sent to prospective employers.

How and when are these exams managed?

Proctored online

Huge efforts are being made to allow learners to sit summative exams online. It’s a complex but not insurmountable problem. Identity, cheating, security and other issues have to be addressed. Iris, fingerprint and voice recognition are just some of the digital identity methods used. Motion sensing and camera identification are also used. Progress is being made. Note that almost all exam methods are subject to cheating. Even proctored offline paper exams do suffer from distributed leaks, teacher and student cheating. One of the advantages of online testing is that questions can be drawn from randomised banks or different numbers laced into test items, and answer options randomised, to prevent the straight copying cheating that exists in physical, paper exams.

Udacity and Coursera both offer online proctored exams at home (a cost of $60–$90) through ProctorU. With ProctorU, you make an appointment, log in to the website and speak to a live proctor who talks you through the process via webcam. You can select a date, time and you are ready to go. At the appointed time, the proctor gets control of your screen and IDs you by requesting photo ID. The proctor will snap photos of you and ask you personal questions, using public databases. They will also make the student do a 360 degree scan of the room with the webcam and ask to see the monitor and its surroundings on the webcam, mirror or CD, left and right. During the exam, the proctor watches the student’s body and eye movement through the webcam.

Proctored test centres

Udacity and edX both offer proctored exams at Pearson VUE test centres. There’s lots of angst around Pearson’s involvement in proctored exams, through Pearson VUE, but why? They have invested in test centres and can deliver this stuff to large numbers of people at low cost. This is how we pass our driving test. We pay for a course to learn the theory and practice (increasingly learning the theory online), then book a test. National networks of centres allow students and adult learners to sit exams at place and time of their own convenience. This frees learners from the tyranny of time and place. 

Pearson VUE has test centres in every US state and over 4400 test centres in 160 countries. These centres have surveillance and biometric systems, in particular a digital fingerprinting system, used by the FBI, that has an almost zero rejection rate.

Innovations

This flurry of activity in MOOCs has produced summative assessment that takes us forward in our thinking:

1. Has different degrees of certification based on demand

2. Caters for different types of learner

3. Offers anytime assessment

4. Offers anywhere exams at home

5. Offers network of test centre exams

6. Sees education funded by volume certification

7. Can be cheaper

8. Pushes Innovation in online testing, like essay marking

9. Makes us see that certification is not always desirable

When people say, there’s nothing new in MOOCs, think again and look at the detail. When we do, there’s some radical changes taking place, not least in exams and certification. The main benefit is in loosening up the whole process and not regarding certification as some sort of one-off, end-of-year, binary pass or fail activity. We can expect more experimentation and innovation, and more is good.

One final note, and this is radical. Why can’t we separate accreditation and testing from the institutions that deliver the learning? It avoids the obvious conflict of interest. Why can’t we have a free Google like service for accreditation?  Wouldn’t that be great for learners?

MOOCs: taxonomy of 8 types of MOOC
MOOCs: Who’s using MOOCs? 10 different target audiences
MOOCs: a breath of fresh air, albeit the same air

MOOCs: more action in 1 year than last 1000 years