Saturday, May 18, 2013

MOOC on Human-Computer Interaction: videos have 7 fails in HCI

Enjoying Stanford’s Coursera MOOC on Human-Computer Interaction but (oh the irony) the screen design and pedagogy of the many videos, is awful. Don’t get me wrong, it’s easy to use, has good content and I’m getting what I thought I’d get - a reasonable course. It is very video heavy, which is OK; let’s face it most HE courses are lecture heavy - at least they’re not an hour long, I can watch these when I want, repeat them and, in Coursera fashion, I get a bit of formative assessment during the videos, something students rarely get in real institutions. But it could have been so much better.
1. Small screen, low retention.
Now I don’t mean to be picky but having a tiny talking head (see above), literally less than 4.5% the size of the screen on the bottom right, is a BAD idea. Nass & Reeves (ironically from Stanford) did a great experiment with video at different screen sizes, showing that the smaller the screen size the lower the retention. Read the research guys.
2. Too much talking head.
It’s dull, the delivery is often poor and poorly edited (i.e. padded out). It’s like watching a very long news item read on one news story, time after time. Even worse is the fact that it’s a medium shot, showing the whole upper body. Go on a video course guys, you need to see the white of their eyes. Use close ups. To be fair there’s more images, graphs and screens with audio only as the course progresses. Think about ‘attention’ guys.
3. Cognitive dissonance
The too much talking head error is compounded by presenting text headings and blocks of text in a huge font over the rest of the screen.Mayer & Clark’s research showed that you don’t show text and video at same time, as you have to hop back and forth from visual (reading - semantic) to audio channels. Even worse, Scott the presenter reads the text but it’s not the text that appears on the screen. Also, the framing of the video, with text cut in half behind the presenter is cognitive noise we can do without. Watch some TV guys.
4. Paucity of images
What’s odd is the fact that when schemas or techniques or procedures are being described there’s often no images shown. This is like PowerPoint without any pictures, just big headings. In many places the point, event or procedure would have been better served by cutting away to what was being described or relevant images. It gets better in some places. Very strange.
5. Presentation style
As it’s often a little dull, I found my attention tended to drift. I can read faster than the presenter speaks, and when in the first video he started looking down and reading points one by one, the video producer in me rebelled. I get impatient with slow, amateurish delivery, which is why I like the edX and Coursera x1.5 speed feature. In fact, so much of this sounds like the reading of written material that it could have been text. Read something on relevant media mix guys. I like Scott Klemmer, but he’s no presenter, and after a while his excessive hand gestures and delivery style start to grate. This can be a problem for the single academic courses - it's like watching a ten week news programme with only one presenter.
6. Poor editing
The in-video questions are poorly edited in, so you often get a snippet of a sentence from the next sequence. Small point but it makes the production seem a little amateurish. Edit it properly guys. Again Nass & Reeves showed that these unexpected and awkward pauses and edits lower retention.
7. Poor question design
In-video questions are made progressively easier and meaningless. This is learning design at its worst.  The same question is posed, with the same options, up to four times. So when you’ve answered 1 out of 4, the next question is 1 out of 3, the next 1 out of 2 and the last a meaningless 1/1. Even worse, is the cardinal sin of two options actually being correct but only one accepted. All of this is bizarre and lazy. Read something on test items guys.
Not all Coursera MOOCs are so poor on video. The University of Edinburgh MOOC on E-learning and Digital Cultures (one of 6 MOOCs attracting 306,000 starters), which ran at the start of the year, didn’t do talking heads, relying on curated video. This caused some consternation with students who expected lectures. This course had much more of a looser structure with discussion, Google Hangouts and social media, none of it moderated by academics. Interestingly, I liked this Edinburgh course less, as I thought it was weak on focus, depth and content. There’s a balance to be struck here and much to do on improving the pedagogy and design of MOOCs. I don't buy the cMOOC/xMOOC thing - it's a simplistic dualism. There a whole variety of pedagogies that lie between straight instruction and social constructivism (I like neither in their pure form).
What Coursera should have done is do what this course recommends, apply the usability test strategies that Krug, Norman and Nielsen recommend. Get in the experts and do ‘voiced trials'. I have spent nearly 30 years designing and producing online learning and would never have got a client to pay for these courses. To be fair, compared to the benchmark of dull one hour long lectures, it’s an improvement and it’s a start. This is a constructive critique of the videos and I must remind you that the course is rich in assignments and practical work. Let’s hope they get some HCI professionals in to make it a little more usable and ‘learner and learning’ friendly. It’s not as if people haven’t done this before.


Rene Kizilcec said...

Hi Donald,

You might want to give The Media Equation's chapter on screen size another read. The section on 'Big Screens and Education' (p. 200-201) clearly states that "there are some good arguments against big pictures. One is that too much arousal can be counter-productive in learning. It is quite possible for emotional experiences to distract."

Mather (2007) reviews the two opposing hypotheses that arousal enhances or impairs binding in memory and presents a framework that facilitates predictions of whether an arousing object will enhance or impair memory-binding.
Mather (2007, pp. 44-45) clarifies that "arousal associated with an object elicits focused attention that enhances within-object binding, making it easier to later remember which features (such as color and location) were associated with that object. But that focused attention does not benefit binding of the arousing object with other objects or with contextual features and sometimes even impairs it."
Given that learning requires the formation of connections between new objects and existing ones, this raises a serious issues with simplistic claims about the association between attention and learning.

Reeves and Nass raise this issue in their physics lecture/roller coaster analogy (p. 200).

- Rene

For your reference:
Mather, M. (2007). Emotional Arousal and Memory Binding: An Object-Based Framework. Perspectives on Psychological Science, 2(1), 33–52. doi:10.1111/j.1745-6916.2007.00028.x

Unknown said...

I was going to say that it appears those who lecture on HCI don't heed their own words. When I was lecturing in that discipline and referred my learners to Neilsen's site for research it was the opposite of what he talked about. Two column layout, awful font choices, navigation could be disorienting.....
I've just checked this out again and at last he seems to be practising what he preaches a bit more. However he still goes for the two column layout. Anyone who has done anything about accessibility should know this is a no no for screen readers.
Its a case of physician heal thyself. We know the rules of HCI so we can ignore them!

Donald Clark said...

Couldn't agree more. Neilsen was famous for not eating his own dogfood. However, in a course on HCI, where usability affectes learning and retention it is astonishing that they break so many rules. The problem is that academics rarely have sufficient knowledge of the psychology of learning, especially online learning. There is a culture of letting the academics do the learning design. They are different disciplines. This is a general problem for MOOCs. I do think that Udacity are ahead of Coursera and edX on this, but all need to improve.

Donald Clark said...

Rene - the study, I think, was clear "Larger screens mean more arousal, stronger memories, and more positive evaluations of the content they displayed". The point you refer to comes up in the general discussion at the end of the paper and concerns distractive arousal, such as high action images of roller coasters that detract from learning, say about gravity. This is a point that Mayer tested and confirmed. Note that Nass & Reeves are NOT saying that arousal is BAD. In fact they correlate arousal with retention. In the case of the Coursera course, there is no fast action distraction. The 4.5% of the screen video is small and does not arouse in the first place - confirming their study.
The Mayer research confirms this. What the Coursera course does on screen is present the SAME competing objects on the SAME screen in two different modes - text/audio. Their research confirmed that this impairs learning. We are not talking about roller-coasters here - but about the presentation of knowledge aimed at semantic memory. I think, in the end you, I, Nass, Reeves and Mayer all agree - media rich does not mean mind rich, but media poor often means mind poor.

Anonymous said...

Would be interesting to see soem 'good' examples