Learning Experience Systems – just more click-through online learning?

I have this image in my lounge. He's skating, a clergyman skating, as we so often do when we think we're learning - just skating over the surface. For all the talk of Learning Experience Systems and ‘engagement’, if all you serve up are flat media experiences, no matter how short or micro, with click-through multiple choice or worse, drag and drop, you’ll have thin learning. Simply rebadging platforms with the word ‘Experience’ in the middle doesn’t cut it, unless we reflect on what those ‘experiences should be. All experience is learning but some experiences are much more effective than others (effortful). Simply plopping the word 'experience into the middle of the old LMS terms is to simply rebadge. 

 As Mayer showed, this does not mean making things media rich; media rich is not mind rich. This often inhibits learning with unnecessary cognitive load.
Neither does it simply mean delivering flat resources. Similarly with some types of explicit gamification, where the Pavlovian rewards become ends in themselves and inhibit learning. Good gamification, does in fact, induce deep thought – collecting coins, leader boards and other ephemera do not, as the gains are short-lived.

The way to make such systems work is to focus on effortful ‘learning’ experiences, not just media production. We know that what counts is effortful, desirable and deliberate practice.

Engagement

Engagement does not mean learning. I can be wholly engaged, as I often am, in all sorts of activities – walking, having a laugh in the pub, watching a movie, attending a basketball game – but I’m learning little. Engagement so often means that edutainment stuff - all tainment and no edu. The self-perception of engagement is, in fact, often a poor predictor of learning. As Bjork repeatedly says, on the back of decades of research, from Roediger, Karpicke, Heustler, Metcalfwe and many others, “we have a flawed model of how we learn and remember”. 

We tend to think that we learn just by reading, hearing and watching. When, in fact, it is other, effortful, more sophisticated practices that result in far more powerful learning. Engagement, fun, learner surveys and happy sheets have been shown to be poor measures of what we actually learn and very far from being optimal learning strategies.

Ask Traci Sitzman who has done the research, Sitzmann (2008). Her work on meta-studies, on 68,245 trainees over 354 research reports, attempt to answer two questions:

Do satisfied students learn more than dissatisfied students?After controlling for pre-training knowledge, reactions accounted for only 2% of the variance in factual knowledge, 5% of the variance in skill-based knowledge, 0% of the variance in training transfer. The answer is clearly no!

Are self-assessments of knowledge accurate? Self-assessment is only moderately related to learning. Self-assessment capture motivation and satisfaction, not actual knowledge levels

Her conclusion based on years of research, and I spoke to her and she is adamant, is that self-assessments should NOT be included in course evaluations and should NOT be used as a substitute for objective learning measures.

Open learning

It’s effort to ‘call to mind’ that makes learning work. Even when you read, it’s the mind reflecting, making links, calling up related thoughts that makes the experience a learning experience. But this is especially true in online learning. The open mind is what makes us learn and therefore open response is what makes us really learn in online learning.

You start with whatever learning resource, in whatever medium you have: text (pdf, paper, book…), text and graphics (PowerPoint…), audio (podcast) or video. By all means read your text, go through a Powerpoint, listen to the podcast or watch a video. It’s what comes next that matters.

With WildFire, in addition to the creation of on line learning, in minutes not months, ae have developed open input by learners, interpreted semantically by AI to. You literally get a question and a blank box into which you can type whatever you want. This is what happens in real life – not selection items from multiple-choice lists. Note that you are not encouraged to just retype what you read saw or heard. The point, hence the question, is to think, reflect, retrieve and recall what you think you know.

Here’s an example, a definition of learning…

What is learning?

Learning is a lasting change in a person’s knowledge or behaviour as a result of experiences of some kind.

Next screen….

You are asked to tell us what you think learning is. It’s not easy and people take several attempts. That’s the point. You are, cognitively, digging deep, retrieving what you know and having a go. As long as you get the main points, that it is a lasting change in behaviour or knowledge through experiences, you’re home and dry. As the AI does a semantic analysis, it accepts variations on words, synonyms and different word order. You can’t cut and paste and when you are shown the definition again, whatever part you got right, is highlighted.  

It’s a refreshing experience in online learning, as it is so easy to click through media and multiple-choice questions thinking you have learnt. Bjork called this the ‘illusion of learning’ and it’s remarkably common. Learners are easily fooled into thinking they have mastered something when they have not.

This fundamental principle in learning, developed in research by Bjork and many others, is why we’ve developed open learning in WildFire. 

Conclusion

Engagement is not a bad thing but it is neither a necessary, and certainly not a sufficient condition, for learning. LXP theory lacks - well theory and research. We know a lot about how people learn, the excessive focus on surface experience may not help. All experience leads to some learning. But that is not the point, as some experiences are better than others. What those experiences should be are rarely understood by learners. What matters is effortful learning, not ice skating across the surface, having fun but not actually learning much – that is click-through learning. 

Bibliography

Alleger et al. (1997) A meta-analysis of the relations among training criteria. Personnel Psychology 50, 341-357.

Sitzmann, T. and Johnson, S. K. (2012). When is ignorance bliss? The effects of inaccurate self-assessments of knowledge on learning and attrition. Organizational Behavior and Human Decision Processes, 117, 192–207.

Sitzmann, T., Ely, K., Brown, K. G., & Bauer, K. (2010). Self-assessment of knowledge: A cognitive learning or affective measure? Academy of Management Learning and Education, 9, 169-191.

Brown, K. G., Sitzmann, T., & Bauer, K. N. (2010). Self-assessment one more time: With gratitude and an eye toward the future. Academy of Management Learning and Education, 9, 348-352

Sitzmann, T., Brown, K. G., Casper, W. J., Ely, K. and Zimmerman, R. (2008). A review and meta-analysis of the nomological network of trainee reactions. Journal of Applied Psychology, 93, 280-295.