Engagement - most used yet most misunderstood word in learning?

Engagement is an odd word in learning. It is clearly a good thing to be engaged with a learning experience but just because people are engaged, does not mean they are learning. I’ve been totally engaged for years watching stand-up comedy experiences but can barely remember a single joke from any of them. Engagement can be a positive sign in learners but is no guarantee of learning. 

Learners can be:

·      engaged but not learning

·      engaged but simply going through things they already know

·      engaged but doing harm to learning 

·      engaged and learning

In other words, engagement can be a dangerous proxy for learning. One can be engaged and give the appearance of learning, without learning taking place, even inhibiting learning. What’s more, there may be little or no long-term or far transfer into actual application in the real world.

Engaged but not learning

Engagement while watching a video may feel like you are learning but the transience effect, fact that you have a 20 second window for attention and can only manipulate 3-4 things in working memory, mean that you are forgetting much of what you have just watched, no matter how engaged you felt. Your mind is like a shooting star burning up memories it leaves behind.

Engaged but simply going through things they already know

Learners can be comfortable skating superficially across content they are familiar with. This can also create the illusion of learning. Graham Nutall, in empirical classroom research, claimed that “In most of the classrooms we have studied, each student already knows about 40–50% of what the teacher is teaching”. Engagement with the easy and familiar is different from effortful engagement with the difficult and unfamiliar, the zone in which learning takes place.

Engaged but doing harm to learning

Then there is the engagement which is over-stimulating and prevents deeper processing, reflection and generation of ideas and retrieval from your own mind. Games can be a total distraction if engagement is with the mechanics of the game and not the learning.

Delusional learning

One thing that research in cognitive psychology has gifted to us over the last decade or so, is clear evidence that learners are delusional when it comes to judgements about what they think is engagement in their own learning. The big name in the field is Bjork, along with many other high quality researchers, who say that learning is “quite misunderstood (by learners)…. we have a flawed model of how we learn and remember”. There’s often a negative correlation between people’s judgements of their learning, what they think they have learnt and how they think they learn best - and what they’ve actually learnt and the way they can actually optimise their learning. In short, our own perceptions of learning can be seriously misleading. This is why ‘fun’ and ‘engagement’ can be misleading proxies. Also, why learner surveys and happy sheets are such bad measures of what is actually learnt and not helpful when designing optimal learning strategies and experiences.

Bjork warns us about ‘illusory learning’ and shows that learners are frequently mistaken about whether and what they have learned. In fact he has described them as ‘delusional’. Despite feeling as though they have been through a strong ‘learning experience’, that experience was the feeling that they had learned but when tested that feeling was misleading. Self-reported or data showing engagement is not, therefore, a reliable method for determining whether things were actually learned in the sense of finding a place in long-term memory and behaviour, the gold-standard for actual learning. It may just signal easy engagement, shallow experiences where the learner is engaged and entertained but does no deep learning. 

Desirable difficulty

Bjork thinks that both teaching and learning can be improved and optimised using techniques that force cognitive effort. Much learning is imagined, as it is too easy. Learning designers must therefore be careful not to allow too ‘easy’ engagement and likely to lead to forgetting. Learning experiences need to be challenging . 

Real, long-term learning requires ‘Desirable’ (accomplishable) and ‘Difficult’ learning with real effort, for high-retention. It is all in the struggle, failure and overcoming of difficulty that real effortful learning takes place. This is why so much online learning fails. To simply click on faces and see speech bubbles of text, drag and drop labels, choose true or false and so on, rarely constitutes desirable difficulty. This is click-through learning.

Desirable difficulties, are potent forms of learner engagement, such as the following learning techniques (just a selection):

·      Active retrieval practice

·      Allow failure

·      Interleaving

·      Generation

Active retrieval practice

To be specific about effortful learning, by effort we mean ‘active retrieval’ as the most powerful learning strategy at your disposal. The brain, the organ that named itself, is a unique organ in that it can test itself to see what it knows or doesn’t know. At the same time this act of retrieval consolidates has been found to be even more powerful than the original learning experience. 

The first solid research on retrieval was by Gates (1917), who tested children aged 8-16 on short biographies. Some simply re-read the material several times, others were told to look up and silently recite what they had read. The latter, who actively retrieved knowledge, showed better recall. Spitzer (1939) made over 3000 11-12 year olds read 600 word articles then tested students at periods over 2 months. The greater the gap between testing (retrieval) and the original exposure or test, the greater the forgetting. The tests themselves seemed to halt forgetting. Tulving (1967) took this further with lists of 36 words, with repeated testing and retrieval. The retrieval led to as much learning as the original act of studying. This shifted the focus away from testing as just assessment to testing as retrieval, as an act of learning in itself. Roediger et al. (2011) did a study on text material covering Egypt, Mesopotamia, India and China, in the real context of real classes in a real school, a Middle School in Columbia, Illinois. Retrieval tests, only a few minutes long, produced a full grade-level increase on the material that had been subject to retrieval. McDaniel (2011) did a further study on science subjects, with 16 year olds, on genetics, evolution and anatomy. Students who used retrieval quizzes scored 92% (A-) compared to 79% for those who did not. More than this, the effect of retrieval lasted longer, when the students were tested eight months later.

So to retrieve and recall what you need to know results in much higher levels of retention. Rather than read, re-read and underline, look away and try to retrieve and recall what you need to know. Repeatedly reading and underline text is not an affective learning experience, looking away from the page to try to recall what you think you know, is better.

This deliberate retrieval requires cognitive effort and that is why recall has a strong reinforcement effect and results in higher retention and subsequent recall. The deeper processing increases retention and subsequent recall. Note that this retrieval practice is not a test or assessment but a learning experience. In fact, it is one of the most powerful learning experiences. 

Rather than click on True or False or an option in a short list (MCQ), look away, think, generate, recall and come up with the answer. The key point here is that research has shown that retrieval is a memory modifier and makes your memory more recallable. Counter-intuitively, retrieval is much more powerful than being presented with the information. In other words it is more powerful than the original ‘teaching’ event.

Roediger and Karpicke (2006) researched studying v retrieval testing (without feedback). One week later the retrieval tested group did much better. They also asked them how much you are likely to remember in one week’s time for each method – oddly, the majority of learners got it completely wrong. Learners are notoriously delusional about learning strategies which is why learning experience design that simply panders to their wishes often fails.

Allow failure

Making errors is also a critical component of successful learning. Yet how often is failure avoided in online learning, even punished? We are so keen on ‘engagement’ and relentless positivity that there is no room for failure.

According to Kornell, Hayes and Bjork (2009), generating the wrong thing, then getting it right, leads to stronger learning. The reason is that you are activating the brain’s semantic network. Retrieval testing does better than reading or watching, as it potentiates recall. Work by Kornell (2009) shows that even unsuccessful testing is better that straight presentations. 

Retrieval testing gives you better internal feedback and works even when you get few or no correct answers. Testing, even before you have access to the material, as a learning experience, also helps learning. Once again, almost bizarrely, Heustler and Metcalfe (2012) asked learners what worked best and they were largely wrong.

Interleaving 

Mixing things up, interleaving, boosts learning. Counterintuitively, blocking learning tasks and doing them in strict order “AAABBBCCC”, seems less efficient than interleaving learning tasks “ABCABCABC”. It would seem that interleaving keeps your brain on its toes with practice that increases desirable difficulty, each task being switched. You have to differentiate or discriminate, between concepts and tasks which strengthens memory associations.

A sizeable body of research, from sports to arts, medical and other subjects and skills, show its efficacy. Then idea is that you’re better off doing separate tasks either within a subject or in different subjects, rather than a series of similar tasks on one topic, subject or drill in say sport.It has also been researched in also in the real world context of the classroom in maths, Rohrer (2011), with astonishing increases in test scores (76%). It has even been shown to be useful in critical think tasks in legal training. 

Creating more diverse connections in the brain seems to help you cope with newer situations as they arise. The research suggests that some familiarity with the subject is needed before interleaving.

Generation

Generative learning is an active, effortful process where the learner generates meaning between what they already know and new knowledge, and then between new concepts and principles that are seen for the first time. This is an extension of rehearsal in the mind and retrieval practice.

Learning experiences that get the learner to generate their own learning and manifest their knowledge and skills, is better than simply reading text. It would also appear to provides the context for greater subsequent recall. Wittrock and Kelly (1984) showed that poor readers who generated relations, connections and headings had a 20% increase in comprehension, when they were asked to come up with a generative plan, comprehension jumped by 23%.

Open input by learners, is one example, which we implement in WildFire, the answers being interpreted by AI semantic analysis.

Conclusion

When learning experiences simply make things easy, in the classroom by low levels or superficial forms of participation or online with shallow clickthrough experiences, learning can be illusory. Learning professionals and learners often engage in practices that are sub-optimal, worse still, practices that may inhibit efficient learning. Bjork recommends techniques that may seem counterintuitive and difficult to implement, as they run counter to most current practice but the evidence suggest that he is right. Technology, of course, has also allowed retrieval practice, interleaving and spaced practice to be more easily implemented over time on a personalized basis.

This research shows that our methods of engagement in online learning are often sub-optimal. The problem we face is that immediate success often means long-term failure. More focus should be given to retrieval, generative learning, interleaving and spaced practice, not presentation or clicking on items and weak multiple-choice questions. We need to be presented with desirable difficulties, through partial or complete open input. This is exactly what we’ve spent the last two years building with WildFire.

There is, of course, a further step in engagement that really matters, and is often completely ignored – transfer.

Bibliography

Bjork, R. A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe & A. Shimamura (Eds.), Metacognition: Knowing about knowing (pp. 185–205). Cambridge, MA: MIT Press.

Ericsson, K. Anders, Krampe, Ralf Th. and Tesch-Romer. Clemens (1993) The Role of Deliberate Practice in the Acquisition of Expert Performance. Psychological Review, Vol. 100. No. 3, 363-406

Nuthall G (2007) The Hidden Lives of Learners. Wellington, NZ: NZCER Press.

Gates, A. I. (1917). Recitation as a factor in memorizing. Archives of Psychology, No. 40, 1-104. 

Spitzer, H. F. (1939). Studies in retention. Journal of Educational Psychology, 30, 641-656.

Tulving, E. (1967). The effects of presentation and recall of material in free-recall learning. Journal of Verbal Learning and Verbal Behavior, 6, 175􏰀184.

Roediger, H. L., Agarwal, P. K., McDaniel, M. A., & McDermott, K. B. (2011). Test-enhanced learning in the classroom: Long-term improvements from quizzing. Journal of Experimental Psychology: Applied, 17, 382-395.

McDaniel, M. A., Agarwal, P. K., Huelser, B. J., McDermott, K. B., & Roediger, H. L. (2011). Test-enhanced learning in a middle school science classroom: The effects of quiz frequency and placement. Journal of Educational Psychology, 103, 399-414

Roediger III, H.L. and Karpicke, J.D., 2006. The power of testing memory: Basic research and implications for educational practice. Perspectives on psychological science, 1(3), pp.181-210.

Kornell, N., Hays, M.J. and Bjork, R.A., 2009. Unsuccessful retrieval attempts enhance subsequent learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(4), p.989.

Kornell, N. and Son, L.K., 2009. Learners’ choices and beliefs about self-testing. Memory, 17(5), pp.493-501.

Huelser, B.J. and Metcalfe, J., 2012. Making related errors facilitates learning, but learners do not know it. Memory & cognition, 40(4), pp.514-527.

Pan, S.C., 2015. The interleaving effect: mixing it up boosts learning. Sci Am, 313(2).

Goode, S. and Magill, R.A., 1986. Contextual interference effects in learning three badminton serves. Research quarterly for exercise and sport, 57(4), pp.308-314.

Hatala, R.M., Brooks, L.R. and Norman, G.R., 2003. Practice makes perfect: the critical role of mixed practice in the acquisition of ECG interpretation skills. Advances in Health Sciences Education, 8(1), pp.17-26.

Kornell, N. and Bjork, R.A., 2008. Learning concepts and categories: Is spacing the “enemy of induction”?. Psychological science, 19(6), pp.585-592.

Rohrer, D., Dedrick, R. F., & Stershic, S. (2015). Interleaved practice improves mathematics learning. Journal of Educational Psychology, 107(3), 900–908. https://doi.org/10.1037/edu0000001

Wittrock, M.C., 1992. Generative learning processes of the brain. Educational Psychologist, 27(4), pp.531-541.