Sunday, January 07, 2018

Astonishing fake in education and training - the graph that never was

I have seen this in presentations by the CEO of a large online learning company, Vice-Chancellor of a University, Deloitte’s Bersin, and in innumerable keynotes and talks over many years. It’s a sure sign that the speaker has no real background in learning theory and is basically winging it. Still a staple in education and training, especially in 'train the trainer' and teaching courses, a quick glance is enough to be suspicious.
Dales’s cone

The whole mess has its origins in a book by Edgar Dale way back in 1946. There he listed things from the most abstract to the most concrete: Still pictures, Visual symbols, Verbal symbols, Radio recordings, Motion pictures, Exhibits, Field trips, Demonstrations, Dramatic participation, Contrived experiences, Purposeful experience and Direct. In the second edition (1954) he added Dramatised experiences through Television and in the third edition, and heavily influenced by Bruner (1966), he added enactive, iconic symbolic.
But let’s not blame Dale. He admitted that it is NOT based on any research, only a simple intuitive model and he did NOT include any numbers. It was, in fact, simply a gradated model to show the concreteness of different audio-visual media. Dale warned against taking all of this too seriously, as a ranked or hierarchical order. Which is exactly what everyone did. He actually listed the misconceptions in his 1969 third edition p128-134. So the first act of fakery was to take a simple model, ignore its original purpose, and the authors warnings, and use it for other ends.
Add fake numbers
First up, why would anyone with a modicum of sense believe a graph with such rounded numbers? Any study that produces a series of results bang on units of ten would seem highly suspicious to someone with the most basic knowledge of statistics. The answer, of course, is that people are gullible, especially to messages that appeal to their intuitive beliefs, no matter how wrong. The graph almost induces confirmation bias. In any case, these numbers are senseless unless you have a definition of what you mean by learning and the nature of the content. Of course, there was no measurement – the numbers were made up.
Add Fake Author
At this point the graph has quite simply been sexed up by adding a seemingly genuine citation from an academic and Journal. This is a real paper, about self-generated explanations, but has nothing to with the fake histogram. The lead author of the cited study, Dr. Chi of the University of Pittsburgh, a leading expert on ‘expertise’, when contacted by Will Thalheimer, who uncovered the deception, said, "I don't recognize this graph at all. So the citation is definitely wrong; since it's not my graph." Serious looking histograms can look scientific, especially when supported by bogus academic credentials.
Add new categories
The fourth bit of fakery was to add ‘teaching others’ to the end, topping it up to, you guessed it – 90%. You can see what’s happening here – flatter teachers and teaching, and they’ll believe anything. They also added the ‘Lecture’ category on at the front – and curiously CD-ROM! In fact, the histogram has appeared in many different forms, simply altered to suit the presenter's point in a book or course. This is from Josh Bersin’s book on Blended Learning. It is easy to see how the meme gets transmitted when consultants tout it around in published books. Bersin was bought by Deloittes. What happens here is that Dales original concept is turned from a description of media into the prescription of methods.
The Colored Pyramid
The third bit of fakery, was to sex it up with colour and shape, going back to Dale’s pyramid but with the fake numbers and new categories added. It is a cunning switch, to make it look like that other caricature of human nature, Maslow’s hierarchy of needs. It suffers from the same simplistic idiocy that Maslow’s pyramid does – that complex and very different things lie in a linear sequence one after the other. It is essentially a series of category mistakes, as it takes very different things and assumes they all have the same output – learning. In fact, learning is a complex thing, not a single output. A good lecture may be highly motivating, there are semantic tasks that are well suited to reading and reflection, discussion groups may be useless when struggling with deep and complex semantic problems and so on. Of course, the coloured pyramid makes it look more vivid and real, all too easy to slot into one of those awful 'train the trainer' or 'teacher training' courses that love simplistic bromides.
What’s damning is that this image and variations of the data have been circulating in thousands of PowerPoints, articles and books since the 60s. Investigations of these graphs by Kinnamon (2002) found dozens of references to these numbers in reports and promotional material. Michael Molenda (2003), did a similar job. Their investigations found that the percentages have even been modified to suit the presenter’s needs. This is a sorry tale of how a simple model published with lots of original caveats can morph into a meme that actually lies about the author, the numbers, adds categories and is uncritically adopted by educators and trainers.
Much of this comes from the wonderful Will Thalhemer’s original work. I wanted to give it an extra and more structured spin on the development of the fakery.
Bruner, J. (1966), Toward a Theory of Instruction
Dale, E (1946), Audiovisual methods in teaching
Dale, E (1954), Audiovisual methods in teaching
Dale, E (1969), Audiovisual methods in teaching
Kinnamon, J. C. (2002). Personal communication, October 25
Kovalchick, A and Dawson,K (2004), Education and Technology
Thalheimer, W Blogpost

Molenda, M. H. (2003). Personal communications

No comments: