Pay attention! Attention is a necessary condition for learning experience design, but poorly understood...

Pay attention! This is something you, and everyone else, are likely to have heard at school and the teachers were right. Attention is almost always a necessary condition for learning. The problem is that attention wavers. It is difficult to sustain and we are easily distracted. There are distractions everywhere these days, your phone, notifications, vibrations, pings, rings. Learning has to compete with a lot of other things that compete for attention. Even within a learning experience online, the temptation to do something more entertaining nags away at users. We live in the attention economy and knowing what attention is, that modern online currency, is precisely why learning experience design matters, as formal learning is a different type of experience, requiring more sustained attention.

We know a lot about attention, which has been studied for well over a century. William James, the father of psychology, in his Principles of Psychology (1890) gave a detailed account of attention, and Pillbury, in his book Attention (1908), outlines many of the ideas that were later confirmed by research. Behaviourism dampened interest in what was seen as too subjective a subject for a long time but Broadbent’s Filter theory had impact in the 60s and attentional capacity was the adopted model in the 1970s. With Daniel Kahneman, best known for his work on bias, attention theory took a different turn. He saw ‘effort’ as important element in attention, something that could be encouraged and trained and it was cognitive psychology and recent attention on attention in UI and UX design that have led to intense scrutiny.

Attention is to selectively focus your mind and effort on that which has to be learned. Attention matters as it is what manages new data that has to be processed in working memory. Focused attention narrows down input from sensory memory and your recall knowledge from long-term memory. At any moment in time attention is what determines what is processed. We must always be aware that attention is the bottleneck through which everything must go in learning, first into working memory, then if you successfully learn, into long-term memory. The selective nature of attention is what regulates and limits cognitive overload.

Remarkably, although the brain takes in 11 million bits of information a second, we only process around 50 bits, even when reading or doing a complex task like playing the piano. Working memory is a complex and efficient filter with an actual attention span of around 20 seconds and we can only manipulate around 3-4 things at any one time. It is therefore time and quantity limited and highly selective. This is the essential issue that needs to be addressed in learning experience design – dealing with this filter and these limitations.

Attention is also proactive. Effects like The Cocktail Party Effect allow us to attend with laser-like focus on one person in conversation even in a busy room. Indeed, attention in learning, really only works in terms of meaning and deeper processing when it attends to one thing. Even the simplest of complications and overloads, such as reading out the words that appear on the screen in online learning can confuse and actually inhibit learning. One of the great mistakes in designing learning experiences is the failure to recognise that less is often more.

This attentive focus is highly selective and subject to selection, then interpretation on the way to consciousness. Many will have seen how actively selective attention can be when watching the famous ‘basketball’ video. You are cued to count the passes and miss the Gorilla walking through the group. Our selective bias is set by probability of events happening, expectations, perceived value, even recent context and memories. Mind readers and magicians work all of this to their advantage!

So, over a century of research has shown that attention is not one thing but a very complex phenomena. We select inputs, interpretations of those inputs, then select plans of action and actions themselves. Attention is tied up with motivation, interest, feelings and ultimately action. It is the basecamp for understanding how learning experiences should be designed.

Arousal

It is time to introduce another word, that is often confused with psychological attention – arousal. Arousal is different from attention in that it refers to the rise or fall in perceptions or physical activity (some quite specific such as sexual arousal). Arousal occurs when we wake up and our mind rises into consciousness. Arousal is momentary and common, when, for example, driving a car or riding a bike. It can also fall due to fatigue. A great deal of work was done on this during the war, when operators had to pay attention for long periods listening or looking for sounds or signals. Fatigue sets in and arousal falls, leading to errors. The opposite happens with over-arousal, where one also fails in tasks. It is thought that stress may be a form of over-arousal. It is thought, therefore, that arousal is an upside down U shape, with optimal operation at the top of the bend.

What can we learn from these ruminations on arousal for learning? Learning needs a modicum of arousal to reach an optimal level and a reasonable level of arousal appears to be good for retention in long-term memory. So designing learning experiences that raise through arousal, then settle at optimal attention seem sensible. However, and this is important, there is also evidence that over-arousal can inhibit learning, Sherwood (1965). Virtual Reality and Gaming, for example, can induce over-arousal and actually inhibit learning. The lesson here is that arousal to a level of optimal attention is advisable but not beyond that optimal level, as performance may fall.

Should you play music while learning

When thinking practically about attention it is useful to take a concrete example and a common one. Should you listen to music when learning? These are two separate, attentional processes both take up cognitive bandwidth and load. Remember that multitasking is largely a myth. 

There are those who extoll the Mozart Effect, extolling the virtues of playing Mozart to learners while they learn.This was sparked off by a paper in Nature by Rauscher, Shaw and Ky (1993), which showed a small improvement in spatial reasoning score (very specific), the effect lasted no longer than 15 minutes, then disappeared. The theory also disappeared, as several follow up studies could not replicate the effect. Rauscher herself, disclaimed the idea, saying that they had made no claim linking the playing of Mozart to intelligence. Steele in a 1999 put the nail in the coffin by showing that such effects are merely the result of short-term and temporary ‘enjoyment arousal'.

But education can never resist a fad and there are always people who can't let a good bandwagon pass, in this case Don Campbell, who published The Mozart Effect (1997) and The Mozart Effect for Children (2000). These books are, quite simply, bogus. His claims bear no resemblance to the actual research and, if you have this idea floating around in your brain, it’s largely down to him trade-marking the effect, then publishing these books, that were then taken up by ill-informed journalists. This is how it ended up in the minds of so many parents and teachers. It was even funded and applied in some states in the US, notably Georgia and Florida.

On the general proposition, that listening to music helps one learn, we also have to be very careful. There is a large and complex literature on this subject, testing the effect of music on various cognitive phenomena and there is some evidence that it improves mood, even motivation, but one must be careful when it comes to actual learning. 

Moreno and Mayer (2000) tried online learning with the following groups: 

Learning with music

Learning with sounds

Both

Neither

When retention and transfer were tested the groups with ‘music’ performed worse than those without music. This is a well known phenomenon where cognitive overload inhibits learning. It's to do with the overloading of working memory, especially with spoken words. One quick experiment you can do with your kids, or students, is to take a random page from a book on a subject they are unfamiliar with. Now tell them to read it in silence. Now choose another page and ask them to read it while repeating the word ‘boing-boing’ over and over. They will be unable to meaningfully learn from the text. The reason is the overloading of working memory, the phonological loop to be exact. Music takes up valuable bandwidth, therefore inhibits learning.

In this interesting study, silence is used as a control, along with the two major components in popular music - music and lyrics. Perham and Currie (2014) created four groups:

Silence
Music without lyrics

Music with lyrics they liked

Music with lyrics they disliked

Revising in silence was significantly better than revising while listening to music with lyrics (liked or disliked). Revising to 'music without lyrics' produced better scores than revising to 'music with lyrics'. In the end, revising in 'silence' predicted learning outcomes better than the other groups. Reaves et al., (2015) found that adults performed significantly worse on the task when exposed to any kind of music. 

There is lots of bad advice around study techniques that focus on superficial, low retention study methods and ignore attention, effort, retrieval and deliberate practice. No doubt some wag will tell us that music is good for those with an auditory learning style... that's also bullshit.

Design for limited attention

Users spend a lot of time online and that time is increasing. This means you are up against a lot of competition for attention, from some of the largest companies in the world who know about attention and spend significant sums fine tuning their service so that you don’t tune out.

The design process is full of trade-offs. You want their attention but not too much as too much arousal will hinder learning. There’s an optimal balance between getting and maintaining attention, which is limited, while not overloading the brain.

First the context and environment. Post-prandial dips in attention are well known, not too hot room temperatures and morning sessions all help, for physiological reasons. But motivation really matters a lot. Motivation is what largely leads to attention, not the other way round, as is sometimes assumed. What kills attention is cognitive overload, where too much is presented, dissonance or confusion sets in or the mind is over-stimulated. Screens with too much media – video, animation, text, sound and effects can easily lead to cognitive overload problems. Media rich does not mean mind rich. Even simple combinations such as asking learners to read text while hearing it spoken can be a problem as dividing attention is bad news. Less is almost always more.

Cues guide attention. Highlights, colour, arrows, bold, italics, underlining, font style, font size and ordering (generally top left to bottom right), layout, framing, movement and so on, are just some of many design techniques used to cue components. This matters more in learning than other tasks, as novices, those who don’t know much in any domain, really do need cued guidance to keep them attentive and learning.

The formatting of text, labels and graphics, so that they are intimately related, helps enormously with attention. Mayer and Moreno (2003) provide research supporting Nine ways to reduce cognitive load in multimedia learning. As we have separate systems for processing image and verbal material, dual-channels, each channel has limits to the amount they can process. Meaningful learning needs to build connections between images and verbal representations. However, cognitive overload commonly results in inhibiting learning. Mayer has published over 500 studies in online learning and has many solutions these problems. He is a must read for learning experience designers.

Mousavi, Low and Sweller (1995), found that explanatory audio can also relieve cognitive load. We use audio only introductions to online learning courses in WildFire, which gains attention, calm the learner and diminishes any anxiety they may feel about the coming learning experience. This is used, for example, by the Khan Academy in teaching mathematics and other subjects. There is no talking head, only the maths and audio. What really matters is the coordination of the elements, so that they are coherent and easily assimilated.

Organizers, an attention-focusing tool particular to learning, that summarise, often in an infographic, what is to come or what has just been covered, also helps cue and focus attention (Asubel; 1960). This gives the learner a high-level, bird’s-eye view of the content and helps with processing to and from long-term memory. It gives you a top-level schema into which you as a learner can fit the details. This leads into building schema in long-term memory to build expertise which takes time and effort by the learner. Schema are built by diversity of problems and effort. Worked examples (Paas & van Merrienboer, 1994), multiple perspectives and varied contexts and analogies all help, especially varying the surface features to embed the deeper principle.

Attention and technology

There has been a Renaissance of interest, dare I say attention, paid to attention in recent times, as tech companies vie for your attention online. UI and UX design on the web has become a marketable skill. Labs measure eye movements in relation to screen real estate and events, all in an attempt to first gain then hold your attention. This can increase the efficacy of online learning but the mistake is to think that it measures actual learning, retention, ability to recall or transfer. The danger is that such things are taken as proxies for learning.

Beyond this, the work being done in AI and mind-machine perception has also created a renewed interest in what it is to be attentive, as the automation of surveillance and data interpretation has become an essential part of robotics and object recognition. Object recognition is a remarkably attentive success in AI. The algorithms used to improve recognition are increasingly attentive and selective in terms of the goals they have to achieve. Exciting times.

Beyond this we have a renewed effort to use technology, in invasive and non-invasive forms. It has already led to clear health benefits for those with physical disabilities. The hope is that they will also help us identify how the brain learns, even act as aids to learning, optimising brain states, such as attention, even, in the far future interpreting and implanting memories.

Bibliography

James, W., 1890. Attention.

Pillsbury, W.B., 1908. Library of philosophy. Attention.

Broadbent, D.E., 2013. Perception and communication. Elsevier.

Kahneman, D., 1973. Attention and effort (Vol. 1063). Englewood Cliffs, NJ: Prentice-Hall.

Sherwood, J.J., 1965. A relation between arousal and performance. The American journal of psychology, 78(3), pp.461-465.

Rauscher, F.H., Shaw, G.L. and Ky, C.N., 1993. Music and spatial task performance. Nature, 365(6447), pp.611-611.

Steele, K.M., Dalla Bella, S., Peretz, I., Dunlop, T., Dawe, L.A., Humphrey, G.K., Shannon, R.A., Kirby, J.L. and Olmstead, C.G., 1999. Prelude or requiem for the ‘Mozart effect’?. Nature, 400(6747), pp.827-827.

Campbell, D., 2009. The Mozart effectHarper Collins.

Campbell, D., 2000. The Mozart Effect for ChildrenHarperCollins.

Perham, N. & Currie, H. (2014). Does listening to preferred music improve reading comprehension performance?. Applied Cognitive Psychology, 28, 279-284.

Reaves, S., Graham, B., Grahn, J., Rabannifard, P. & Duarte, A. (2015). Turn off the music! Music Impairs Visual Associative Memory Performance in Older Adults. The Gerontologist, 56, 569-577.

Moreno, R. and Mayer, R.E., 2000. A coherence effect in multimedia learning: The case for minimizing irrelevant sounds in the design of multimedia instructional messages. Journal of Educational psychology, 92(1), p.117.

Mousavi, S.Y., Low, R. and Sweller, J., 1995. Reducing cognitive load by mixing auditory and visual presentation modes. Journal of educational psychology, 87(2), p.319.

Mayer, R.E. and Moreno, R., 2003. Nine ways to reduce cognitive load in multimedia learning. Educational psychologist, 38(1), pp.43-52.

Ausubel, D.P., 1960. The use of advance organizers in the learning and retention of meaningful verbal material. Journal of educational psychology, 51(5), p.267.

Paas, F.G. and Van Merriënboer, J.J., 1994. Instructional control of cognitive load in the training of complex cognitive tasks. Educational psychology review, 6(4), pp.351-371.

 

Transfer - why is it ignored? Here's how to fix it...

Transfer is something that is often completely ignored in experience design. But what is the point of having learning experiences of they don’t transfer to actual application and performance? Learning experiences may not only fail to transfer but actually stop transfer.

You must design with transfer in mind and blends or learning journeys must move learning forward towards action, towards doing, towards practice and performance. No matter how much training you deliver, it can be illusory in the sense of not leading to transfer from cognitive change to actual performance, which in turn has impact on the organisation.

Doing and Practice are experiences. In fact without doing or practice it is unlikely to be retained long-term. Your design must move from experiences that match whatever type of learning you need, cognitive, psychomotor and affective, but practice and application experiences also matter. Your design should provide transfer pathways towards mastery, through actual doing and practice in the formal learning as well as practice and extension activities beyond the initial learning experiences.

Note that observable behaviours can be used but this is notoriously difficult, except in very formal apprenticeship-type learner journeys. Behaviour is notoriously difficult to measure and arguably behaviour must result in an impact in the organisation. It is better to go for data on KPIs, as they are commonly found in organisations. Note also that learning in workflow increases transfer as you are using it immediately. The training is proximate to the task. 

Knowledge can lie inert (Renkl et al 1996) and fail to transfer. Research focused on the idea that elements in the learning must be identical to those in the real world if transfer is to succeed (Singley & Anderson 1989). But it was Tulving who focused more on the retrieval of specific ‘cues’ in memory (Tulving & Thompson, 1973), recommending that such cues be designed into the learning experience, along with retrieval and spaced practice. We use this 'cues' technique in WildFire, where AI is used to create online learning, with cues, in minutes not months.

Near and Far transfer

A useful distinction is between ‘Near’ and ‘Far’ transfer Near transfer is where the task is simple and routine, such as learning how to ‘cut and paste’ in a word processor, where the contexts are similar. Far transfer involves troubleshooting or problem solving, using learned knowledge and skills, such as management skills or learning experience design!, as the contexts, where skills are applied, will be very hugely varied. Far transfer is what is often pointed to as a key component in and increasing number of future jobs, as routine tasks are automated.

Near transfer is easy to design for using methods such as varied worked examples, retrieval, deliberate, directed and spaced practice. 

Far transfer is far trickier. You will want to present the training in as realistic a way as possible, so that the cues can be embedded in the training. So, when doing management training, use real imagery or video within a real office. This suggests that we avoid cartoon representations or imagery that does not match the actual environment in which the training is to be applied. Flight simulators provide a good example of congruence between the training and environment. 

Note that it is often necessary to design learning experiences that are not too open and sophisticated, as the novice would suffer from overload and confusion (Caroll, 1992). In software training, for example, you narrow down the options with guided, step-by-step instruction, so as not to overwhelm the learner. The constraints may be loosened as expertise is built. Similarly in language learning, early learning will be of basic vocabulary and grammar, leading to guided and supported use and finally immersion.

Far transfer needs variation in context so that the principles can be applied in new situations as they arise. Variation in worked examples and applications by the learner will give them the flexibility to adapt what they learn to future problems, so support far transfer.

The classroom is often a poor environment for transfer, whereas on-the-job training provides real cues and context. Transfer is therefore strong argument for learning in the workflow, where you learn and do it immediately in the real world. What is needed is something that approximates the old apprenticeship model, now perhaps re-named as Blended Learning. A true Blended Learning experience integrates theory and practice, providing a process for progress, from novice to expert. The process may take weeks or months and not be restricted to a simple one or two day course or online learning experience. Learning needs transfer and transfer takes time. Specific features of an optimal Blended Learning design, may be experiences that allow you to apply what you learn in the real world, working through real case studies, models of expert performance, make changes and see how they affect the outcome, voice or articulate what you do as you do it to others and, of course, learning from mistakes. In other words learning experiences benefit from actual experiences.

Situated learning, where you learn in the job context, allows for actual results to act as a measure of success, if practiced in a safe environment. But pure situated practice can take a long time and is difficult to execute. It may, as Anderson et al. 1996 found, be an exaggeration to think that it is the optimal solution. Marshall 1995 found that a blend or combination of theory and examples works best and some job or workflow training. 

Spaced practice is one way to overcome delayed performance. Interestingly transfer may be increased by delayed feedback. There is ample evidence to show that spaced-practice will increase retention and transfer.

Druckman and Bjork 1994, showed that delaying feedback, allows learners to make mistakes, learn from those mistakes, so be careful in always giving immediate feedback. We do this in WildFire created content, where you get the presented learning experience, do retrieval practice but only at the end of the online module, where you get Red, Amber and Green feedback, are you asked to go back and correct your Red and Amber mistakes. Giving learners room to think, reflect and make mistakes will in crease retention, subsequent retrieval and transfer.

Technology and transfer

Technology also gives us the opportunities to practice and therefore transfer learning. Simulations have long provided powerful practice and transfer. Pilots really do learn how take-off, fly, land and cope with rare emergencies using simulators. So why are simulators not more commonly used in learning? Well, the pilot goes down with the plane. There is not much hyperbolic discounting when your imagination and the reality of your job takes you to 35,000 feet in 300 tons of metal and 600 passengers. 

One common error is to assume that full fidelity is always needed. This may not be possible on cost and it is vital that a distinction be made between ‘physical’ and ‘psychological’ fidelity. Most tasks require careful design around the psychological or cognitive processes in learning. In fact, low fidelity simulations can be as effective as very expensive high fidelity simulations, if the psychological fidelity is strong. Cox et al. 1969, showed that a cardboard box and photograph simulation could be as effective as a high-fidelity simulator. You often see simple, cockpit and control set-ups in pilot training facilities. I know of one trainer who was an expert in buying old junk equipment from planes for training. They had a hanger full of the stuff. It did the job.

Simple and mini-simulations may be useful for limited tasks. Branched scenario training where decision making is needed. This requires the careful selection of scenarios, based on their most likely occurrence in real life. This breathes life into learning as well as increasing the chances of transfer. Variety of scenario should match the variety of probable real life scenarios as much as possible. Probabilistic presentation of scenarios is also possible. I’ve been involved in high-end scenario training around conflict in healthcare that involved a wide range of scenarios from alcohol and drug users in A&E, to violent patients, colleagues and even those visiting patients. Customer service may require a careful selection of customer types. I designed an airport check-in scenario-based simulator that took a carefully calibrated selection of typical customer types; impatient business traveller, large family, nervous single traveller etc. and integrated the interpersonal skills with the software skills on using the check-in system as well as the physical skills of handling luggage and labels.

A LXP (Learning Experience Platform) may nudge and challenge you to do things in the workflow. When learning experiences are delivered, with an understanding of context and at the point of need, they have a far higher chance of transfer, as they are likely to be applied immediately and in a real world context. One can push out to learners, using predictive techniques, knowing that they are likely to need learning or react to them pulling content when they feel they need the support.

VR also offers opportunities for practice and therefore transfer in expensive, rare and dangerous environments. Oil rigs, inside vehicles, emergency incidents, down to the micro-level or out in space. VR can give high-fidelity environments and now, with haptic experience providing the physical feel of handling objects, as well as cable-free headsets, the freedom to move and experience worlds you may at some time encounter.

Technology evolves fast as we have seen how simulations, LXPs and VR can certainly enable transfer.

Making it work

We must be careful with transfer, as Weinbauer-Heidel who wrote What Makes Training Work, a book on transfer, warns us against transfer strategies if the ‘capability’ is NOT there. So often training cannot deliver on practice and application. She also recommends that NO course certificates are issued, unless transfer has been shown. For her, transfer needs to be levered at the personal, training and organisational levels. 

Personally, the learner has to want to follow through to action and transfer and be confident that they can perform. They must be made aware of the value of doing this in practice. This can be done by increasing relevance and proximity to the actual tasks, which is why learning in the workflow gives a powerful boost to transfer.

Training needs to be clear about what is expected in terms of application and doing, not get stuck in and just stop at pure theory. It means designing experiences that are relevant and practical to individuals, with practice included during and after the training.

Organisationally, learners should be expected to practice, with time available and support from line managers. There are many ways to do this, through nudges, challenges, projects, mentoring, deliberate practice schedules, short apprenticeships. Opportunities for practice experiences can be deliberately recommended and created, such as designing a website, using a spreadsheet or handling things in a lab, in safe environment with no bad consequences for the organisation. This is a matter of people to supervise, space and time to practice. In short, training and managers must take the horse to water AND make it drink.

Yet, what does designing a practice experience actually mean? At some point you must let go and had the reins over to the learner. It is a matter of suggesting, structuring or simulating practice and application, not once but repeatedly. We must understand what deliberate practice means, what spaced practice means.

The danger should be obvious, that we focus on shallow media presentation to get fun or engagement but what really matters is an understanding of how the mind of the learner actually learns. Learning is not like other experiences, such as entertainment, we have learned through many decades of research, that it is a complex issue, that needs to be worked on to optimise the learning experience and result in actual transfer and application of learning. 

Bibliography

Renkl, A., Mandl, H. and Gruber, H., 1996. Inert knowledge: Analyses and remedies. Educational Psychologist, 31(2), pp.115-121.

Singley, M.K. and Anderson, J.R., 1989. The transfer of cognitive skill (No. 9). Harvard University Press.

Tulving, E. and Thompson, D., 1973. Encoding Specificity and retrieval process in episodic process. Journal of Experimental Psychology, 87, pp.353-373.

Carroll, J.M., 1992. Minimalist documentation. Handbook of human performance technology.

 Anderson, J.R., Reder, L.M. and Simon, H.A., 1996. Situated learning and education. Educational researcher, 25(4), pp.5-11.

Marshall, S.P., 1995. Schemas in problem solving. Cambridge University Press.

Druckman, D.E. and Bjork, R.A., 1994. Learning, remembering, believing: Enhancing human performance. National Academy Press.

Cox, J.A., Wood Jr, R.O. and Thorne, H.W., 1965. Functional and Appearance Fidelity of Training Devices for Fixed-Procedures Tasks (No. HUMRRO-TR-65-4). GEORGE WASHINGTON UNIV ALEXANDRIA VA HUMAN RESOURCES RESEARCH OFFICE.

Weinbauer-Heidel, I. 2018. What Makes Training Really Work 

Let's move on from 'Unintelligible Intelligences' - IQ, Multiple Intelligences, Emotional Intelligence, Artificial Intelligence...

 Eysenck (1916-1997) - IQ, assessment and personality...

Binet, the man responsible for inventing the IQ (intelligence quotient) test, never saw it as being a ‘fixed’ for individuals. Sadly, his waning was ignored as education, keen as ever on selection, sought out single measure for intelligence. The 20^th C was dominate at first by the Intelligence Quotient, forever associated with Eysenck (1916-1997). This widened out towards the end of the century, first with Gardner’s Multiple Intelligences, then Goleman’s Emotional Intelligence. None have stood the test of scrutiny and time. With a renewed interest in Artificial Intelligence, as we moved into the 21^stcentury, here has been renewed interest in the word ‘intelligence’. As the measurement of man has become a growing obsession with ever widening definitions of intelligence, unfortunately, much of it was damaging, badly researched and, at times, used for nefarious purposes. As IQ morphed into MI then EQ and AI the same mistakes were made time after time.

Hans Eysenck was the figure around whom much of the IQ debate figured in the 20^th century. What is less well known is his work on personality types and his opposition to psychoanalysis and Freud in particular, explained in The Decline and Fall of the Freudian Empire.

A controversial figure, he put forward the proposition that intelligence had a hereditary component and was not wholly, socially determined. Although this area is highly controversial and complex, the fact that genetic heritability has some role has become the scientific orthodoxy. What is still controversial is the definition and variability of ‘intelligence’ and the role that intelligence and other tests have in education and training. The environment has been shown to play an increasing role but the nature/nurture debate is a complex area, now a rather esoteric debate around the relevance of different statistical methods.

IQ theory has come under attack on several fronts. Stephen Jay Gould’s 1981 book The Mismeasure of Man is only one of many that have criticised IQ research as narrow, subject to reification (turns abstract concepts into concrete realities) and linear ranking, when cognition is, in fact, a complex phenomenon. IQ research has also been criticised for repeatedly confusing correlation with cause, not only in heritability, where it is difficult to untangle nature from nurture, but also when comparing scores in tests with future achievement. Class, culture and gender may also play a role and the tests are not adjusted for these variables. Work by Howe and Eriksson and others explains extraordinary achievement as being the result of early specialisation and a focused investment in over 10,000 hours of practice and not measurable IQ.

The focus on IQ, a search for a single unitary measure of the mind, is now seen by many, as narrow and misleading. Most modern theories of mind have moved on to more sophisticated views of the mind as with different but interrelated cognitive abilities. More modular theories and theories of multiple intelligence have come to the fore. Sternberger’s three-part (analytic, creative, practical) was followed by Gardner’s eight intelligences in Frames of Mind.

Goleman’s Emotional Intelligence (EQ), reflected in other more academic and well researched work, also challenged the unitary theory of intelligence, with its emphasis on the ability to harness emotion in self-awareness, thinking, decision making and in dealing with others. It is not that IQ is the antithesis of EQ, they are merely different. However, even Gardner and Goleman have come under criticism for lacking rigour. In general, however, educational systems in many countries have been criticised for failing to teach this wider set of skills that many now agree are useful in adult life.

Eysenck worked with Cyril Burt at the University of London, the man responsible for the introduction of the standardised 11+ examination in the UK, enshrined in the 1944 Butler Education Act, an examination that, incredibly, still exists in parts of the UK. Burt was subsequently discredited for publishing largely in a journal that he himself edited, falsifying, not only the data upon which he based his work, but also co-workers on the research.

This is just one of many standardised tests that have become common in education but many believe that tests of this type serve little useful purpose and are unnecessary, even socially divisive. On the other hand supporters of test regimes point towards the meritocratic and objective nature of tests. Some, however, argue that standard tests have led to a culture of constant summative testing, which has become a destructive force in education, demotivating and acting as an end-point and filter, rather than a useful mark of success. Narrow academic assessment has become almost an obsession in some countries, fueled by international pressure from PISA.

Interestingly, when measuring IQ, the Flynn Effect, taken from military records, shows that scores have been increasing at the rate of about 3 points per decade and there is further evidence that the rate is increasing This was used by Stephen Johnson in his book Everything bad is Good for You to hypothesize that exposure to new media is responsible, a position with which Flynn himself agrees. This throws open a whole debate and line of research around the benefits of new media in education and learning. Highly complex and interactive technology may be making us smarter. If true, this has huge implications for the use of technology in education and society in general.

Unfortunately, Eysenck and many other psychologists, throughout the middle of the 20^th century may have focused too much on narrow IQ tests. This has led to some dubious approaches to early assessment, such as the 11+, that has, to a degree, socially engineered the future educational opportunities and lives of young people. IQ theorists like Eysenck tended to focus on logical and mathematical skills, to the detriment of other abilities, leading some to conclude that education has been over-academic. This, they argue, has led to a serious skew on curricula, assessment and the funding of education to the detriment of vocational and other skills.

 

Multiple Intelligences… uncanny resemblance to current curriculum subjects…

Howard Gardner’s theory of multiple intelligences opposes the idea of intelligence being a single measurable attribute. His is a direct attack on the practice of psychometric tests and behaviourism, relying more on genetic, instinctual and evolutionary arguments to build a picture of the mind. He also disputes the Piaget notion of fixed developmental stages, claiming that a child can be at various stages of development across different intelligences.

For Gardner, intelligence is “the capacity to solve problems or to fashion products that are valued in one or more cultural setting” (Gardner & Hatch, 1989). To identify he nature of intelligence he sought evidence from repots of brain damage showing isolated abilities, the existence of idiot savants, prodigies and other exceptional individuals, an identifiable core operation or group of operations, specific development histories with definable 'end-state' performances, an evolutionary history (at least plausible), evidence from experimental psychology, psychometric findings and the ability to express such intelligences in a symbolic way. In other words, he took a holistic, not a purely experimental or scientific, approach to evidence.

What popped out of studying these criteria was a list of eight ‘intelligences’. To be fair this original list of eight has developed over time but his thoughts on what constitute intelligence have developed over time, as the theory was scrutinized. It opened up the meaning of intelligence to purely rational and logical abilities, which were long held as the essential measures of intelligence.

Loosely speaking, the first two have been typically valued, some would say over-valued, in education; the next three are often associated, but not exclusively, with the arts; the final three are what Gardner called 'personal intelligences':

1. Linguistic: To learn, use and be sensitive to language(s).

2. Logical-mathematical: Analysis, maths, science and investigative abilities.

3. Musical: Perform, compose and appreciate music, specifically pitch, tone and rhythm.

4. Bodily-kinaesthetic: Co-ordination and use of whole or parts of body.

5. Spatial: Recognise, use and solve spatial problems both large and confined.

6. Interpersonal: Ability to read others’ intentions, motivations, desires and feelings.

7. Intrapersonal: Self-knowledge and ability to understand and use one’s inner knowledge.

8. Naturalist: Ability to draw upon the immediate environment to make judgements.

These intelligences complement each other, work together as blends of intelligences. Individuals bring multiple subsets of these intelligences to solve problems.

Gardner also wrote a full set of recommendations on the use of multiple intelligence theory in schools in The Unschooled Mind, Intelligence Reframed, and The Disciplined Mind, to look at how the theory can be applied in education. As John White observed, one problem with the theory is that it bears an uncanny resemblance to the current curriculum subjects, opening it up to the charge that it reflects what we want to teach, rather than having cognitive certainty. It can look like a simple defence of the classic curriculum.

This has led to a broader more holistic view of education, being less rigid about abstract and academic learning. It demands knowledge of these intelligences among teachers, an aspirational approach to learning, more collaboration between teachers of different disciplines, better and more meaningful curriculum choices and a wider use of the arts.

Many have also criticized the choices as being based on general observations, subject to personal and cultural bias, rather than universal cognitive abilities based on empirical evidence. There is always the problem with identifies ‘intelligences’ such as these not mapping onto the many different forms of cognitive functions, sensory, memory and others. In many of these supposed intelligences, multiple and complex cognitive operations are at work.

Like many forms of measurement in education, from learning styles, through to MBTI and intelligences, the theory can be criticized as it leads to stereotyping and pigeon-holing learners, pushing them towards narrower roads that they would otherwise have been exposed to. It may be their perceived weaknesses that should be addressed not necessarily the most obvious strengths. Like learning styles, it may do more harm than good.

Gardner himself was shocked and often frustrated by the way multiple intelligences was crudely applied in schools, among “a mish-mash of practices…Left Right brain contrasts….learning styles….NLP, all mixed up with dazzling promiscuity”. Some schools in the US even redesigned the whole curriculum, classrooms and entire schools around the theory. His point was that teachers should be sensitive to these intelligences, not to let them prescribe all practice. In his 2003 paper Multiple Intelligences after Twenty Years, for the American Educational Research Association, you could feel his frustration, when he writes, “I have come to realize that once one releases an idea – “meme” – into the world, one cannot completely control its behaviour – any more than one can control those products of our genes we call children.”. Like many of these theories, the problem was its simplification and seductiveness. It gave us permission to say anything goes. Rather than promoting a focus on a wider, but still rigorous and relevant curriculum, it was used to confirm he view that here are almost innate ‘talents’ and that young people simply express those through interest. On the other hand it also provided some defence against those who want to labour away at maths all day at the expense of many other subjects or get overly obsessed with STEM subjects.

Like many theories, they develop over time and many teachers who quote and use the theory are unlikely to have fully understood its status and further development by Gardner himself. Few will have understood that it is not supported in the world of science, despite the perception by educators that it arose from that source. Gardner’s first book, Frames of the Mind: The Theory of Multiple Intelligences (1983) laid out the first version of the theory, followed 16 years later by a reformulation in Intelligence Reframed (1999), then again in Multiple Intelligences after Twenty Years (2003). Few have followed its development after 1983 or the critiques and Gardner’s subsequent distancing of the theory from brain science.

Lynn Waterhouse laid out the lack of scientific evidence for the theory in Multiple Intelligences, the Mozart Effect, and Emotional Intelligence: A Critical Review in Educational Psychologist. In many areas of learning, such as reason, emotion, action, music, language and so on, are characterised by their intersecting, distributed and complex patterns of activity in the brain. Islands of functional specificity are extremely rare. In short, Gardner seems to suffer from conceptual invention and simplicity. In short, brain science appears not support the theory. Gardner responded to this absence of neurological evidence for his separate 'intelligence' components, by redefining his intelligences as “composites of fine-grained neurological sub-processes but not those sub-processes themselves”(Gardner and Moran, 2006). Pickering and Howard-Jones found that teachers associate multiple intelligences with neuroscience, but as Howard-Jones states, “In terms of the science, however, it seems an unhelpful simplification as no clearly defined set of capabilities arises from either the biological or psychological research”. However, Project SUMIT (Schools Using Multiple Intelligences Theory) does claims to have identified real progress across the board in schools that have indeed been sensitive to Gardner’s theories. The problem is that Gardener claims that the science has yet to come, but teachers assume it is already there and that the theory arose from the science.

The appeal of Gardner’s Multiple Intelligences is obvious. It can take on the mantle of science, even neuroscience, and claim to have reinforced the view, not that specific knowledge and skills mater but that all knowledge and skills matter. It plays to the socially constructivist idea that anything goes, in a sea of constructions. Critics are right in holding his feet to the fire of experimental rigour and science, to show that these are indeed identifiable ‘intelligences’ and not just his, or the current educational system’s curricular preferences. They also seem to support the popular movement towards separate, so-called 21^st century skills, as. A generic set of skills that can be taught beyond knowledge. In other words it chimes with other popular, and possibly erroneous myths in learning. On the other hand, while the theory may be rather speculative, his identified intelligences represent real dispositions, abilities, talents and potential, which many schools could be said to downgrade or even ignore. 

So far it has been one step forward, getting away from the idea of a single measure of intelligence, as a core entity in the mind, towards a more general theory of multiple entities and measures of intelligence. The problem is that this step wasn’t really solid enough to remain stable. It failed to be supported by solid evidence.  But we have a glimpse here of the dangers of the word ‘intelligence’, its tendency to invite forms of essentialism. Like the allure of gold it attracts ‘miners of the mind’ looking for this singular intelligence or multiple set of essential intelligences. It turns out that what is mined is Fool’s Gold. It may look like gold but, on examination, it is rigid and non-malleable.

The ‘intelligence’ movement, then took a surprising turn, as it swung into the affective or emotional territory. IQ ignored this, Multiple Intelligences tried to widen out to include interpersonal skills but the emotional side was still outside of their scope. So along came another form of intelligence ‘emotional intelligence’.

 

Emotional Intelligence – is it even a 'thing'?

Michael Beldoch wrote papers and a book around emotional intelligence in the 1960s and is credited with coming up with the term. But it was Daniel Goleman’s Emotional Intelligence (1995) that launched another education and training tsunami. Suddenly, a newly discovered set of skills, classed as an ‘intelligence’ could be used to deliver yet another batch of courses.

Emotional Intelligence (EQ) is seen by Goleman as an a set of competences that allow you to identify, assess, and control the emotions which you and others have.

He identified five types of Emotional Intelligence: 

Self-awareness: Know your own emotions and be aware of their impact on others

Self-regulation: Manage your own negative and disruptive emotions

Social skill: Manage emotions of other people

Empathy: Understand and take into account other people’s emotions

Motivation: Motivate yourself

For Goleman, these emotional competencies can be learned. They are not entirely innate, but learned capabilities that must be worked on and can be developed to achieve outstanding performance. 

We now have some good research on the subject which shows that the basic concept is flawed, that having EI is less of an advantage than you think. Joseph et al (2015) published a meta-analysis of 15 carefully selected studies, easily the best summary of the evidence so far. What they found was a weak correlation (0.29) with job performance. Note that 0.4 is often taken as a reasonable benchmark for evidence of a strong correlation. This means that EI has a predictive power on performance of only 8.4%. Put another way, if you are spending a lot of money and raining effort on this, it is largely wasted. The clever thing about the Joseph paper was their careful focus on actual job performance, as opposed to academic tests and assessments.

What became obvious as they looked at the training and tools, was that there was a bait and switch going on. EI was not a thing-in-itself but an amalgam of other things, especially personality measures. Indeed, when they unpacked six of the EI tests, they found that many of the measures were actually personality measures, such as conscientiousness, industriousness and self-control. These had been literally lifted from other personality tests. So, they did a clever thing and ran the analysis again, this time with controls for established personality measures. This is where things got really interesting. The correlation between EI and job performance dropped to a shocking -0.2.

Like many fads in HR, such as learning styles, an intuitive error lies at the heart of the fad. It just seems intuitively true that people with emotional sensibility should be better performers but a moment’s thought will make you realize that many forms of performance may rely on many other cognitive traits and competences. In our therapeutic age, it is all too easy to attribute positive qualities to the word ‘emotional’ without really examining what that means in practice. HR is a people profession, people who genuinely care, but when they bring their biases to bear on performance, as with many other fads, such as learning styles, Maslow, Myers-Briggs, NLP and mindfulness, emotion tends to trump reason. When it is examined in detail, EI like these other fads, falls apart. Eysenck, the doyen of intelligence theorists, dismissed Goleman’s definition of ‘intelligence’ and thought his claims were unsubstantiated.

EI tests

Goleman’s claims, that general EI was twice as useful as either technical knowledge, or general personality traits, has been dismissed as nonsense, as is his claim that it accounts for 67% of superior, leadership performance. This undermines lots of Leadership training, as EI is often used as a major plank in its theoretical framework and courses. Føllesdal looked at test results (MSCEIT) of 111 business leaders and compared these with the views of those same leaders by their employees. Guess what – there was no correlation.

Tests often lie at the heart of these fads, as they can be sold, practitioners trained and the whole thing turned into pyramid selling. Practitioners, in this case are sometimes called ‘emotional experts’, who administer and assess EI tests. However, the main test, the MSCEIT, is problematic. First, the company administering the tests (Multi-Health systems) was found by Føllesdal to be peddling a pig with lipstick. To be precise, 19 of the 141 questions were actually being scored wrongly. They quietly dropped the scoring on these questions, while keeping them in the test. Reputations had to be maintained. More fundamentally, the test is weak, as there are no correct answers, so it is not anchored in any objective standard. As a consensus scored test, it is foggy.

Way forward?

Emotional Intelligence has all the hallmarks of other HR fads – the inevitable popular book, paucity of research, exaggerated claims, misleading language, the test, ignoring research that shows it is largely a waste of training time. This is not to say that ‘emotion’ has no role in competences or learning. Indeed, from Hume To Haidt, we have seen that reason is often the slave of the passions. Gardner’s mistake was to over-rationalise emotion. In particular, his use of the word ‘intelligence was misleading.

 

Education became fixated with the search and definition of a single measure of intelligence – IQ. The main protagonist being Eysenck and it led to fraudulent policies, such as the 11+ in the UK, which is still used for selection into schools at age 11. It was promoted on the back of fraudulent research by Cyril Burt. Out of this obsession also came the language of the gifted and talented, still popular in education, despite the fact that the measures are flawed.

Many have criticised IQ research as narrow in definition. This is a key point. Cognitive science has succeeded in unpacking many of these complexities without reducing them to singular measures or short lists. The focus on IQ, a search for a single, unitary measure of the mind, even a small set of such measures, is now seen by many as narrow and misleading. Gardener tried to widen its definition into Multiple Intelligences (1983) but this is weak science and lacks any real rigour. 

Goleman wanted to add another, Emotional Intelligence, but this turned out to be little more than a marketing slogan. The search for ‘intelligence’ still suffers from a form of academic essentialism. Most modern theories of mind have moved on to more sophisticated views of the mind as with different but interrelated cognitive abilities. 

Goleman’s confusing ‘intelligence’ or ‘competences’ with personality traits is telling. Eysenck also contributed (with his wife) to the area of personality traits with idea that personality can be defined in terms of psychoticism, extraversion and neuroticism. This provided the basis for the now widely respected OCEAN model proposed by Costa & McCrae:

Openness

Conscientiousness

Extraversion

Agreeableness

Neuroticism

Eysenck rejected the Costa & McCrae model but in the end it has become the more persuasive theory. This well researched area of ‘personality types’ has largely been ignored in learning, in favour of the more faddish ‘learning styles’ theory. However, it has been argued that this type of differentiation is far more useful when dealing with different types of learners than the essentialism of Eysenck, Gardner and Goleman.

Why we need to drop the word ‘intelligence’

More recently, the rise of AI has produced a lot of debate on what constitutes ‘intelligence’. I discuss this in my book ‘AI for Learning’. Turing’s seminal paper Computing Machinery & Intelligence (1950), along with its nine defences, set the standard on whether machines can think and be intelligent. Yet the word ‘intelligence’ is never mentioned in his sense in the actual paper. But it was John McCarthy who invented the term at the famous Dartmouth Conference in 1956, that is seen as the starting point of the modern AI movement.

We would do well to abandon the word ‘intelligence’, as it carries with it so much bad theory and practice. Indeed AI has, in my view, already transcended the term, as it had success across a much wider sets of competences (previously intelligences), such as perception, translation, search, natural language processing, speech, sentiment analysis, memory, retrieval and other many other domains. All of this was achieved without consciousness. It is all competence without comprehension.

Machine learning has led to successes all sorts of domains beyond the traditional field of IQ and human ‘intelligences’. In many ways it is showing us the way, going back to a wider set of competences that includes both ‘knowing that’ (cognitive) and ‘knowing how’ (robotics) to do things. This was seen by Turing as a real possibility and it frees us from the fixed notion of intelligence that got so locked down into human genetics and capabilities. We can therefore avoid the term ‘intelligence(s)’ thereby avoiding the anthropomorphism around transferring human ideas around intelligence on to non-comprehending, but competent, performance. ‘Intelligence’ embodies too many assumptions around conscious comprehension in a field where man is NOT the measure of all things.

Beyond brains

The brain is the organ that named itself and created all that we are discussing but it is a odd thing. It takes over 20 years of education before it is even remotely useful to an employer or society. To attribute ‘intelligence’ to he organ is to forge that, compared to machines, it can’t pay attention for long, forgets most of what you teach it, is sexist, racist, full of cognitive biases, sleeps 8 hours a day, can’t network, can’t upload, can’t download and, here’s the fatal objection -  it dies. This should not be the gold standard for intelligence, as it is an idiosyncratic organ that evolved for circumstances other than those we find ourselves in.

Let’s take this idea further. Koch (2014) claimed that ALL networks are, to some degree ‘intelligent’. As the boundary for consciousness and intelligence changed over time to include animals, indeed anything with a network of neurons, he argues that intelligence is a property that can be applied to any communicating network. As we have evidence that intelligence is related to networked activity, whether these are brains or computers, could intelligence be a function of this networking, so that all networked entities are, to some degree, intelligent? Clark and Chalmers (1998) in The Extended Mind, laid out the philosophical basis for this approach. This opens up the field for definitions of ‘intelligence’ that are not benchmarked against human capabilities or speciesism. If we consider the idea of competences residing in other forms of chemistry and substrates, and see algorithms and their productive capabilities, as being independent of the base materials in which they arise, then we can cut the ties with the word ‘intelligence’ and focus on capabilities or competences. 

Few would argue that AI has progressed faster than expected, with significant advances in machine learning, deep learning and reinforcement learning.  In some cases the practical applications clearly transcend human capabilities and competences in all sorts of fields, calculation, image recognition, object detection and the may fruits of natural language processing, such as translation, text to speech, speech to text. We do not need to see ‘intelligence’ as the sun the centre of this solar system. The Copernican move is to remove this term and replace it with competences and look to problems that can be solved without comprehension. The means to ends are always means, it is the ends that matter. 

What is wonderful here is the opening up of philosophical issues around the idea of ‘intelligence(s)’. We are far from the existential risk to our species that many foresee but there are many more near-term issues to be considered. Ditching old psychological relics is one. Artificial smartness is with us it need not be called 'intelligent'.

Bibliography

Eysenck, H.J. (1967) The Biological Basis of Personality. Springfield, IL: Charles C. Thomas.

Eysenck, H.J. (1971) The IQ Argument: Race, Intelligence, and Education. New York: Library Press.

Eysenck, H.J. (1985) Decline and Fall of the Freudian Empire

Eysenck, H.J. & Eysenck, S.B.G. (1969). Personality Structure and Measurement. London: Routledge.

Gould, S. J. (1981).The mismeasure of man. New York: Norton.

Beldoch, M. and Davitz, J.R., 1964. The communication of emotional meaning. McGraw-Hill.

Gardner, H. (1983). Frames of mind: The theory of multiple intelligences. New York: Basic Books.

Goleman, D. (1995). Emotional intelligence. New York: Bantam Books.

Howe, M. J. A. (1999). Genius explained. Cambridge, U.K: Cambridge University Press.

Johnson, S. (2005). Everything bad is good for you. London: Allen Lane.

McCrae, R. R., & Costa, P. T. (2003). Personality in adulthood: A five-factor theory perspective. New York: Guilford Press.

Bloom (1956). Bloom's Taxonomy of the Cognitive Domain.

Dennett, D. (1995). Consciousness Explained.

Clark and Chalmers (1998) The Extended Mind

Dreyfus, H., & Dreyfus, S. (1997). Why Computers May Never Think Like People. Knowledge Management Tools, 31-50.

Ebbinghaus, H. (1908). Psychology: An elementary textbook. New York: Arno Press.

Gardner, H. (1983) Frames of mind: The theory of multiple intelligences, New York: Basic Books.

Frey B.C. Osborne M.A. (2013). The Future of Employment, Oxford Martin School.

Harari, Y.N. (2016). Homo Deus: A Brief History of Tomorrow. Harvill Secker, London.

Haugland, J. (1997). Mind design II: Philosophy, psychology, artificial intelligence. Cambridge, MA: MIT Press.

Kahneman, D. (2011). Thinking, fast and slow. New York: Farrar, Straus and Giroux.

Koch (2014). "Is Consciousness Universal". Scientific American Mind.

Searle, J. (1980). Minds, Brains and Programs. The Behavioral and Brain Sciences.3, pp. 417–424. (1980)

Susskind, R., & Susskind, D. (2015). The Future of the Professions. Oxford: Oxford University Press.

Turing, A. M. (1950). I.—Computing Machinery And Intelligence. Mind, LIX(236), 433-460.