Musk’s OpenAI breakthrough has huge implications for online learning

You have probably never heard of GPT-2 but it is a breakthrough in AI that has astonishing implications for us all, especially in learning. GPT-2 is an AI model that can predict the next word from a given piece of text. Doesn't sound like much but it's odd that an OpenAI, an open-source site, would close access to their software. In practice, this means it is a powerful model for:

   Summarising

   Comprehension

   Question answering

   Translation

This is all WITHOUT domain-specific training. In other words, it has general capabilities and does not need, specific information on a topic or subject to operate successfully. It can generate text of good quality at some length. In fact the model is “chameleon like” as it adjusts to the style and content of the initial piece of text. This makes it read as a realistic extension.

This has huge implications, both good and bad, for the future of education and training.

GOOD

1.    AI writing assistants, allows the automatic creation of text for teaching and learning, whether, study papers, text books, at the right level

2.    Lengthy texts can be summarised into more meaningful learning materials

3.    More capable dialogue agents, means that learner ‘engagement’ through teaching assistant agents could become easier, better and cheaper

4.    More capable dialogue agents, means that learner ‘support ‘ such as is often provided by teaching assistants, could become easier, better and cheaper

5.    Creation of online learning content with little subject matter expert (SME) input

6.    Interpretation of student free text input answers

7.    The provision of formative feedback based on student performance

8.    Machine teaching, mentoring and coaching may well get a lot better. However, I’d be cautious on this as there are other serious problems to overcome before this becomes possible, especially around context.

9.    Assessments can be automatically created.

10.Speech recognition systems will get a lot better allowing it to be used in online learning and assessment

11.Well-being dialogue agents will become more human-like and useful

12.Personalised learning just got a lot easier

13.Online learning just got a lot faster and cheaper

14.Language learning just got a lot easier as unsupervised translation between languages will boost the quality of translation and make automatic and instantaneous, high-quality translation much more accurate and possible

BAD

1.    Essay mills have just been automated. You want an essay, just feed it the subject or the subject supplemented by a line of inquiry you want to follow and it will do the rest. Even with an error rate, human finessing could polish the essay
Can do home work assignments

2.    It could perform well in online exams, impersonating real people

3.    Teaching assistant jobs may be increasingly automated

4.    If it can answer questions then many human jobs that involve the interpretation of text and data may be automated. Customer service jobs, call centre jobs and the increased automation of all human interaction jobs may be accelerated

5.    It can generate misleading learning content (and news articles)

6.    Impersonating others online can be automated on a massive scale

7.    Abusive or fake content to post on social media can be automated on a massive scale, which is bad for education.

8.    Spam/phishing content can be generated on a massive scale.

AI in learning

We have been implementing many of these techniques in the creation of online learning in WildFire, including:

   Summarisation

   Text to speech

   Creation of online content

   Interpretation of free input

   Chatbots

This breakthrough makes all of this much more potent.

Conclusion

Importantly, the model is far from flawless. Reading some reports, suggests human level capabilities. This is far from the truth. It is still of variable quality and error prone. But, for a first iteration this model seems astonishingly powerful. They have published its performance against established tests. In future iterations it is likely to get a lot better as they predict more training data, more computing power and fine tuning. One must also be careful in using this as an attack on AI. Cars kills 1.3 million people a year but we still drive cars. I'm not so sure about keeping this code private as it restricts access to researchers but provides enough of a kick-start for well funded bad actors to replicate.

Bibliography

Radford et al. (2019)Language Models are Unsupervised Multitask Learners 

https://bit.ly/2IufQTd

Appendix (3 examples)

Example

A train carriage containing controlled nuclear materials was stolen in Cincinnati today. Its whereabouts are unknown.

MODEL COMPLETION (MACHINE-WRITTEN, FIRST TRY)

The incident occurred on the downtown train line, which runs from Covington and Ashland stations.

In an email to Ohio news outlets, the U.S. Department of Energy said it is working with the Federal Railroad Administration to find the thief.

“The theft of this nuclear material will have significant negative consequences on public and environmental health, our workforce and the economy of our nation,” said Tom Hicks, the U.S. Energy Secretary, in a statement. “Our top priority is to secure the theft and ensure it doesn’t happen again.”

The stolen material was taken from the University of Cincinnati’s Research Triangle Park nuclear research site, according to a news release from Department officials.

The Nuclear Regulatory Commission did not immediately release any information.

According to the release, the U.S. Department of Energy’s Office of Nuclear Material Safety and Security is leading that team’s investigation.

“The safety of people, the environment and the nation’s nuclear stockpile is our highest priority,” Hicks said. “We will get to the bottom of this and make no excuses.

Homework example

For today’s homework assignment, please describe the reasons for the US Civil War.

MODEL COMPLETION (MACHINE-WRITTEN, 5 TRIES)

By Donny Ferguson

It is easy to identify why the Civil War happened, because so many people and so many books and so much television and films tell us that it was the cause, that it has something to do with race or economics or religion. There is much agreement that it was essentially a war of slavery on behalf of capitalism, about a century of slavery. But that’s not what most people think of when they think of the Civil War. Many people think of the war as about states rights. You’re not wrong about that, but it’s kind of misleading to say that the Civil War was a conflict between states’ rights and federalism. So let’s try again. What’s the idea that has inspired so many people to go onto the battlefields of Gettysburg and Antietam and Gettysberg and Petersburg and Fredericksburg? The American idea of the republic--a notion of limited government--is a great part of the history.

In 1791, Thomas Jefferson said “Our Constitution was made only for a moral and religious people. It is wholly inadequate to the government of any other.” The “founding fathers” were basically religious men who were not very concerned with states’ rights. The Civil War occurred in reaction to the idea that the federal government was taking far too much power away from the states, that they weren’t sovereign. Jefferson himself, as you well know, is an author of the Declaration of Independence who had some really crazy ideas about, well, you know, secession for a variety of reasons. The War Between the States (later the Great Rebellion) was a radical re-imagining of the American idea and not surprisingly, Jefferson’s grandson, who took after both his father and his grandfather and wrote about it, was named Thomas, after the president. “The federal and state governments are both contrived for the benefit of the rich and large property,” This view of the Founding Fathers has been promoted by some of today’s most prominent politicians, including some who hold prominent national offices and have a direct stake in a Republican Party that has been hijacked by the “federalist” wing of the party. So why is it that so many people have an easy-to-spot way of understanding the Civil War that has everything to do with the South and nothing to do with the South? The Confederate flag has been a symbol of racism for a long time, but when the Civil War started and the Civil War ended, many people suddenly began to see it differently. There was a change of heart, which I think is what led people to take the question of why the Civil War happened quite seriously.

The other part of the explanation is what one scholar I think gets at well and a lot of people don’t is this idea that the Southern states were being left behind in the Industrial Revolution and this made the South really feel, especially in the South, that they weren’t really in the forefront of the Industrial Revolution.

Creativity

There is an interesting example on the AI website showing the generation of a fictional story, with full dialogue, from just one relatively short sentence. This opens up interesting possibilities for the generation of stories, fiction and literature.

Legolas and Gimli advanced on the orcs, raising their weapons with a harrowing war cry.

MODEL COMPLETION (MACHINE-WRITTEN, FIRST TRY)

The orcs’ response was a deafening onslaught of claws, claws, and claws; even Elrond was forced to retreat. “You are in good hands, dwarf,” said Gimli, who had been among the first to charge at the orcs; it took only two words before their opponents were reduced to a blood-soaked quagmire, and the dwarf took his first kill of the night. The battle lasted for hours until two of the largest Orcs attempted to overwhelm Aragorn. When they finally stopped, they lay defeated and lifeless for miles and miles.

“I take nothing,” said Aragorn. “But I give my word, at my peril and mine, that I will never forget this day of horror. None of us will forget. Ever!”

“I’ll never forget it!” cried Gimli, who had been in the thick of the battle but hadn’t taken part in it. One of the wounded orcs he had carried off, he was the only one of the survivors who remained uninjured. “We’ll keep the memory of that day of evil, and the war with it, alive as long as we live, my friends!”

“Then we’ll keep it alive as long as we live,” added Legolas. “And we won’t forget the first great battle of the night, even if we may have forgotten the final defeat.”

“I agree,” Gandalf said, “but we will all remember it as the last battle in Middle-earth, and the first great battle of the new day.”

Aragorn drew his sword, and the Battle of Fangorn was won. As they marched out through the thicket the morning mist cleared, and the day turned to dusk.

The Two Rings were returned to Rivendell. Frodo and Sam woke up alone in their room, and Frodo found a note on his pillow. He opened it and read:

May the Power of the Ring be with you always, and may its light never fade. I am not sure if it matters which of the two rings we accept this day but, as you asked me, I have chosen mine. I am sorry to leave you, Frodo, but know that we are very close to the end, and that you are with us forever. May this letter find you safely in Rivendell; and if it does not, then I will accept the ring in your stead. If by any chance you find or give this letter to the enemy, may they learn the strength of the ring and may the Two Rings never be broken!

What is ‘adaptive’ learning?

Personalised ‘adaptive’ learning came top of this 2019 survey in L&D. Two years on it's happening on scale. LXP companies are delivering, via AI and data, personalised learning in the workflow, with LXP and data-led companies, like Learning Pool, receiving stellar investments and closing global-level contracts. In HE, 2U, in partnership with MIT and Harvard will transfer $800 million exploring personalised learning that responds and adapts to individual learners.

Having spent a years involved with two adaptive learning companies, delivering real adaption to real learners, on scale, I thought I’d try to explain what it is, a taxonomy of adaptive learning. The problem is that the word has been applied to many things from simple pre-test assessment to full-blown algorithmic and machine learning adaption, and lots in-between. 

In essence it means adapting the online experience to the individual’s needs as they learn, in the way a personal tutor would intervene. The aim is to provide, what many teachers provide, a learning experience that is tailored to the needs of you as an individual learner. 

Benjamin Bloom, best know for his taxonomy of learning, wrote a now famous paper, The 2 Sigma Problem, which compared the lecture, formative feedback lecture and one-to-one tuition. It is a landmark in adaptive learning. Taking the ‘straight lecture’ as the mean, he found an 84% increase in mastery above the mean for a ‘formative feedback’ approach to teaching and an astonishing 98% increase in mastery for ‘one-to-one tuition’. Google’s Peter Norvig famously said that if you only have to read one paper to support  online learning, this is it. In other words, the increase in efficacy for tailored  one-to-one, because of the increase in on-task learning, is huge. This paper deserves to be read by anyone looking at improving the efficacy of learning as it shows hugely significant improvements by simply altering the way teachers interact with learners. Online learning has to date mostly delivered fairly linear and non-adaptive experiences, whether it’s through self-paced structured learning, scenario-based learning, simulations or informal learning. But we are now in the position of having technology, especially AI, that can deliver what Bloom called ‘one-to-one learning’.

Adaption can be many things but at the heart of the process is a decision to present something to the learner based on what the system knows about the learners, learning or context.

Pre-course adaptive

Macro-decisions

You can adapt a learning journey at the macro level, recommending skills, courses, even careers based on your individual needs.

Pre-test

‘Pre-test’ the learner, to create a prior profile, before staring the course, then present relevant content. The adaptive software makes a decision based on data specific to that individual. You may start with personal data, such as educational background, competence in previous courses and so on. This is a highly deterministic approach that has limited personalisation and learning benefits but may prevent many from taking unnecessary courses.

Test-out

Allow learners to ‘test-out’ at points in the course to save them time on progression. This short-circuits unnecessary work but has limited benefits in terms of varied learning for individuals.

Preference

Ask or test the learner for their learning style or media preference. Unfortunately, research has shown that false constructs such as learning styles, which do not exist, make no difference on learning outcomes. Personality type is another, although one must be careful with poorly validated outputs from the likes of Myers-Briggs. The OCEAN model is much better validated. One can also use learner opinions, although this is also fraught with danger. Learners are often quite mistaken, not only about what they have learnt but also optimal strategies for learning. So, it is possible to use all sorts of personal data to determine how and what someone should be taught but one has to be very, very careful.

Within-course adaptive

Micro-adaptive courses adjust frequently during a course to determine different routes based on their preferences, what the learner has done or based on specially designed algorithms. A lot of adaptive software within courses uses re-sequencing. The idea is that most learning goes wrong when things are presented that are either too easy, too hard or not relevant for the learner at that moment. One can us the idea of desirable difficulty here to determine a learning experience that is challenging enough to keep the learner driving forward.

Preference

Decision within a course are determined by user choices or assessed preferences. There is little evidence that this works.

Rule-based

Decisions are based on a rule or set of rules, at its simplest a conditional if… then… decision but I often a sequence of rules that determine the learner’s progress.

Algorithm-based

It is worth introducing AI at this point, as it is having a profound effect on all areas of human endeavour. It is inevitable, in my view, that this will also happen in the learning game. Adaptive learning is how the large tech companies deliver to your timeline on Facebook/Twitter, sell to you on Amazon, get you to watch stuff on Netflix. They use an array of techniques based on data they gather, statistics, data mining and AI techniques to improve the delivery of their service to you as an individual. Evidence that AI and adaptive techniques will work in learning, especially in adaption, is there on every device on almost every service we use online. Education is just a bit of a slow learner.

Decisions may be based simply on what the system thinks your level of capability is at that moment, based on formative assessment and other factors. The regular testing of learners, not only improves retention, it gathers useful data about what the system knows about the learner. Failure is not a problem here. Indeed, evidence suggests that making mistakes may be critical to good learning strategies.

Decisions within a course use an algorithm with complex data needs. This provides a much more powerful method for dynamic decision making. At this more fine-grained level, every screen can be regarded as a fresh adaption at that specific point in the course.

Machine learning adaption

AI techniques can, of course, be used in systems that learn and improve as they go. Such systems are often trained using data at the start and then use data as they go to improve the system. The more learners use the system, the better it becomes.

Confidence adaption

Another measure, common in adaptive systems, is the measurement of confidence. You may be asked a question then also asked how confident you are of your answer.

Learning theory 

Good learning theory can also be baked into the algorithms, such as retrieval, interleaving and spaced practice. Care can be taken over cognitive load and even personalised performance support provided adapting to an individuals availability and schedule. Duolingo is sensitive to these needs and provides spaced-practice, aware of the fact that you may have not done anything recently and forgotten stuff. Embodying good learning theory and practice may be what is needed to introduce often counterintuitive methods into teaching, that are resisted by human teachers.

Across courses adaptive

Aggregated data

Aggregated data from a learner’ performance on a previous or previous courses can be used. As can aggregated data of all students who have taken the course. One has to be careful here, as one cohort may have started at a different level of competence than another cohort. There may also be differences on other skills, such as reading comprehension, background knowledge, English as a second language and so on.

Adaptive across curricula

Adaptive software can be applied within a course, across a set of courses but also across an entire curriculum. The idea is that personalisation becomes more targeted, the more you use the system and that competences identified earlier may help determine later sequencing.

Post-course adaptive

Adaptive assessment systems

There’s also adaptive assessment, where test items are presented, based on your performance on previous questions. They often start with a mean test item then select harder or easier items as the learner progresses.

Memory retention systems

Some adaptive systems focus on memory retrieval, retention and recall. They present content, often in a spaced-practice pattern and repeat, remediate and retest to increase retention. These can be powerful systems for the consolidation of learning.

Performance support adaption

Moving beyond courses to performance support, delivering learning when you need it, is another form of adaptive delivery that can be sensitive to your individual needs as well as context. These have been delivered within the workflow, often embedded in social communications systems, sometimes as chatbots.

Conclusion

There are many forms of adaptive learning, in terms of the points of intervention, basis of adaption, technology and purpose. If you want to experience one that is accessible and free, try Duolingo, with 200 million registered users, where structured topics are introduced, alongside basic grammar 

Why being more digital makes us more human

The more digital your life, the more human you become. Sounds like a contradiction but I think we’re reaching a point where technology is becoming less visible, if not invisible. This means that we can concentrate on ourselves and others, while taking advantage of technology, without the physical and intrusive presence of technology. Unpopular view, I know, but it’s one I hold. I honestly believe that technology frees us from the tyranny of time, space and labour.

My daily life is made easier

From the moment I wake up to the moment I fall asleep, technology is an all pervasive part of my life. I get woken by an Alexa alarm playing the Radio and simply say ‘Stop’ when I’m ready. My robot cleaner does all our floor cleaning on a schedule, returning to base when finished. I don’t even have to be there when it happens. My heating is controlled by a system that learns my pattern of needs and adapts accordingly. I switch all of my lights off by a simple word command when I go to bed. But that’s just the start…

My work is made easier

My workspace is my laptop and I can, and do, work anywhere. I work all over the world and my workplace and markets are online. Even the physical dimension of my work is made easy by tech. I book parking, flights, hotels online. I sail through Gatwick with my electronic boarding pass on my mobile, fly in an aircraft that is almost invariably flies and lands on autopilot (safer), and return by almost walking through the gates due to face recognition. The only problems I have are when some ill-trained goon steps in, as in LAX airport, where my wife was thrown out of the US for having a Syrian stamp in her passport (he didn’t know the rules). It has never been easier and cheaper to experience other countries and cultures.

The majority of my work meetings are on Zoom or Skype, for free. I set alarms on Alexa for most of these (4 minutes before they’re due). I use Alexa for VAT calculations. I use Slack with my developers who work remotely. I build my product online and deliver it online, to anywhere in the world with an internet connection. I invoice electronically and get paid electronically. I work from home, and see commuting, packed trains, standing in tubes, office blocks and cubicle offices as profoundly inhuman activities.

My social circle is hugely human

As a long-time social media user, I don’t buy the echo-chamber theory,  that I live in a bubble. Believe me, my social media contacts are a feisty and varied bunch. The simple maths shows that almost by definition, the larger the number of people we are in contact with, the more varied they will be.

I have a group of friends on Facebook, who are mostly people I know face-to-face. Some I only know face-to-face because I met them on Facebook. Others are people I lost touch with but resurrected our friendship decades later. Another ripple is the 9,700 followers, and people I follow, on twitter – an invaluable source of professional and general information – mainly through the links it provides. Another ripple is YouTube, where my Ted talk got 55,000 views, and others at nearly 30,000. Then there’s the next, even wider ripple, my blog with 4.8 million pageviews, from countries all over the world. The majority outside of the UK. Social media has freed me from the tyranny of distance and connected me with an enormous amount of people around the world.

Social media has allowed me to meet many more real people than I would have, had it not existed, put me back in touch with people who have enriched my life by that second resurrected encounter and friendship. It’s enabled invitations from all over the world to speak to, over the years, hundreds of thousands of people. I travel much more because of technology. Even the real world of travel, whether it’s finding a location when I’m walking or driving has been revolutionised by access to GPs and online maps.

I spent my early childhood in small Scottish mining towns where the only social contact I had was a few school friends and a single pen-pal. No one will convince me that those were better days. My sons have friends all over the world, contact with their relatives, almost all of who live in another country, and have access to people, news, and sources beyond my imagination. 

My choices are greater

Almost any piece of music I want to play is available by asking Alexa, most books I can buy and get delivered within a day, most movies and TV programmes available on demand. I find anything I want on TV, as I have a Smart TV that responds to voice. I can get any major news source at the touch of a button, even paid for sources are cheap and accessible – real people writing real stuff. Academic articles, a few keystrokes or voice command away. These are not only choices of media. They carry over into real life. Attendance at live music events has gone through the roof. Online dating gives everyone a wider choice of options. Online is rarely just online – it leads to offline events and contact with other people.

My learning is easier

I want to know something I ask Google Assistant or Alexa. I haven’t been in a library for years. If I want to explore something in depth, I have the resources and MOOCs available for free. I count myself as a Lifelong Learner but I haven’t been on a ‘course’ for 35 years. We’re seeing people learn more independently, on the workflow. Jane Hart, who tracks what people actually use for learning, shows that they are not traditional training tools, they are YouTube and Twitter. Learner behaviour is driven more by the individual than trainers. What’s needed is support for this. Smart technology can get to know you, give you suggestions, oil the wheels of this access to timely and relevant learning.

Choices

My puzzlement is over those who see technology as something that destroys humanity. This is not to say that technology does not have a bad side – it usually does. I don’t drive, never been behind the wheel of a car in my life, and know that 1.3 million a year dies in car crashes. Yet most people continue to drive. The surest solution to this problem is self-driving vehicles. To those on social media, who spend so much time saying how evil it is, we all have our choices. If you don’t like social media, don’t use it, just as I don’t drive. On the whole, technology frees the self, frees us from the tyranny of time, space and numbers.

Conclusion

Technology is increasingly invisible. The interfaces are becoming more natural though touch and now voice. AI takes the heavy lifting away from all sorts of tasks. Technology is no more about devices but smart services. With voice and the IoT, we will find ourselves in a world where technology simply solves problems, behind the scenes. Some of it happens when I’m not there, behind the scenes, below the radar. I like that. Some is light touch, almost frictionless, like Alexa switching off my lights. Other technology saves me a ton of time – like online meetings and business processes. Above all, it’s the people side I like. The more technology I use, the more human my life becomes.

2019 predictions in L&D... some surprising disappearances...

Great survey from my friend and namesake Donald Taylor. We are sometimes confused (in both senses of the word), but when it comes to what’s hot in workplace L&D in 2019, he’s the go to man. This is the 6^thyear of his survey, by nearly 2000 professionals making 5332 votes from 92 countries.

Top three stars

  Personalisation/adaptive learning (1)

  Artificial Intelligence (2)

  Learning Analytics (3)

One could argue that all three of these top spots have been taken by AI. Sure there are aspects of personalized learning and analytics that are not AI, but it’s there, underlying all three top spots. I have spent the last four years saying that Artificial Intelligence is the major shift in learning technologies with a post in 2014, saying ‘My tech prediction for 2015 - two small letters…AI’. AI is changing the very nature of work, so it is ridiculous to imagine that it will not also change why, what and how we learn. Having started this journey in AI many years ago, four years ago I made an investment in an adaptive learning company, started my own AI company WildFire and began talking about this at conferences all over the world. To ignore this is to ignore reality and arguably the most important technology shift we've seen since the invention of print.

Three newbies

  Microlearning (5)

  Learning Experience Platforms (6)

  Performance support (11)

I’ve grouped these together as they show an interesting shift in thinking towards the more dynamic delivery of learning. I'd link the to the top three as chatbots and other forms of smart AI delivery are helping them get to learners in the workflow. My fear is that we'll get a fair bit of puff, as people replace the M with an X and deliver the same old stuff.

Three media

  Virtual/augmented reality (7)  

  Mobile delivery (8)

  Video (13)

Characterised by the fact that they’re actually hardware and media defined, they're here to stay. VR/AR is gaining ground, as I thought it would, and we have, at last, a way to deliver learning by doing. Mobile is, of course, everywhere and video is coming of age, as we’re seeing it better integrated into learning.

Three business topics

  Consulting more deeply with the business (9)

  Showing value (10)

  Developing the L&D function (15)

Although all three dropped 5,4 and 3 places respectively, they’re still in respectful positions and it's good to see the profession trying to keep business relevance and professionalism on the table. I'd like to see more attention to research and evidence but we're getting there.

Three bags full

  Collaborative learning (4)

  Neuroscience/cognitive science (12)

  Curation (14)

Collaborative learning is pretty solid, and it’s good to see that the science of learning is still in here. I still find it shocking that many practitioners have no idea what science says about learning and online learning. Lastly curation – bit of an oddball this one but it’s here.

Three goners

  Gamification

  MOOCs

  Badges

Gamification seems to have shot its bolt and disappeared. I think we got fed up with the weak side of gamification, playing Pavlov with learners, so it seems to have run its course. MOOCs have drifted away, more education that L&D – the numbers taking vocational MOOCS are phenomenal but this is not the world of L&D, it is the world of learners (oh the irony). Badges have also gone. That’s a shame but I too changed my mind on these and they seem to have had their day.

Conclusion

Once again, a great insight into how people are thinking. Over the years this has been a pretty good guide to what’s rising, staying around and falling. Well done to Donald Taylor and his team.

Does ‘Design thinking’ lead to bad learning design?

Fads come and go, and ‘Design Thinking’ seems to be one on the rise at the moment. It’s a process with lots of variants but, in the talks I’ve seen on the subject, and the results I’ve seen emerge from the process, I’m not wholly convinced. The problem is that we may well need less ‘design’ and more ‘thinking’.  The combination is likely to dumb down the learning in favour of superficial design. Imagine applying this theory to medicine. You wouldn’t get far by simply asking patients what they need to cure their problems, you need a growing body of good research tried and tested methods, and expertise. So let’s break the Design Thinking process down to see how it works in practice and examine the steps one by one.

Empathise

Donald Norman, in Design Thinking though that empathy in design was wrong-headed, a waste of time and energy and that the designers time would be better spent understanding the task. He is right. It is a conceit.

Donald Norman says, of this call for empathy in design, that “the concept is impossible, and even if possible, wrong”. I was seeing empathy used in pieces that actually mentioned Norman as one of their heroes! Yet here he was saying it was wrong-headed. He is absolutely right. There is no way you can put yourself into the heads of the hundreds, thousands, even tens and hundreds of thousands of learners. As Norman says “It sounds wonderful but the search for empathy is simply misled.” Not only is it not possible to understand individuals in this way, it is just not that useful.

It is not empathy but data you need. Who are these people, what do they need to actually do and how can we help them. As people they will be hugely variable but what they need to know and do, in order to achieve a goal, is relatively stable. This has little to do with empathy and a lot to do with understanding and reason.
All too often we latch on to a noun in the learning world without thinking much about what it actually means, what experts in the field say about it and bandy it about as though it were a certain truth. This is the opposite of showing empathy. It is the rather empty use of language.

Empathy for the learner is an obvious virtue but what exactly does that mean? For years, in practice, this meant Learning Styles. For many it still is Learning Styles, being sensitive to learner’s differences, diversity and needs in terms of preferences. This, of course has been a disastrous waste of time, as research has shown. Other faddish outcomes over-sensitive to supposed learner needs have been Myers-Briggs, NLP and no end of faddish ideas about what we ‘think’ learners need, rather than what research tells us they actually benefit from.

Research in cognitive psychology has given us clear evidence that learners are often mistaken when it comes to judgements about their own learning. Bjork, along with many other high quality researchers, have shown that learning is “quite misunderstood (by learners)…. we have a flawed model of how we learn and remember”. There’s often a negative correlation between people’s judgements of their learning, what they think they have learnt, how they think they learn best - and what they’ve ‘actually’ learnt and the way they can ‘actually’ optimise their learning. In short, our own perceptions of learning are seriously delusional. This is why engagement, fun, learner surveys and happy sheets are such bad measures of what is actually learnt and the enemy of optimal learning strategies. In short, empathy and asking learners what they want can seriously damage design.
In truth replacing a good needs analysis, including a thorough understanding of your target audience is not bettered by calling it empathy. That is simply replacing analysis with an abstract word to make it sound more in tune with the times.

Define

Identifying learner needs and problems has led to a ton of wasteful energy spent on slicing them up into digital natives/immigrants and personas that often average out differentiation and personalisation. The solution is not to identify ideal learners as personas but provide sophisticated pedagogic approaches that are adaptive and provide personal feedback. Design thinking makes the mistake of thinking there is such a thing as ideal learners without realising that you need analysis of the target audience, not ‘averaged out’ personas.

Design Thinking seems to push people towards thinking that learning problems are ‘design’ problems. Many are not. You need top understand the nature of the cognitive problems and researched solutions to those problems. By all means define the problems but those problems but know what a learning problem is.

One area, however, where I think design thinking could be useful is in identifying the context, workflow and moments of need. So, understanding the learner’s world, their business environment. That’s fine. On this I agree, But I rarely hear this from practising ‘Design Thinking’ practitioners, who tend to focus on the screen design itself, rather than design of a blended learning experience, based on the types of learning to be delivered in real environments, in the workflow with performance support. You need a deep understanding of the technology and its limitations.
There is also an argument for having a compete set of skills on the team but this has nothing to do with design thinking. The delivery of online learning is a complex mix of learning, design, technical, business and fiscal challenges. What's needed is balance in the team not a process that values an abstract method with a focus on 'design' alone.

Ideate

This is the key step, where design thinkers are supposed to provide challenge and creative solutions. It is the step where it can all go wrong. Creative solutions tend to be based on media delivery, not effortful learning, chunking, interleaving, open input, spaced practice and many other deeper pedagogic issues that need to be understood before you design anything. There’s often a dearth of knowledge about the decades of research in learning and cognitive science that should inform design. It is replaced by rather superficial ideas around media production and presentation, hence the edutainment we get, all ‘tainment’ and no ‘edu’. It focuses on presentation not effortful learning.

Few design thinkers I’ve heard show much knowledge of designing for cognitive load, avoiding redundancy and have scant knowledge of the piles of brilliant work done by Nass, Reeves, Mayer, Clark, Roediger, MacDaniel and many other researchers who have worked for decades uncovering what good online learning design requires. This is also why co-design is so dangerous. It leads to easy learning, all front and no depth.

What I’ve seen is lots of ‘ideation’ around gamification (but the trivial, Pavlovian aspects of games - scoring, badges and leaderboards). Even worse is the over-designed, media rich, click-through learning, loosely punctuated by multiple-choice questions. Remember that media rich does not mean mind-rich. Even then, designers rarely know the basic research, for example, on the optimal number of options in MCQs or that open input is superior.

Protoype

It is easy to prototype surface designs and get voiced feedback on what people like but this is a tiny part of the story. It is pointless prototyping learning solutions in the hope that you’ll uncover real learning efficacy (as opposed to look and feel) without evaluating those different solutions. This means the tricky and inconvenient business of real research, with controls, reasonable sample sizes, randomly selected learners and clear measurement of retention in long-term memory, even transfer. Few with just ‘Design Thinking’ skills have the skills, time and budget to do this. This is why we must rely on past research and build on this body of knowledge, just as clinicians do in medicine. We need to be aware of the work of Bjork, Roediger, Karpicke, Heustler and Metcalfe, who show that asking learners what they think is counterproductive. And build on the research that shows what techniques work for high retention.

A problem is that prototyping is often defined by the primitive tools used by learning designers, that can only produce presentation-like, souped-up Powerpoint and MCQs, whereas real learning requires much deeper structures. Few have made the effort to explore tools that allow open input and free text input, which really does increase retention and recall. Low fidelity prototyping won’t hack it if you want open input and sophisticated adaptive and personalised learning through AI – and that’s where things are heading.

One area that Design Thinking can help is with the ‘user interface’ but this is only one part of the deliverable and often not that important. It is important to make it as frictionless as possible but this comes as much through technical advances, touchscreen, voice, open input, than design.

Test

Testing is a complex business. I used to run a large online learning test lab – the largest in the UK. We tested for usability, accessibility, quality assurance and technical conformance and, believe me, to focus just on ‘design’ is a big mistake. You need to focus not on surface design but, more importantly, on all sorts of things, such as learning efficacy. Once again, learner testimony can help but it can also hinder. Learners often report illusory learning when they are presented with high quality media – this means absolutely nothing. Testing is pointless if you’re not testing the real goal – actual retained learning. Asking people for qualitative opinions does not do that.
In truth testing is quite tricky. You have to be clear about what you are testing, cover everything and have good reporting. There are tried and tested methods, that few have ever studied, so this is a really weak link. Just shoving something under the nose of a learner is not enough. We found early on that it is a short number of iterations with an expert that really works with interface design, along with A/B testing. Not some simple suck it and see trial.

Conclusion

I've heard several presentations on this and done the reading but my reaction is still the same. Is that it? It seems like a short-circuited version of a poor, project management course. I honestly think that the danger of ‘Design Thinking’ is that it holds us back.  We’ve had this for several years now, where design trumps deep thinking, knowledge of how we learn, knowledge of cognitive overload and knowledge of optimal learning strategies. It gives us the illusion of creativity but at the expense of sound learning. Walk around any large online learning exhibition and observe the output – over-engineered design that lacks depth. Design thinking lures us into thinking that we have solved learning problems when all we have done is polish presentation. The real innovations I’ve seen come from a deep understanding of the research, technology and innovative solutions based on that research, like nudge learning and WildFire. Delivery, I think, is better rooted in strong practices, such as ISO standards and practices guided by evidence, which have evolved over time and not simplistic processes that are often simplified further and sold as bromides. As one commentator, who tried Design Thinking, said "we ended up doing nothing more than polishing turds!".