Tuesday, February 12, 2019

What is ‘adaptive’ learning?

Personalised ‘adaptive’ learning came top of this 2019 survey in L&D. Having spent a few years involved with an adaptive learning company, delivering real adaption to real learners, on scale, I thought I’d try to explain what it is, a taxonomy of adaptive learning. The problem is that the word has been applied to many things from simple pre-test assessment to full-blown algorithmic and machine learning adaption, and lots in-between. 
In essence it means adapting the online experience to the individual’s needs as they learn, in the way a personal tutor would adapt. The aim is to provide, what many teachers provide, a learning experience that is tailored to the needs of you as an individual learner. 
Benjamin Bloom, best know for his taxonomy of learning, wrote a now famous paper, The 2 Sigma Problem, which compared the lecture, formative feedback lecture and one-to-one tuition. It is a landmark in adaptive learning. Taking the ‘straight lecture’ as the mean, he found an 84% increase in mastery above the mean for a ‘formative feedback’ approach to teaching and an astonishing 98% increase in mastery for ‘one-to-one tuition’. Google’s Peter Norvig famously said that if you only have to read one paper to support  online learning, this is it. In other words, the increase in efficacy for tailored  one-to-one, because of the increase in on-task learning, is huge. This paper deserves to be read by anyone looking at improving the efficacy of learning as it shows hugely significant improvements by simply altering the way teachers interact with learners. Online learning has to date mostly delivered fairly linear and non-adaptive experiences, whether it’s through self-paced structured learning, scenario-based learning, simulations or informal learning. But we are now in the position of having technology, especially AI, that can deliver what Bloom called ‘one-to-one learning’.
Adaption can be many things but at the heart of the process is a decision to present something to the learner based on what the system knows about the learners, learning or context.

Pre-course adaptive
You can adapt a learning journey at the macro level, recommending skills, courses, even careers based on your individual needs.
‘Pre-test’ the learner, to create a prior profile, before staring the course, then present relevant content. The adaptive software makes a decision based on data specific to that individual. You may start with personal data, such as educational background, competence in previous courses and so on. This is a highly deterministic approach that has limited personalisation and learning benefits but may prevent many from taking unnecessary courses.
Allow learners to ‘test-out’ at points in the course to save them time on progression. This short-circuits unnecessary work but has limited benefits in terms of varied learning for individuals.
Ask or test the learner for their learning style or media preference. Unfortunately, research has shown that false constructs such as learning styles, which do not exist, make no difference on learning outcomes. Personality type is another, although one must be careful with poorly validated outputs from the likes of Myers-Briggs. The OCEAN model is much better validated. One can also use learner opinions, although this is also fraught with danger. Learners are often quite mistaken, not only about what they have learnt but also optimal strategies for learning. So, it is possible to use all sorts of personal data to determine how and what someone should be taught but one has to be very, very careful.

Within-course adaptive
Micro-adaptive courses adjust frequently during a course to determine different routes based on their preferences, what the learner has done or based on specially designed algorithms. A lot of adaptive software within courses uses re-sequencing. The idea is that most learning goes wrong when things are presented that are either too easy, too hard or not relevant for the learner at that moment. One can us the idea of desirable difficulty here to determine a learning experience that is challenging enough to keep the learner driving forward.
Decision within a course are determined by user choices or assessed preferences. There is little evidence that this works.
Decisions are based on a rule or set of rules, at its simplest a conditional if… then… decision but I often a sequence of rules that determine the learner’s progress.
It is worth introducing AI at this point, as it is having a profound effect on all areas of human endeavour. It is inevitable, in my view, that this will also happen in the learning game. Adaptive learning is how the large tech companies deliver to your timeline on Facebook/Twitter, sell to you on Amazon, get you to watch stuff on Netflix. They use an array of techniques based on data they gather, statistics, data mining and AI techniques to improve the delivery of their service to you as an individual. Evidence that AI and adaptive techniques will work in learning, especially in adaption, is there on every device on almost every service we use online. Education is just a bit of a slow learner.
Decisions may be based simply on what the system thinks your level of capability is at that moment, based on formative assessment and other factors. The regular testing of learners, not only improves retention, it gathers useful data about what the system knows about the learner. Failure is not a problem here. Indeed, evidence suggests that making mistakes may be critical to good learning strategies.
Decisions within a course use an algorithm with complex data needs. This provides a much more powerful method for dynamic decision making. At this more fine-grained level, every screen can be regarded as a fresh adaption at that specific point in the course.
Machine learning adaption
AI techniques can, of course, be used in systems that learn and improve as they go. Such systems are often trained using data at the start and then use data as they go to improve the system. The more learners use the system, the better it becomes.
Confidence adaption
Another measure, common in adaptive systems, is the measurement of confidence. You may be asked a question then also asked how confident you are of your answer.
Learning theory 
Good learning theory can also be baked into the algorithms, such as retrieval, interleaving and spaced practice. Care can be taken over cognitive load and even personalised performance support provided adapting to an individuals availability and schedule. Duolingo is sensitive to these needs and provides spaced-practice, aware of the fact that you may have not done anything recently and forgotten stuff. Embodying good learning theory and practice may be what is needed to introduce often counterintuitive methods into teaching, that are resisted by human teachers.

Across courses adaptive
Aggregated data
Aggregated data from a learner’ performance on a previous or previous courses can be used. As can aggregated data of all students who have taken the course. One has to be careful here, as one cohort may have started at a different level of competence than another cohort. There may also be differences on other skills, such as reading comprehension, background knowledge, English as a second language and so on.
Adaptive across curricula
Adaptive software can be applied within a course, across a set of courses but also across an entire curriculum. The idea is that personalisation becomes more targeted, the more you use the system and that competences identified earlier may help determine later sequencing.

Post-course adaptive
Adaptive assessment systems
There’s also adaptive assessment, where test items are presented, based on your performance on previous questions. They often start with a mean test item then select harder or easier items as the learner progresses.
Memory retention systems
Some adaptive systems focus on memory retrieval, retention and recall. They present content, often in a spaced-practice pattern and repeat, remediate and retest to increase retention. These can be powerful systems for the consolidation of learning.
Performance support adaption
Moving beyond courses to performance support, delivering learning when you need it, is another form of adaptive delivery that can be sensitive to your individual needs as well as context. These have been delivered within the workflow, often embedded in social communications systems, sometimes as chatbots.

There are many forms of adaptive learning, in terms of the points of intervention, basis of adaption, technology and purpose. If you want to experience one that is accessible and free, try Duolingo, with 200 million registered users, where structured topics are introduced, alongside basic grammar 

 Subscribe to RSS

Friday, February 08, 2019

Why being more digital makes us more human

The more digital your life, the more human you become. Sounds like a contradiction but I think we’re reaching a point where technology is becoming less visible, if not invisible. This means that we can concentrate on ourselves and others, while taking advantage of technology, without the physical and intrusive presence of technology. Unpopular view, I know, but it’s one I hold. I honestly believe that technology frees us from the tyranny of time, space and labour.
My daily life is made easier
From the moment I wake up to the moment I fall asleep, technology is an all pervasive part of my life. I get woken by an Alexa alarm playing the Radio and simply say ‘Stop’ when I’m ready. My robot cleaner does all our floor cleaning on a schedule, returning to base when finished. I don’t even have to be there when it happens. My heating is controlled by a system that learns my pattern of needs and adapts accordingly. I switch all of my lights off by a simple word command when I go to bed. But that’s just the start…
My work is made easier
My workspace is my laptop and I can, and do, work anywhere. I work all over the world and my workplace and markets are online. Even the physical dimension of my work is made easy by tech. I book parking, flights, hotels online. I sail through Gatwick with my electronic boarding pass on my mobile, fly in an aircraft that is almost invariably flies and lands on autopilot (safer), and return by almost walking through the gates due to face recognition. The only problems I have are when some ill-trained goon steps in, as in LAX airport, where my wife was thrown out of the US for having a Syrian stamp in her passport (he didn’t know the rules). It has never been easier and cheaper to experience other countries and cultures.
The majority of my work meetings are on Zoom or Skype, for free. I set alarms on Alexa for most of these (4 minutes before they’re due). I use Alexa for VAT calculations. I use Slack with my developers who work remotely. I build my product online and deliver it online, to anywhere in the world with an internet connection. I invoice electronically and get paid electronically. I work from home, and see commuting, packed trains, standing in tubes, office blocks and cubicle offices as profoundly inhuman activities.
My social circle is hugely human
As a long-time social media user, I don’t buy the echo-chamber theory,  that I live in a bubble. Believe me, my social media contacts are a feisty and varied bunch. The simple maths shows that almost by definition, the larger the number of people we are in contact with, the more varied they will be.
I have a group of friends on Facebook, who are mostly people I know face-to-face. Some I only know face-to-face because I met them on Facebook. Others are people I lost touch with but resurrected our friendship decades later. Another ripple is the 9,700 followers, and people I follow, on twitter – an invaluable source of professional and general information – mainly through the links it provides. Another ripple is YouTube, where my Ted talk got 55,000 views, and others at nearly 30,000. Then there’s the next, even wider ripple, my blog with 4.8 million pageviews, from countries all over the world. The majority outside of the UK. Social media has freed me from the tyranny of distance and connected me with an enormous amount of people around the world.
Social media has allowed me to meet many more real people than I would have, had it not existed, put me back in touch with people who have enriched my life by that second resurrected encounter and friendship. It’s enabled invitations from all over the world to speak to, over the years, hundreds of thousands of people. I travel much more because of technology. Even the real world of travel, whether it’s finding a location when I’m walking or driving has been revolutionised by access to GPs and online maps.
I spent my early childhood in small Scottish mining towns where the only social contact I had was a few school friends and a single pen-pal. No one will convince me that those were better days. My sons have friends all over the world, contact with their relatives, almost all of who live in another country, and have access to people, news, and sources beyond my imagination. 
My choices are greater
Almost any piece of music I want to play is available by asking Alexa, most books I can buy and get delivered within a day, most movies and TV programmes available on demand. I find anything I want on TV, as I have a Smart TV that responds to voice. I can get any major news source at the touch of a button, even paid for sources are cheap and accessible – real people writing real stuff. Academic articles, a few keystrokes or voice command away. These are not only choices of media. They carry over into real life. Attendance at live music events has gone through the roof. Online dating gives everyone a wider choice of options. Online is rarely just online – it leads to offline events and contact with other people.
My learning is easier
I want to know something I ask Google Assistant or Alexa. I haven’t been in a library for years. If I want to explore something in depth, I have the resources and MOOCs available for free. I count myself as a Lifelong Learner but I haven’t been on a ‘course’ for 35 years. We’re seeing people learn more independently, on the workflow. Jane Hart, who tracks what people actually use for learning, shows that they are not traditional training tools, they are YouTube and Twitter. Learner behaviour is driven more by the individual than trainers. What’s needed is support for this. Smart technology can get to know you, give you suggestions, oil the wheels of this access to timely and relevant learning.
My puzzlement is over those who see technology as something that destroys humanity. This is not to say that technology does not have a bad side – it usually does. I don’t drive, never been behind the wheel of a car in my life, and know that 1.3 million a year dies in car crashes. Yet most people continue to drive. The surest solution to this problem is self-driving vehicles. To those on social media, who spend so much time saying how evil it is, we all have our choices. If you don’t like social media, don’t use it, just as I don’t drive. On the whole, technology frees the self, frees us from the tyranny of time, space and numbers.
Technology is increasingly invisible. The interfaces are becoming more natural though touch and now voice. AI takes the heavy lifting away from all sorts of tasks. Technology is no more about devices but smart services. With voice and the IoT, we will find ourselves in a world where technology simply solves problems, behind the scenes. Some of it happens when I’m not there, behind the scenes, below the radar. I like that. Some is light touch, almost frictionless, like Alexa switching off my lights. Other technology saves me a ton of time – like online meetings and business processes. Above all, it’s the people side I like. The more technology I use, the more human my life becomes.

 Subscribe to RSS

Tuesday, February 05, 2019

2019 predictions in L&D... some surprising disappearances...

Great survey from my friend and namesake Donald Taylor. We are sometimes confused (in both senses of the word), but when it comes to what’s hot in workplace L&D in 2019, he’s the go to man. This is the 6thyear of his survey, by nearly 2000 professionals making 5332 votes from 92 countries.

Top three stars

  Personalisation/adaptive learning (1)
  Artificial Intelligence (2)
  Learning Analytics (3)
One could argue that all three of these top spots have been taken by AI. Sure there are aspects of personalized learning and analytics that are not AI, but it’s there, underlying all three top spots. I have spent the last four years saying that Artificial Intelligence is the major shift in learning technologies with a post in 2014, saying My tech prediction for 2015 - two small letters…AI. AI is changing the very nature of work, so it is ridiculous to imagine that it will not also change why, what and how we learn. Having started this journey in AI many years ago, four years ago I made an investment in an adaptive learning company, started my own AI company WildFire and began talking about this at conferences all over the world. To ignore this is to ignore reality and arguably the most important technology shift we've seen since the invention of print.

Three newbies
  Microlearning (5)
  Learning Experience Platforms (6)
  Performance support (11)
I’ve grouped these together as they show an interesting shift in thinking towards the more dynamic delivery of learning. I'd link the to the top three as chatbots and other forms of smart AI delivery are helping them get to learners in the workflow. My fear is that we'll get a fair bit of puff, as people replace the M with an X and deliver the same old stuff.

Three media

  Virtual/augmented reality (7)  
  Mobile delivery (8)
  Video (13)
Characterised by the fact that they’re actually hardware and media defined, they're here to stay. VR/AR is gaining ground, as I thought it would, and we have, at last, a way to deliver learning by doing. Mobile is, of course, everywhere and video is coming of age, as we’re seeing it better integrated into learning.

Three business topics

  Consulting more deeply with the business (9)
  Showing value (10)
  Developing the L&D function (15)
Although all three dropped 5,4 and 3 places respectively, they’re still in respectful positions and it's good to see the profession trying to keep business relevance and professionalism on the table. I'd like to see more attention to research and evidence but we're getting there.

Three bags full

  Collaborative learning (4)
  Neuroscience/cognitive science (12)
  Curation (14)
Collaborative learning is pretty solid, and it’s good to see that the science of learning is still in here. I still find it shocking that many practitioners have no idea what science says about learning and online learning. Lastly curation – bit of an oddball this one but it’s here.

Three goners

Gamification seems to have shot its bolt and disappeared. I think we got fed up with the weak side of gamification, playing Pavlov with learners, so it seems to have run its course. MOOCs have drifted away, more education that L&D – the numbers taking vocational MOOCS are phenomenal but this is not the world of L&D, it is the world of learners (oh the irony). Badges have also gone. That’s a shame but I too changed my mind on these and they seem to have had their day.
Once again, a great insight into how people are thinking. Over the years this has been a pretty good guide to what’s rising, staying around and falling. Well done to Donald Taylor and his team.

 Subscribe to RSS

Saturday, February 02, 2019

Does ‘Design thinking’ lead to bad learning design?

Fads come and go, and ‘Design Thinking’ seems to be one on the rise at the moment. It’s a process with lots of variants but, in the talks I’ve seen on the subject, and the results I’ve seen emerge from the process, I’m not wholly convinced. The problem is that we may well need less ‘design’ and more ‘thinking’.  The combination is likely to dumb down the learning in favour of superficial design. Imagine applying this theory to medicine. You wouldn’t get far by simply asking patients what they need to cure their problems, you need a growing body of good research tried and tested methods, and expertise. So let’s break the Design Thinking process down to see how it works in practice and examine the steps one by one.
Empathy for the learner is an obvious virtue but what exactly does that mean? For years, in practice, this meant Learning Styles. For many it still is Learning Styles, being sensitive to learner’s differences, diversity and needs in terms of preferences. This, of course has been a disastrous waste of time, as research has shown. Other faddish outcomes over-sensitive to supposed learner needs have been Myers-Briggs, NLP and no end of faddish ideas about what we ‘think’ learners need, rather than what research tells us they actually benefit from.
Research in cognitive psychology has given us clear evidence that learners are often mistaken when it comes to judgements about their own learning. Bjork, along with many other high quality researchers, have shown that learning is “quite misunderstood (by learners)…. we have a flawed model of how we learn and remember”. There’s often a negative correlation between people’s judgements of their learning, what they think they have learnt, how they think they learn best - and what they’ve ‘actually’ learnt and the way they can ‘actually’ optimise their learning. In short, our own perceptions of learning are seriously delusional. This is why engagement, fun, learner surveys and happy sheets are such bad measures of what is actually learnt and the enemy of optimal learning strategies. In short, empathy and asking learners what they want can seriously damage design.
In truth replacing a good needs analysis, including a thorough understanding of your target audience is not bettered by calling it empathy. That is simply replacing analysis with an abstract word to make it sound more in tune with the times.
Identifying learner needs and problems has led to a ton of wasteful energy spent on slicing them up into digital natives/immigrants and personas that often average out differentiation and personalisation. The solution is not to identify ideal learners as personas but provide sophisticated pedagogic approaches that are adaptive and provide personal feedback. Design thinking makes the mistake of thinking there is such a thing as ideal learners without realising that you need analysis of the target audience, not ‘averaged out’ personas.
Design Thinking seems to push people towards thinking that learning problems are ‘design’ problems. Many are not. You need top understand the nature of the cognitive problems and researched solutions to those problems. By all means define the problems but those problems but know what a learning problem is.
One area, however, where I think design thinking could be useful is in identifying the context, workflow and moments of need. So, understanding the learner’s world, their business environment. That’s fine. On this I agree, But I rarely hear this from practising ‘Design Thinking’ practitioners, who tend to focus on the screen design itself, rather than design of a blended learning experience, based on the types of learning to be delivered in real environments, in the workflow with performance support. You need a deep understanding of the technology and its limitations.
There is also an argument for having a compete set of skills on the team but this has nothing to do with design thinking. The delivery of online learning is a complex mix of learning, design, technical, business and fiscal challenges. What's needed is balance in the team not a process that values an abstract method with a focus on 'design' alone.
This is the key step, where design thinkers are supposed to provide challenge and creative solutions. It is the step where it can all go wrong. Creative solutions tend to be based on media delivery, not effortful learning, chunking, interleaving, open input, spaced practice and many other deeper pedagogic issues that need to be understood before you design anything. There’s often a dearth of knowledge about the decades of research in learning and cognitive science that should inform design. It is replaced by rather superficial ideas around media production and presentation, hence the edutainment we get, all ‘tainment’ and no ‘edu’. It focuses on presentation not effortful learning.
Few design thinkers I’ve heard show much knowledge of designing for cognitive load, avoiding redundancy and have scant knowledge of the piles of brilliant work done by Nass, Reeves, Mayer, Clark, Roediger, MacDaniel and many other researchers who have worked for decades uncovering what good online learning design requires. This is also why co-design is so dangerous. It leads to easy learning, all front and no depth.
What I’ve seen is lots of ‘ideation’ around gamification (but the trivial, Pavlovian aspects of games - scoring, badges and leaderboards). Even worse is the over-designed, media rich, click-through learning, loosely punctuated by multiple-choice questions. Remember that media rich does not mean mind-rich. Even then, designers rarely know the basic research, for example, on the optimal number of options in MCQs or that open input is superior.
It is easy to prototype surface designs and get voiced feedback on what people like but this is a tiny part of the story. It is pointless prototyping learning solutions in the hope that you’ll uncover real learning efficacy (as opposed to look and feel) without evaluating those different solutions. This means the tricky and inconvenient business of real research, with controls, reasonable sample sizes, randomly selected learners and clear measurement of retention in long-term memory, even transfer. Few with just ‘Design Thinking’ skills have the skills, time and budget to do this. This is why we must rely on past research and build on this body of knowledge, just as clinicians do in medicine. We need to be aware of the work of Bjork, Roediger, Karpicke, Heustler and Metcalfe, who show that asking learners what they think is counterproductive. And build on the research that shows what techniques work for high retention.
A problem is that prototyping is often defined by the primitive tools used by learning designers, that can only produce presentation-like, souped-up Powerpoint and MCQs, whereas real learning requires much deeper structures. Few have made the effort to explore tools that allow open input and free text input, which really does increase retention and recall. Low fidelity prototyping won’t hack it if you want open input and sophisticated adaptive and personalised learning through AI – and that’s where things are heading.
One area that Design Thinking can help is with the ‘user interface’ but this is only one part of the deliverable and often not that important. It is important to make it as frictionless as possible but this comes as much through technical advances, touchscreen, voice, open input, than design.
Testing is a complex business. I used to run a large online learning test lab – the largest in the UK. We tested for usability, accessibility, quality assurance and technical conformance and, believe me, to focus just on ‘design’ is a big mistake. You need to focus not on surface design but, more importantly, on all sorts of things, such as learning efficacy. Once again, learner testimony can help but it can also hinder. Learners often report illusory learning when they are presented with high quality media – this means absolutely nothing. Testing is pointless if you’re not testing the real goal – actual retained learning. Asking people for qualitative opinions does not do that.
In truth testing is quite tricky. You have to be clear about what you are testing, cover everything and have good reporting. There are tried and tested methods, that few have ever studied, so this is a really weak link. Just shoving something under the nose of a learner is not enough. We found early on that it is a short number of iterations with an expert that really works with interface design, along with A/B testing. Not some simple suck it and see trial.
I've heard several presentations on this and done the reading but my reaction is still the same. Is that it? It seems like a short-circuited version of a poor, project management course. I honestly think that the danger of ‘Design Thinking’ is that it holds us back.  We’ve had this for several years now, where design trumps deep thinking, knowledge of how we learn, knowledge of cognitive overload and knowledge of optimal learning strategies. It gives us the illusion of creativity but at the expense of sound learning. Walk around any large online learning exhibition and observe the output – over-engineered design that lacks depth. Design thinking lures us into thinking that we have solved learning problems when all we have done is polish presentation. The real innovations I’ve seen come from a deep understanding of the research, technology and innovative solutions based on that research, like nudge learning and WildFire. Delivery, I think, is better rooted in strong practices, such as ISO standards and practices guided by evidence, which have evolved over time and not simplistic processes that are often simplified further and sold as bromides. As one commentator, who tried Design Thinking, said "we ended up doing nothing more than polishing turds!".

 Subscribe to RSS