In 2014, at EPFL (Ecole Polytechnique Federale de lausanne)in Switzerland, I was speaking at a MOOC conference explaining how MOOCs would eventually move towards satisfying real demand in vocational courses and not, as most attendees thought at the time, deliver standard HE courses. This proved to be right. The most popular MOOCs worldwide are not liberal arts courses but IT, business and healthcare courses. Udacity, Coursera, Udemy and EdX have all recognised this and shifted their business model. Futurelearn remains in the doldrums.
At that same conference I spoke to Andrew Ng and asked him about the application of AI (he’s a word-class AI expert) to online content. I had been pushing this since April 2013. His reply was interesting, “It’s too early and the problem is the difficulty in authoring the courses to be suitable for the application of AI”. He was right then, but he’s now just done exactly that in Coursera.
AI Recommendation engines
Recommender systems are now mainstream in almost every area of online delivery: search (Google), social media (Facebook, Twitter, Instagram), e-commerce (Amazon) and entertainment (Netflix). AI is the new UI (User Interface), as almost all online experiences are mediated by recommendation engines.
Their success has been proven in almost every area of human endeavour, even more recently in health where they provide diagnoses, comparable, and at times superior, to that of clinicians. Yet they are rare in education and training. This is changing as companies such as WildFire, Coursera, Udacity and others, offer Recommendation services that allow organisations and learners to identify and select courses, based on existing skillsets and experience. We can expect this to be area of real growth over the next few years.
Recommender engines in learning
Recommender systems, are used at various levels in learning:
- Macro-level insights from training to solve business problems
- Macro-level curricula selection for organisations
- Macro-level course selection for individuals
- Micro-level insights and decisions within courses for individuals through adaptive learning
- Micro-level recommendations for external content within context while doing a course
- Micro-level insights and decisions for individuals through assessment
Macro-level insights from training to solve business problems
Chris Brannigan is the CEO of Caspian Learning is a neuroscientist who uses what he calls ‘Human Performance Intelligence’ to investigate, diagnose and treat business problems within organisations. This can be compliance issues, risk or performance of any kind. The aim is to do a complete health check, using AI-driven, 3D simulated training scenarios, sophisticated behavioural analysis, right through to predictive analysis and recommendations for process, human and other types of change. The ambition is breathtaking.
Let’s take a real example, one that Chris has completed. How do you know that your tens or hundreds of thousands of employees perform under high risk? You don’t. The problem is that the risk is asymmetric. A few bad apples can incur the wrath of the regulators, who are getting quite feisty. You really do need to know what they do, why they do it and what you need to do to change things for the better. Their system learns from experts, so that there is an ideal model, then employees go through scenarios (distributed practice) which subtly gathers data over 20 or so scenarios, with lot of different flavours. It then diagnoses the problems in terms of decision-making, reasoning and investigation. A diagnosis, along with a financial impact analysis is delivered to senior executives and line managers, with specific actions. All of this is done using AI techniques that include machine learning and other forms of algorithmic and data analysis to improve the business. It is one very smart solution.
Note that the goal is not to improve training but to improve the business. The data, intelligence, predictive analytics, all move towards decisions, actions and change. The diagnosis will identify geographic areas, cultural problems, specific processes, system weaknesses – all moving towards solutions that may be; more training, investment decisions, system changes or personnel changes. All of this is based on modelling business outcomes.
Macro-level curricula selection for organisations
Coursera have 31 million registered users, huge data sets and 1400 commercial partners and, as mentioned, are using AI to improve organisational learning. Large numbers of employees are taking large numbers of courses and the choice of future courses is also large and getting bigger. Yet within an organization, the data gathered on who takes what courses is limited. This data, when linked to the content (courses and elements within courses), if gathered and put to use through AI, can provide insights leading to recommendations for both organisations about demand, needs and activity, but also at the organizational level, recommendations on what courses to supply and to whom. It is not just Coursera who are doing this, Udacity also have an AI team who have produced interesting tools using sentiment analysis and chatbots that recommend courses. We've also done this at WildFire.
Macro-level course selection for individuals
At WildFire, we’ve developed an AI recommendation engine that doeS some nifty analysis on data from completed courses and recommends other courses for you from that data. At WildFire we take databases of course interactions (courses taken and user-course interactions with learners), then use model-based collaborative filtering to create a scoring matrix. These matrixes are sparse and need to be filled out using correlations that create unknown user interaction grades, and these are used to cluster courses into groups, based on similarities. We then use unsupervised learning to identify clusters of similar specialization, to find areas of overlapping specialization. We have sets of specialisations that are similar to each other, insights that can lead to a better investment in certain courses and determine what courses should be taken by whom.
Micro-level insights and decisions within courses for individuals through adaptive learning
In learning, recommendation engines can also be used to recommend routes through courses, even recommend learning strategies. They promise to increase the efficacy of online delivery through personalised learning, each learning experience being unique to each learner, drawing on data about the learner, other learners who have taken the course, as well as all data from other courses taken by those learners. As learners will vector through learning experiences at speeds related to their competences, they will save time, especially for the faster learners as well as reducing drop-out from courses, by learners who need more individualised support.
Recommender engines lift traditional online learning above the usual straight HTML delivery, with little in the way of intelligent software, making the experience more relevant and efficient for the learner. They also provide scale and access from anywhere with online access, at anytime. If their efficacy can be proved, there is every reason to suppose that their adoption will be substantial.
Micro-level recommendations for external content within context while doing a course
In WildFire, we create content, in minutes not months, using various AI techniques. In sense the AI identifies learning points and creates user interactions within that content but it also has a ‘curation’ engine that recommends external content, linked to that specific piece of learning and produces links to that content automatically. This creates content that satisfies both those who need more explanation and detail, as well as the more curious learner.
This is exactly how experienced learners learn. One thing sparks off curiosity in another. In this case, we formalise the process with AI to find links that satisfy those needs. Sometimes it will be a simple internal document, at others external links to a definition, YouTube video, TED talk or other trusted and selected source.
Micro-level insights and decisions for individuals through assessment
Coursera have been using IRT (Item Response Theory) into machine learning software to analyse learner’s performance from assessments. This allows you to gauge the competences of your employees relative to each other but also other companies in your sector or generally. This is similar to the work done by Chris Brannigan at Caspian Learning.
Conclusion
LandD talk a lot about business alignment but often don’t get very far down that track. We can see that AI has succeeded because it moves beyond LandD into other business units. What it gathers for an organisation is a unique data set that really does deliver recommendations for change. It’s light years ahead of happy sheets and Kirkpatrick. What’s more interesting is that it is the polar opposite of much of what is being done at present, with low key, non-interventionist training. Even in the online learning world blind adherence to SCORM has meant that most of the useful data, beyond mere completion, has not been gathered. This blind adherence to what was an ill-conceived standard, will hold us back unless we move on.
This AI approach draws on behaviour of real people, uses sim/scenario-based data gathering, focuses on actual performance, captures expert performance, uses AI/machine intelligence to produce concrete recommendations. It’s all about business decision making and direct business impact. And here’s the rub – it gets better the more it is used.
There is no doubt in my mind that AI will change why we learn, what we lean and how we learn. I’ve been showing real examples of this for several years now at conferences, built an AI in learning company that creates online learning in minutes not months - WildFire, invested in others. Learning and Development has always been poor on data, evaluation and return on investment, relying on an old, outdated form of evaluation (Kirkpatrick). It’s time to move on and address that issue square on, with evaluation based on real data and efficacy based on actual demand, using that data.
No comments:
Post a Comment