Wednesday, December 23, 2015

An AI online tool that students really do use to gets results

Interesting chat with a bunch of high performing students doing a high level course on artificial intelligence, at a top European University The course is around 50% pure maths, the rest data analysis, AI research, coding and neuroscience. Around 100 students started the course, unbelievably (and immorally in my view) there’s less than 30 survived through to the second year. These second year students are no slouches.
Lectures inadequate
Lectures are not recorded, much to the disgust of the students, who constantly complain about this. (even formally). The students complain that, for high-level maths, this makes absolutely no sense, in terms of teaching or learning. The lectures are experiences where the students desperately try to fathom out what is being covered in the course. They salvage what they can, then the real teaching and learning begins. Their lectures are all two hours long (wow!), with as many as four, two-hour lectures in one day. There’s a huge amount of contact time, as well as projects. What is not clear is how productive that contact time is. Complaints fall on deaf ears. In a sense this has forced the students to create their own teaching methods.
What online tools were they using?
They do, of course, use a huge range of online resources both after and (interestingly) during lectures. Khan Academy, academic papers, online textbooks,  but one tool surprised me. It’s called ‘Anki’, a spaced-practice, flashcard system. The tool encapsulates several simple principles in learning theory; self-generated content, chunking, active learning, recall as reinforcement and repeated practice. Theses are well-researched principles that are largely ignored in higher education. We have known about spaced practice for over a century, one of the most researched principles in learning, yet studiously, even wilfully, ignore the principle in practice. It is not difficult, in our connected networked age, to deliver this type of learning. Our learners have devices, laptops, tablets and especially mobiles that form an umbilical chord for learning. What impressed me wasn’t the tool, which I knew of, but the way in which they used the tool.
Supervised learning
You rate your own answers by choosing a rating; a spaced-practice interval. But here’s the clever bit, the algorithm, which is very sophisticated, also takes part in deciding what is delivered at what point in the future. (The algorithm is based on a well known algorithm for spaced-practice called SuperMemo.) On top of this, the ‘supervised learning’ is a combination of learner and system judgements. This no simple repeated practice system. The algorithm is very sophisticated, as is the input by the learner. These are AI students who fully realise the power of supervised learning, the combination of human and technology to optimise learning. They’re soaked in machine learning but also recognise its limitations, which they are constantly pushing out.
Souped-up note taking
These students were writing their own cards. They all use HTML or LaTeX, as it copes with the maths. They are way beyond Word. This act of self-creation is a great learning task in itself. Self-generated content is like souped-up note taking and results in significant increases in retention and recall. The students I spoke to recognised that they’re reinforcing content when they create the cards, as well as when they practice using the card decks. The addition of audio for language learning is also useful. 
Shared production
They even share out the production task, so that each converts certain topics or lectures into cards for shared use. This pooling saves everyone some time, time that can be more usefully spent practising. One of the great things about this course is its project work. They’re set high-level tasks and randomly put into groups, then marked on the results. If you're going to do a computer acinece or Ai degree do one that has tons of project work. It works.
Shared gamification
They even run ANKI group sessions in twos or threes. Flashing a question up and going for the quickest to answer, a sort of live, gamification technique. This is a pedagogic technique invented by smart learners for other smart learners. An interesting angle here is that they especially like answering questions written by their colleagues, as this shows deeper understanding and prepares them for more possibilities in their understanding and exams.
Shared server
There’s a server AnkiWeb that synchronises delivery across a range of devices. Not that they'd use a tablet, as that would be a waste fo time and money in IT. There’s also a huge number of decks, especially for language learning, that are available for free.
It also has its own analytics to show you what you've been doing across your decks, in terms of usage and success. That's useful feedback when you're trying to optimise your learning time.
Languages & plug-ins
There's a huge range of plug-ins, such as text to speech with playback/recording, editing features and even Memrise, courses in languages and other subjects, with adaptive learning. The card system is heavily used by students learning a second language as it has this text to speech capability in many languages. It even copes with Japanese and Chinese (well used).
Mobile learning
Cards are produced on a laptop and they use it on laptops but the mobile version is used for revision, especially in cafes, on trains and so on. Fascinatingly there’s lots of use in lectures (when the lecturer is poor). This really is that rare beast – optimised, mobile learning.

What is odd is the gap that has emerged between learners and those who teach. The gap has widened as ‘lecturers’ hold doggedly on to the twin pedagogic pillars of lectures and essays, while students find their own ways to learn. It is this lack of interest in learning and the tools that help students learn, that is puzzling. But what realy puzzles these smart students is why no one told them about these tools. They had to discover them by themselves. Isn't that odd? Maybe not. There’s an odd form of pedagogy that has been around a long time in our Universities, of not making life too easy for students, the idea that struggle is part of the process. This tool should be recommended for all students. You need not make any of this compulsory but surely giving students tools that take them beyond the lectures that fail them is worth trying. It is, of course, more suited to learning underlying knowledge but they have also moved into creating cards that pose questions that need the application of that knowledge or principle. This is not just a fact checker, it is high-level knowledge and skills. The one line I heard time and time again was “I wished I knew about this at school or at the start of the course”. These students know good pedagogy when they see it because they create and use good pedagogy for themselves, things they know work. Their views of lectures are, by and large, one of complete contempt.


sailjack said...

Donald could you please elaborate on how these students use Anki and integrate it with more classical pedagogical approaches?

I am a student in a highly quantitative field and, despite using Anki on a regular basis to learn foreign languages, I find it's use it pretty useless when applied to university related content, especially in quantitative fields.

I've used to help me prepare for some courses but these are my findings:

1. You loose the big picture ---> in order for the cards to be effective, they must contain a limited amount of informations; frequently this means that you have to fragment a concept in tiny bits of information, de facto losing the big picture.I find the cards useful to memorize a mathematical formula, or the properties of a certain object (in my example it could be the properties of OLS regression residuals) or the text of a certain theorem but I find it useless to learn how to derive that formula or how to prove a theorem. In my opinion flash cards promote an isolated learning experience not inviting the student to build links between different (but related) concepts.

2. It promotes a memorization mindset instead of comprehension one ---> with flashcards you are lured to just create a cards with the excuse to reinforce learning, for example putting in it a mathematical expression, instead of trying to understand what the expression really means, how it can be derived, how it can be visualized etc. Sure, it could be useful on a short term basis, when you need to learn that particular expression or concept in order to excel in a test, but in the long term you just lose it because you did not understand the concept when you learned it. I found a visualization approach or an approach like this ( much more effective in the long term. Even for languages I found that on a long term you recall better the words learned not using the flashcards but by placing them in context in order to understand how they are used by native speakers (in this case the real-world use of the words better reinforce the neural patterns while the flashcards system makes you learn the word but it does not force you to use the word, sure, in the future when you will see the word you will probably recall the meaning but it's as probable you won't be able to use it in an active way)

3. It is time consuming ---> especially when you have to deal with online textbooks, different papers, online lessons etc. to build your knowledge base (and I find this situation pretty frequent), using flashcards is just an added burden to the learning process. Writing cards is time consuming: it's true, if done in the correct way it could help you to reinforce the content, but speaking with other students using the flashcards system, I found that the most of them just put information in the cards without really thinking about it; they know the method will produce some sort of yield in terms of better retention and with this idea in mind they just produce an incredible amount of cards wasting enormous amounts of time.

4. recalling is not always so efficient ---> when producing a vast amount of cards, even recalling becomes time consuming. When learning new words you usually put a roof on the maximum number of words you want to learn in a day (let's say 20). Depending on your proficiency in recalling the words, even with a learning rate of 20 words per day you find yourself reviewing 30/40 words learned in the past days, every day. With more complex topics this roof idea does not work, you are not able to limit yourself to a certain amount of cards and in this way you may find yourself in the need of reviewing up to 100 cards in a single day, not an ideal situation.

Since from what I can see in your article these students are really into learning with the flashcard system, I would like to know how they blend this approach with more traditional ones.

Donald Clark said...

They generate their own questions and take care to construct questions that truly reflect deeper understandingthan just atomic facts. This takes time, hence their willing ness to work in groups to generate good content. You seem to be after a silver bullet that solves all problems. ANKI is not that bullet. It is just one of many tools one can use to learn.