Monday, December 23, 2024

Raising in Arizona with AI: groundbreaking journey in AI ducation

Arizona has taken a bold leap in education by approving a pioneering model: a charter school where artificial intelligence takes the lead in teaching. Set to launch next year, Unbound Academy will be a fully online institution catering to students in grades four through eight. 

Traditional teaching will be replaced with two hours of daily academic instruction tailored entirely by AI. Leveraging platforms like IXL and Khan Academy, the AI will adapt lessons to each student's pace and abilities, analysing emotional cues, task completion times, and responses to create a personalised, frustration-free learning experience designed to maximise engagement and progress. That's the claim at least. But there have been many attempts at online schooling.

For the rest of the school day, students will take life-skills workshops on topics like financial literacy, entrepreneurship, and critical thinking, guided not by teachers but by facilitators known as 'guides'. This innovative approach stems from Unbound’s success with its private school in Texas, where it claims to have doubled academic outcomes with fewer instructional hours. Inspired by Elon Musk, Unbound aims to refine this model for potential expansion into states such as Arkansas and Utah.

They see this as a much-needed shift to address inefficiencies in traditional education while preparing students for an evolving world. Backed by influential supporters like the Bill and Melinda Gates Foundation, Unbound’s AI-driven school could mark the dawn of a transformative era in education—or spark ongoing debate over its impact on accessibility and quality. Let's see...

Saturday, December 21, 2024

Why Latin should NOT be in the school curriculum


The venerable Mary Beard invited me to debate the ‘Future of Latin’ at the British Museum some time ago and every few years, normally in line with the electoral cycle, the issue arises again. So how did it go and what were my arguments?

Sell-out

A sell out, with 350 paying Latinists, who turned up to hear Mary chair a debate which pitched David Aaronovitch and I against Peter Jones and Natalie Haynes. As we walked on stage and introduced ourselves (I found that I was the only person who didn’t go to Oxford), everybody seemed to know each other (except me). This is exactly the sort of challenge I like, as although it was a sell-out, I’m not in the habit of selling-out on my beliefs and principles.

Neither contrarian nor philistine

To be clear, I was not there as a contrarian or philistine, as I’ve been in love with the classics since I was a boy. My first secondary school lay astride the Antonine Wall and for over 40 year I’ve been to every corner of the Roman, Greek and Egyptian Empires, from Scotland to Syria. I cycled Hadrian’s wall, still go to Greece every year and never miss an opportunity to visit sites, especially on the Peloponnese. and I go to Egypt, as I do almost every year, for another dose of Egyptology in December, and have done since 1989. 

I am, however, also a rationalist and realist, and my 30 plus years of experience in the learning game have made me deeply suspicious of the position of Latin, among many other things, in our culture and school curriculum. 

As Bertrand Russell said:

I was made to learn Latin and Greek, but I resented it, being of opinion that it was silly to learn a language that was no longer spoken. I believe that all the little good I got from years of classical studies I could have got in adult life in a month.

I was there to argue that it should NOT be taught in schools at all. These were, and are, my arguments. Note that I am not against Latin the language as an object of study, I am against it being taught in schools. The arguments for each of these propositions were presented by my opponents and audience members. These were my responses.

1. LATIN does NOT help you learn other languages

Why scratch your ear by going over the top of your head? Isn’t it obviously easier to just get on and learn Spanish, Italian and French, rather than the convoluted route through Latin. Researchers Thorndike, Sherwin, Haag & Stern all think so. In the Sherwin meta-study 'Research and the teaching of English, “the study of Latin does not necessarily increase the ability to learn another language… No consistent experimental evidence in support of this contention was found.” Learners have limited time, that time is clearly better spent on the target language itself. 

In fact, Latin can make learning a new language MORE difficult. In Search of the Benefits of Latin by Haag and Stern (2003), who followed up on Thorndike’s work nearly a century earlier, in the Journal of Educational Psychology is the key paper. They took two groups of German students, one who studied French, the other Latin as their second language. Both groups were given a course in Spanish and the results measured. When the results were analysed by a Spanish assessor (who didn’t know who had taken French or Latin) the French students made significantly fewer grammatical errors than the Latin students. As predicted the Latin students wrongly transferred the rules of Latin to Spanish. For example “misconstructions in verbs emerged to be either highly reminiscent of or identical to Latin verbs”. The French group turned out to be much better prepared to cope with Spanish grammar.

Psychologically the Latin students had suffered from negative transfer using false friends in their new language. The problem with understanding Latin is that you need to pay close attention to word endings; case markers on nouns and time markers on verbs. But in English and Romance languages word order and prepositions are more important. Endings play a minor role. The fact that the grammatical similarities between modern Romance languages are much greater than that between Latin and modern Romance languages, means that the defenders of Latin are flogging a dead horse. Thorndike was right – transfer of the wrong kind occurs.

2. LATIN does NOT have an edge in improving cognitive skills

This argument is greatly loved by the parents of ‘gifted children’ although I rarely come across a middle-class parent whose child is not gifted. For gifted, read ‘pushed’ (not a bad thing but very different). Again Haag and Stern (2000), in a review of the literature found that Thorndike, “did not find any differences in science and maths in students who learned Latin at school and those who did not”. Two groups of comparable students, where one studied Latin, the other English, were assessed after two years, “No differences were found in either verbal or non-verbal IQ or grades in German or Maths”. This again had been predicted by Thorndike decades before, namely that transfer needs common ground in the source and target.

3. LATIN does NOT give significant advantages in using English

English is a Germanic language – we are largely speaking in Old English rooted language. The TOP 100 words are Old English (sorry three are Old Norse – THEY, THEIR, THEM). As for one audience member’s argument that it is necessary as all children need to be able to understand etymology, I disagree. Is there anything more annoying than the dinner-party bore who stops you and explains the root of a word, as if it made any difference to your argument or its contemporary meaning. If anything a good course in Old English would be better. Do you really have to go through years of Latin so that you understand the roots of Homo sapiens?

4. LATIN does NOT guides us correct use of English grammar

Stephen Pinker, Harvard’s world renowned expert in psycholinguistics backs this up in The Language Instinct, “Latin declensional paradigms are not the best way to convey the inherent beauty of grammar”. He recommends computer programming and universal grammar on the grounds that they are “about living minds and not dead tongues”.

Pinker has a go at the Latin language mavens who want to pointlessly foist Latinate rules of grammar into English. As Pinker explains, this snobbery took root in 18th century London, when Latin was used as a mark of social class (still true today) and Latin grammar rules were crudely pasted into books on English grammar, for example, ‘don’t split infinitives’ and ‘don’t end a sentence with a preposition’. Latin simply doesn’t allow you to split an infinitive and to stupidly insist that it’s wrong in English, is as stupid as making us all wear togas.

English does NOT have Latin grammar. English grammar fundamentally Germanic. Latin highly inflected, role of word in a sentence indicated by its endings. English relies primarily on word order to determine grammatical relationships. Latin nouns gendered English are not.

5. LATIN is necessary for SCIENCE, LAW and MEDICINE

One girl in the audience, from Merseyside was adamant that Lawyers needed to have studied Latin. Incidentally, if you’ve heard the argument that Latin helps medical students learn and understand the considerable amount of medical vocabulary that has to be learned in medical schools. This also turns out to be false as shown in Pampush and Petto (2010). The Latin vocabulary in law, medicine and biology is there but one can simply learn the words, without knowing their etymology - most lawyers, doctors and scientists do exactly that. Why spend years learning Latin to help you in a small selection of vocabulary? The pay-off is, as Russell pointed out, is tiny.

6. LATIN brings the joy of ideas and literature

One contribution from the audience I did like was the idea that Latin bridges us to rich tradition of thought and literature. He mentioned Roman literature but also the Magna Carta, Bacon and Newton. First, anyone studying history will not lose out by working with translations of the Magna Carta or Bacon. And does anyone really need to read Newton’s tortuous Latin, other than scholars in the history of maths? I think not. 

And why not Greek? Wouldn’t you prefer the riches of Plato and Aristotle, Homer’s Iliad and Odyssey, along with the works of any one of Aeschylus, Sophocles, Euripides and Aristophanes to the largely derivative Roman philosophers (few can name any) and dramatists. Even in History, the Greeks Thucydides and Herodotus are a match for any Roman writer, including Tacitus. In politics, our democratic traditions are largely Greek. Even English law is not Roman (although in Scotland it is). Then there’s the politics and democratic traditions that are fundamentally Greek. Even in maths and science the mighty Pythagoras, Euclid and Archimedes trump the Romans.

7. LATIN’s history of exclusion

Of course, Latin was introduced to this country as the language of the clergy, it was not, as is sometimes supposed introduced by the Romans, as few Latinate words come from that era. As the language of the church it largely excluded the laity, as most remained illiterate and spoke various forms of English. It was then used as the gatekeeper for learning. This had some benefits, when Latin was sort of European Esperanto, but continued for centuries after that died and was long used as the gatekeeper at Oxbridge and other institutions. David Aaronovitch made the telling point that it is still a key ‘marketing’ differentiator for independent schools.

8. LATIN would NOT die if not taught in schools

There are plenty of scholars in subjects that are not taught at schools. The bottom line with any dead language, especially Latin, is that there’s little that is new and to be uncovered. Compare this to the vast amounts of Sumerian cuneiform tablets that still need to be both deciphered and excavated. In the end I agree with one of our greatest living Latin scholars Mary Beard, “the overall strength of the classics is not to be measured by exactly how many young people know Latin or Greek from school or University. It is better measured by asking how many believe that there should be people in the world who do know Latin and Greek.” This about sums up my position. 

I am not against the study of Latin or any other historic languages. This is largely a matter of proportionality for our Universities. By all means let a few study Latin. What I am against is too prominent a role for Latin in contemporary school curricula. Our young people have enough on their plate at 5-18, as the range of subjects expands to include a wider range of science subjects, IT and other vocational skills. A dead language, in the sense of no longer being spoken or used for scholarship, at this stage is merely the dead hand of educational history being played out by interested parties.

9. LATIN is NOT about choice

Several people argued that Latin is a matter of choice in schools. The curriculum is crowded enough with increasing demand every year as new subjects, such as computer studies, emerge. As we have seen from recent educational lunacy in policy, has resulted in the destruction of choice in the curriculum. Professor Alison Wolf, Gove’s lapdog, ignored advice from industry and education experts to crush 3100 vocational qualifications. That strange beast, the EBacc, which had Latin as a core choice, deliberately excluded all vocational subjects, even ICT. Gove's successors , such as Williamson did the same, with special temporary funding, creating a one-sided system that simply reinforces the old apartheid system we have in this country between academic and vocational learning. 

This is not to say that all education should be aimed at utility and employment, just that a contemporary curriculum in schools is always a trade-off, with new subjects and content always arriving, so that everything we taught in the past can’t just remain. We need to weed and feed a curriculum, which means some hard choices. My preference would be to focus more on modern languages themselves, where we have declining numbers.

10. LATIN is a political issue

It has now become a political pawn. A largely (not exclusively) right wing movement to impose private school norms (7%) on state schools (93%) became an electoral issue, hence Williamson's special funding.  Labour won and implemented their promised VAT policy on private schools and in reassessing budgets they also scrapped Williamson's artificial subsidy for Latin in state schools. We can argue for or against Latin in schools but this was democracy in action.

Conclusion

I rather liked this audience and I especially liked Peter Jones.  He was my opponent but put to bed those old tropes about Latin improving your ability to learn languages, improve intelligence or think logically. He was remarkably free from cant and any sense of snobbery. What he loved was Latin and his plea was for the beauty and intrinsic value of the language. With that I have no argument.

Friday, December 20, 2024

AGI is here...


As easy as one-two-three? No. Easy as one-three. OpenAI has skipped o2 to call their new ‘frontier’ model o3 (Altman hinted that this may have been due to Telco o2 issue). I like the idea that AI is undermining everything we were told makes good marketing - no read branding, messaging, just get it out there with benchmarks.

Social media was full of people saying Google had destroyed OpenAI. But their pre-Christmas release single was smashed off the Charts by OpenAI’s o3.

Transcending human intelligence

Not sure we've realised that we now have AGI. Arc-AGI solved with 87.7% (human threshold 85%) as well as other benchmarks... we will, of course, move the bar higher, find ways to avoid recognising the achievement... then pass it again... Arc Prize is a not-for-profit with an AGI benchmark. There are new skills so that the AI cannot memorise them.  

To give you some idea of the speed and scale of progress:

Software engineering

On real-world software tasks, evaluations on o3 are more than 20% better than o1 models at 71%. AI has already established itself as solid, useful and widely used coding tool. People have been letting AI code then iterating  iterate. This takes it to another level. One wonders how long the job of coder will survive at this rate.

Maths and science

Also superb at maths and PhD science questions at 88.7 %. A typical PhD gets around 70%. Frontier is toughest mathematical dataset – extremely hard problems. o3 has 25% accuracy which is a strong result.

AGI

Artificial General Intelligence suffers from a problem of definition, as do most abstractions at this level. Does it mean:

Specific human reasoning skills (maths, science etc)

Average human competences

Greater than all of humanity

There is no singularity, there is a spectrum or constellation of possible targets here. What is clear is that these targets are being hit, not in one go, but one by one, sometimes in clusters. Maths and science are easy to measure but also fiendishly difficult to achieve, so this is a real milestone. Yet they focus on clear rationality. To be fair critics were telling us that AI would never get here, never mind get here so fast. We should celebrate this as many of the problems we face with climate, energy, healthcare and education may well be solved, not in the sense of final solutions but better solutions.

Problem solving in real life is messier and more of a challenge. That's why the agentic move is so interesting, as it tackles this set of human capabilities. We have brains that have evolved into a specific environment where we had to solve specific problems. This is where the dynmaic interrogative, dialogic nature of AI helps enormously. It has already made great strides in this direction.

Embodied AI, in the physical world of elevators, cars, cabs, trucks, drones, ships and submersibles has also taken great leaps. This is another set of targets that are being hit one by one. One could argue that neurological targets are also on our hotlist - Neuralink is a good example, where we enhance our neurological deficits.

AI may help us understand neuroscience and the brain, solve engineering problems to accelerate fusion, help with drug discovery and many other intractable problems. What we can be sure of is the increased impact of AI on productivity. The leaps in efficacy prove this.

Conclusion

This is a warning to people who claim that scaling is over. Sutskever was right – we have a way to go and other techniques are clearly delivering the goods. It is clear that AI is delivering faster than expected. The consequences of AGI are closer than expected with huge productivity gains on the horizon. That is the subject of the book I am currently writing.

Future issues

One issue needs discussion - compute costs. I think this will be solved. We saw a 250x decline in token costs in 20 months. So $20 for hard problems are likely to come down to cents.

One can now start to ask how the cost of compute compares to human costs in organisations for similar tasks and roles. The productivity game looks as though it will start with coding, where much of it can be automated.

On problems to solve:

  • fusion
  • medical research
  • personalised tutors
  • optimising political policies
  • next-generation batteries
  • cheap renewables

A final though on AI being self-generative. At what point does this technology start working on itself to solve the frontier problems and advance even quicker?


 

Fast technology and the slowcoach brain – fascinating paper

The findings in this paper by Zheng and Meister 'The Unbearable Slowness of Being: Why do we live at 10 bits/s?are quite startling when you consider the vast disparity between our sensory input capacity and our cognitive processing speed. It's almost as if we're living in slow motion within a world of high-speed data. This realisation challenges our intuitive understanding of human intelligence and efficiency. 

Shaped by evolution

It's astonishing that despite our sensory systems being capable of absorbing vast amounts of data, our cognitive output remains so limited. The surprise at this slow processing suggests that perhaps our brain isn't designed for speed but for careful, deliberate processing which might be more suited to survival in our evolutionary context. Humans and other species might operate at rates sufficient for their ecological niches, suggesting evolutionary constraints. Human cognition tends towards serial processes, possibly due to evolutionary origins where multitasking was not advantageous.

Humans process information at an oddly slow rate of about 10 bits per second (bps). This contrasts starkly with our sensory input capacity of approximately 10^9 bps. Even expert typists operate at around 10 bps, English narration speed suggests an information rate of about 13 bps, blindfolded speed-cubing has a perception rate during the puzzle inspection phase of around 11.8 bps. Top-end memory tasks like binary digit memorisation or speed card inspection reveal rates from 5 to 18 bps, showing that even under high memory demands, the rate remains within the 10 bps ballpark.

Bottleneck

The bottleneck is not neural capacity, it is throughput. Sensory neurons, particularly photoreceptors, can handle gigabits of data, while the brain's behavioural output remains at 10 bps. The paradox is that, despite having neurons capable of transmitting information quickly, when it comes to human behaviour and cognition, we are slowcoaches. This limitation seems to be fundamental.

Photographic memory claims turn out to be false. Even with exceptional memory, the data acquisition rate doesn't significantly exceed 10 bps. Our perceived richness of visual scenes is an illusion, as actual perception is much more limited, especially outside the focal point.

Let’s think about this, albeit in a limited and slow fashion! Even with advanced technology like Neuralink, we'd still be bound by this 10 bps limit for cognitive throughput. This is a sobering thought. If our brain operates at such a slow rate, it might imply that human ‘intelligence’ is less about raw computational power and more about the quality and nature of information processing that focuses on depth rather than breadth.

Conclusion

Neurons are not inherently inefficient; rather, the brain's architecture seems optimised for serial processing over parallel. With AI we have several dimensions of scale, in terms of relevant data, retrieval and parallel processing. AI may be offering a way out of this human envelope, that of being fixed, linear and slow, to augmenting our abilities with technology that is flexible, parallel and fast.


Does ChatGPT enhance student learning? Meta-study with some surprises!

Great to see a meta-study asking this bold question:

Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies.

See study here.

STUDY

Nice spread of subjects and focus on classroom teaching. Predominantly university-level interventions (84%), with a smaller focus on K-12 (14.49%), language education is most studied (32%), followed by computing, health, physics, and education. Most of the interventions occur in classroom environments (87%) with a duration of 5 to 10 weeks. The AI was primarily used as a direct learning tool (80%), with some integration into broader educational platforms.

RESULTS

The results were, I’s say, quite startlingly positive. Many are decrying this technology as something that depresses academic performance, demotivates students, and blocks higher-order critical thinking. The study suggests the opposite.

Academic Performance

Significant improvement with ChatGPT use, though concerns about whether this is due to the intervention or the quality of AI-generated content.

Affective-Motivational States

Positive effects on motivation and emotional engagement

Higher-Order Thinking Propensities

Positive influence on self-reported propensities.

Mental Effort

Significant reduction, suggesting less cognitive strain with ChatGPT use.

Self-Efficacy

No significant impact observed.

The study covers a broad spectrum of educational contexts and subject areas, providing a pretty wide view of ChatGPT's application in education. I’ll leave the methodological rigour questions for those more qualified but I liked the ambitious approach to examining a range of learning outcomes, offering insights into some key dimensions of student learning. I was surprised that self-efficacy was not positive – interesting finding as it runs against the affective-motivational finding.

Sure there are weaknesses; some small sample sizes, sometimes little pre-testing and one odd one - ChatGPT sometimes used in assessments. But this is a good start.


Sunday, December 15, 2024

Reclaiming Productivity: Aligning Work, Learning, and Societal Needs

‘Productivity is being able to do things that you were never able to do before,’ said Franz Kafka. That’s pretty much the same as learning. I’d like to see education and workplace learning be more aware of productivity as a worthy idea in this wider sense of the word.

Nobel Prize-winning economist Paul Krugman also said, “Productivity isn't everything, but in the long run, it is almost everything”. He has a point. Producing autonomous and productive adults is surely what we aspire to as a collective goal of a nation. A country's ability to improve its standard of living depends on its ability to raise its output per worker. Isn't that what we want from L&D?

There is a sense that we’ve lost something; people not caring, huge numbers not in employment, yet massive skills shortages. In the UK we have an economically-inactive group of 9 million working-age people. Around 1.5 million are unemployed, about 2.7 million are students (here, I’d argue, we have too many staying on for too long in HE). We have 2.3 million ‘sick’, which is shocking and 1.5 million as carers. We should be tackling this head on.

We have an education system way out of kilter with societal needs. A realignment of education to life and a living would help. The looking down upon, defunding and eradication of any vestige of vocational learning from curricula in schools and Universities has not helped. We want more houses but don’t have the skilled people to build them; green energy but have a generation who can hold placards, sit in front of traffic and throw paint at art but can’t actually do anything practical to build the green technology we need; more nurses but have turned it into a graduate profession disenfranchising working class entrants.

In workplace learning we need to stop distracting ourselves with abstractions. The 'Skills-Based Organisation' has long been an empty trope, because we have been seduced into thinking that abstract nouns like leadership (lots of spend but so little of it), culture, diversity, equality, values, inclusion, resilience etc are 'skills' or some mysterious miasma that will encourage and produce skills. 

The very idea that skills have anything to do with text-based learning and assessment or sitting around tables scribbling on pieces of flipchart paper is laughable. A renewed focus on productivity would help but we're not much interested in actual outputs, only abstract inputs. Training so often kills productivity by distracting people from reality. People love a cosy chat with others who agree with them. But this is so often a zero-sum game - the net gain in terms of change, skills, productivity is zero. In fact it may be a negative-sum game as it takes up tons of time, costs and tends to fix people into old, fossilised behaviours, meaning absolutely nothing changes or improves.

Productivity is the ratio of output to inputs, which can be applied to people, not cruelly or mechanically but sensitively. Yet we flee from evaluation and measurement as we’re scared of what we may find. Productivity can be difficult to measure and improve but it translates into more satisfied employees a better standard of living for all and the crazy idea that your kids will be better off than you. 

Astounding talk by Sutskever

A 30 minute talk given yesterday by Ilya Sutzkever already being seen as seminal.


We have gone through a decade of astonishing advancements in neural network learning. We were right to equate an artificial neuron with a human neuron. Biologically inspired AI has been hugely successful. It led to huge success through scaling. We forget that we were using crude string matching, tables and statistical techniques. We worked in this world and from 2014 were successfully producing real product. But we knew its limitations.

PEAK DATA
Then came GTP 2.0 and LLMs. These have been astonishingly successful. On speed alone they do what takes humans ages to complete, they are also often better on quality. But Sutskever’s big message was that pre-training will end due to data limitation, peak data, the fossil fuel of AI.

What do we do after pre-training? Agents, synthetic data and OpenAI o1 are examples of attempt to move forward. He draws from biology showing the relationship between body size of mammals and their brains. Hominids have an accelerated brain to body ratio. We are different. But so is AI, which can also be different. This is evolution on steroids. What took billions of years to achieve through evolution to hominids, is being done in just a few years.

SUPERINTELLIGENCE
Progress in AI has been remarkable but he thinks we are heading towards ‘superintelligence’. At the moment we have amazing systems which in many areas (evaluated) exceed human capabilities. They still get a little confused at times, who doesn’t. Hallucinations happen but models will eventually autocorrect. This may already be happening.

We have lots to gain from current AI, as it is so incredibly useful. Even if we stopped right now there is so much that can be done with the current technology to keep us busy for a decade. It is a productivity boost in may areas of human endeavour as well as being a discovery tool.

Systems are only very slightly agentic but agency, reasoning and self-awareness will happen – why? They are useful. The human brain, stopped growing in size but we got stronger cognitively, so humanity kept advancing. Similarly, the agents and tools on top of LLMs will fuel the progress through agentic behaviour. This future AI will possess unpredictable capabilities and self-awareness, transforming their capabilities.

FUTURE
He left us hanging and is very careful not to become too flippant and certain of himself, like some silly, keynote futurist. He has worked on the leading edge of this technology for decades and has contributed hugely to the field. While careful about predicting the exact nature of future development, which he thinks we cannot predict with certainty, he nonetheless, paints a picture of rapid progress and achievement.

Monday, December 09, 2024

Tyranny of Text: Education, Work, and the AI Revolution


I have just visited the Ramesseum, and the statue that inspired Shelley’s ‘Ozymandias’. This part of the world seems cursed by being the place where ‘writing’ was first invented, religious books written, each granting different groups a sense of stupid, eternal entitlement. They’ve been at each other’s throats ever since. When people dream of text-based heavenly Paradises, they can't see worldly Paradises in front of their eyes.

We are drowning in a sea of 'text' in learning. From 5 to now 25, young people spend almost all of their time reading, writing and critiquing ‘text’ in an educational system because it is easy, creating the illusion that you can assess most skills through text -  you can’t. 

We have decimated vocational learning by sucking up funding into often purely text-based Degree subjects. Lecturing is easy, teaching is hard. Setting essays is easy, assessment is hard. Producing text is now easy, doing things is still hard. Once you see LLMs as producing text as calculators produce good numerical solutions, you relax a bit on AI.

Less is more

AI is not about generating more text. Its true purpose has been in generating LESS text; summarising beautifully, producing grammatically perfect text with no spelling or punctuation errors. Most text is not in books and essays but in mundane communications, as emails, messaging and social media – and AI is largely an aid to communication.  In work and life, communications is often over-long, badly written and error prone. That problem is being solved.

Young people are quite adept at short, concise messaging, they do it all day every day. The problems come from people fed on a diet of long-form text, who tend to see everything as a potential essay, so in organisations and government bureaucracy, which existed long before AI, is text-heavy, the production of unnecessary forms, documentation and reports. AI will optimise and automate this.

Learn by doing

Most of work and life is about doing, most education is about writing. Yet much text production in white collar work may well be automated out of existence. Let’s recognise that AI is now fundamentally multimodal, using speech, images and video, also creating 3D worlds, robots and automated vehicles. This is not to eliminate text, just see it as not primary and over-egged in education, work and life. With multimodal AI and robotics, AI is moving fast into the real of doing, both teaching us how to do things and doing things for us and in place of us. The lines are now blurring, with increased scepticism over text-only education. Education is for both life and living.

AI may have seemed like a text-based phenomenon, but it is proving to be more about communication, speech, robots, data and automation. We will be made more productive by having less text, automating as much as we can out of the system, dissolving text-laden bureaucracy. Most of the critics of AI come from those who deal with text as a living, in education and work.

PODCAST on Invention of writing
PODCAST on Literacy & Orality

Tuesday, December 03, 2024

How do emotions impact learning and mental effort? Maybe not as we think


I’m deeply suspicious, not of the claim that emotions are important in learning, but of the assumption made on the back of this, that emotion is quite simply a good thing in learning. It leads to all sorts of wrong assumptions about fun, gaming and no end of odd theorising about happiness.

Like the shallow side of social constructivists, who simply conclude that all learning should be ‘social’, whatever that means, there is a tendency to see ‘emotion’ as a good thing, no matter what. This problem has been exacerbated in recent times by the therapeutic assumption that emotions are somehow intrinsically virtuous.

New study

So this study caught my eye (thanks Carl Hwenrdrick) as it digs into something I had always thought may be true, that positive emotions don’t work the way you think. 

https://www.sciencedirect.com/science/article/pii/S1041608024001900

While feeling good (like being happy or motivated) definitely helps with learning, it doesn’t seem to do so by lowering the mental effort (via cognitive load) needed to process new information. This challenges earlier ideas that positive emotions expand your mental resources.

Emotions aren’t set in stone and shift while you learn. For example, frustration at the start of a tough task can turn into satisfaction as students figure things out. So, frustration isn’t always a bad thing—it can be part of the process. Emotions also shape cognitive load. We usually think of cognitive load as tied to how complex the information is, but this study points out that emotions directly influence how much mental effort a task feels like it takes. Finally, not all positive emotions are equal. While being in a good mood generally helps, too much excitement or overconfidence can backfire. Students might rush through material, miss details, or oversimplify because they are ‘feeling too good’. This last point is important. 

Emotions and learning

Affective learning deals with the emotional side of learners, their emotions and feelings. These feelings cover a wide range of positive and negative attitudes, interests, beliefs and motivations before, during and after learning. 

Teachers, lecturers and trainers are professional learners and often understate the role that emotions play in learning. Yet speak to any learner and many learners will report not what they achieved in learning but how they felt. Few get through school without feeling bored or indifferent to lessons and subjects that seem dull, remote and irrelevant. Fewer still get through a degree without feeling numbed in boring lectures. On the other hand, successful learners report excitement, engagement and feelings of pride and achievement. The point is that this can go both ways.

This complex world of feelings and emotions is often sidelined by the dominance of the purely rational, academic cognitive side of learning theory. This is partly down to the dominance of Bloom’s silly taxonomy, the cognitive domain being only one of three, the other two the psychomotor and affective are often completely ignored. 

Kahneman

Kahneman posits the idea that we have two brains, in Thinking Fast and Slow; System 1 - fast, emotional and instinctive, also System 2 - slower and rational. I am no longer convinced that the distinction is as clear as we think it is (sometimes expressed as elephant and rider). Our brain has substantial weaknesses, due to its long and messy evolutionary history. We know that it stubbornly procrastinates, fails to remain attentive (attention being a necessary condition for most learning) and easily distracted. It is also subject to emotional pulls and mood swings, even depression. This is both a blessing and a curse. The emotive dimension of learning is often underestimated but it can also distract and over-stimulate. 

Krathwohl

Everyone knows Bloom, but we hear little about the man who completed Bloom’s work in the affective domain, the less known David Krathwohl. Although Bloom's original taxonomy consisted of six categories, when Krathwohl revised it in 2001, he put emphasis on the interaction between the cognitive and affective. With Lori Anderson he also helped reduce Bloom’s cognitive taxonomy down to four categories or knowledge dimensions:

   Factual knowledge

   Conceptual knowledge

   Procedural knowledge

   Metacognitive knowledge

For each of these four, smaller dimensions were identified. He also changed the cognitive processes to verbs and renamed Evaluation and Synthesis as Creation.

Krathwahl then proposed six levels of affective learning:

   Characterization

   Organization

   Valuing

   Responding

   Receiving

My own view is that this Affective Taxonomy suffers from the same hierarchical rigidity as Bloom’s taxonomy in the Cognitive domain. It is far too rigid and hierarchical. Some even argue that there is no real taxonomy of affective learning as it emerges from or is part of the cognitive domain. Affective factors are also difficult to identify and assess as they involve feelings, attitudes, and beliefs, so ignored as something difficult to measure, vague and unimportant. While there is recognition that feelings and emotions play a strong role in motivation and learning, it is rarely be seen as being on a par with its cognitive counterpart.

There is certainly the tendency for schools and academia to focus on text-based, pure reason, as their primary skillset, at the expense of other aspects of learning. This has led to a paucity of research in the area. 

Speak to workplace trainers or sports coaches and you will hear far more about affective learning, as it really does matter. These are ever present in learning and can also be internalised, either to hinder learning or harnessed and used positively by the learner to move forward. So feelings play a strong role in both demotivation and motivation. Understanding their role is essential if you are a learning professional, yet few could name a single theorist in this area.

Panksepp

Jaak Panksepp introduces the evolutionary origins of emotions and warns us that although emotions are vital in learning, they can also hinder learning. Panksepp saw life as being empty without emotions, emotions being survival features, as part of our evolutionary heritage. We do not teach or learn these seven PRIMARY affective systems, as they are innate:

  SEEKING (expectancy)

  FEAR (anxiety)

  RAGE (anger)

  LUST (sexual excitement)

  CARE (nurturance)

  PANIC/GRIEF (sadness)

  PLAY (social joy)

We can learn from these emotions, but we do not learn them, only learn to modulate them,  He did think that they formed the basis of our personality, different emphases producing different personality types. SECONDARY emotional processes are learnt through classical and operant conditioning and TERTIARY emotions are sensory (taste, pleasure, pain) and homeostatic affects (hunger and thirst). 

We, unlike animals, are cognitive creatures, but he regrets the common disregard of emotions and our evolutionary heritage in understanding the foundations of learning and higher cognitive processes. Much of what is presented in traditional learning theory, whether rewards, punishments or reinforcements actually rely on the emotional responses of the brain. Yet emotions are a double-edged sword.

Some emotions, such as RAGE, FEAR and PANIC are not conducive to learning and may inhibit or hinder learning. On the other hand, learning may benefit from the SEEKING emotion, with its feeling of enthusiasm, as it is instinctive for survival, it promotes learning through purpose, anticipation and curiosity. Its absence diminished a disposition towards learning.

Damasio & Immordino‐Yang

Damasio & Immordino‐Yang see emotions and reason as entwined or enmeshed. They not only not only regulate our lives, they regulate learning. Emotion is therefore critical to learning and memories well as playing a powerful role in learning as motivators.

Conclusion

A complex area but we must be careful of being too shallow on our consideration of emotions. It may be that acts of learning or thinking induce emotions, not that emotions are always the well spring for learning. It is also clear that they may limit, cap or damage learning. We must keep a close eye of the detailed research in this area, rather than trite statements about the important and efficacy of learning. 


 

Tuesday, November 26, 2024

Do DEI programmes need a reality check?


You may feel uncomfortable reading this, as it is a contentious area. But contention must not be left to fester, sides becoming increasingly polarised. 

Companies and educational institutions are pouring billions into Diversity, Equity, and Inclusion (DEI) initiatives. But do they work? DEI has to face up to the evidence, and there is a case for rethinking diversity training. Are we getting it all wrong with billions spent, but does DEI deliver? Mandatory training seems to have a backlash problem and unintended consequences. Turns out, the evidence does not support this spend. Could this training be hurting more than helping and do they need a reality check?

PAPER

This paper by the NCRI points out some serious flaws in some DEI approaches, and adds to the growing body of larger and wider studies that came to the same conclusion.

They mostly have no significant effect as they focus on the wrong thing – training not process. Indeed, some DEI programs stir up resentment and suspicion, making things worse rather than better. 

Mandatory training has led to a backlash, as Forcing people into diversity training can create discomfort and even hostility. Nobody likes feeling blamed and you donpt win people over by accusatory tactics.

‘Anti-oppression’ rhetoric tends to polarise people as it seems explicitly political. Materials (identified in the paper) that emphasise systemic oppression and victimhood often make participants view others through a hostile lens.

Another danger is that perception becomes reality, where DEI narratives lead people to see prejudice and unfairness in neutral situations. This exposure to anti-racism or anti-Islamophobia narratives can increase participants’ willingness to punish perceived ‘oppressors’ without clear proof of wrong-doing.

There is also an interesting  discussion in the paper surfacing DEI narratives to controlling tendencies. Some DEI content appears to foster controlling and punitive mindsets, authoritarian traits of authoritarianism This is interesting, and I true to a degree.

CONCLUSION

I’ve been writing about this for nearly 20 years and this is yet another paper that calls for an honest evaluation of this gargantuan spend and shines a much-needed light on the unintended consequences of many DEI approaches. It’s shocking that so much money is spent on DEI without robust evidence of effectiveness or harm mitigation. Evidence like this should be a wake-up call for organisations to rethink how they handle diversity training—but it won’t. Why not? We are in an evidence-free zone, where even discussion of the evidence cannot happen. That’s why The New York Times and Bloomberg shelved coverage 

EVIDENCE

There is a consistent theme here - mandatory, accusatory, or one-off DEI efforts can do more harm than good. The key is building trust, creating buy-in, and focusing on systemic change rather than short-term fixes. On that front I highly recommend Simon Fanshawe’s excellent book ‘The Power of Difference’. He was a founder of Stonewall and has fought for Gay rights all of his life.

An early study, Kidder et al. (2004), tackled the backlash that DEI programs can trigger when framed as affirmative action. Turns out, calling it affirmative action often sets off alarms about reverse discrimination, making people feel unfairly targeted. Instead of promoting diversity, it can lead to resentment, undermining the very goal of the programs. This research was one of the first to flag the danger of how DEI is framed.

Then along came a big one Kalev, Dobbin, & Kelly (2006), who found that mandatory diversity training doesn’t just fail—it can actually backfire. Employees often resist being told they have unconscious biases, which can foster defensiveness. The researchers suggested that voluntary training or pairing it with programs like mentorship schemes works far better, making employees feel part of the solution rather than the problem. Legault, Gutsell, & Inzlicht (2011) looked at anti-prejudice messaging and found something surprising: when people feel pressured or coerced into thinking a certain way, they dig their heels in. Instead of reducing bias, these messages can actually make it worse. Giving people more autonomy in how they engage with DEI efforts might be a smarter move. Fast forward a few years, to Moss-Racusin et al. (2016) who found that programs dangling carrots or sticks to enforce diversity goals often fail. People respond better when their motivation comes from within. The takeaway? If you want people to embrace DEI, make it about shared values, not external pressure.

Yes it’s Dobbin & Kalev (2016) again, showing why so many DEI programs flop. Surprise, surprise, forcing people to participate doesn’t help. In fact, mandatory sessions can spark resistance. Instead, they found that voluntary programs, mentorships, and leadership support yield better results by creating buy-in and trust. Leslie (2019) shines a light on unintended consequences. Leslie found that bias-focused training can backfire by reinforcing stereotypes and creating perceptions of unfairness. The lesson? A clumsy approach to diversity can end up dividing people rather than bringing them together.

More recently we have a meta-analysis is a big deal because it pulled together loads of research on DEI training by Moss-Racusin et al. (2016).  The verdict? There’s not enough evidence showing these programs work, especially the quick-fix ones like single seminars. Long-term, systemic change is what’s needed, not just ticking the “we did training” box. Iyer (2022) then dug into why people oppose DEI programs, especially those from so-called “advantaged” groups. The big issue? They often feel threatened or unfairly targeted. Ignoring their concerns only fuels resistance, so a more inclusive approach that engages everyone might be the way forward.

Burnett & Aguinis (2024) pointed out that DEI training aimed at specific groups can lead to discomfort and defensiveness, which defeats the purpose. They suggest reframing these programs to be less accusatory and more about shared goals to avoid making people feel unfairly singled out.

Bibiography

Kidder, D. L., Lankau, M. J., Chrobot‐Mason, D., Mollica, K. A., & Friedman, R. A. (2004). Backlash toward diversity initiatives: Examining the impact of diversity program justification, personal and group outcomes. International Journal of Conflict Management, 15(1), 77–102. https://doi.org/10.1108/eb022908

Kalev, A., Dobbin, F., & Kelly, E. (2006). Best practices or best guesses? Assessing the efficacy of corporate affirmative action and diversity policies. American Sociological Review, 71(4), 589–617. https://doi.org/10.1177/000312240607100404

Legault, L., Gutsell, J. N., & Inzlicht, M. (2011). Ironic effects of antiprejudice messages: How motivational interventions can reduce (but also increase) prejudice. Psychological Science, 22(12), 1472–1477. https://doi.org/10.1177/0956797611427918

Moss-Racusin, C. A., van der Toorn, J., Dovidio, J. F., Brescoll, V. L., Graham, M. J., & Handelsman, J. (2016). A “scientific diversity” intervention to reduce gender bias in a sample of life scientists. Cultural Diversity and Ethnic Minority Psychology, 22(3), 295–306. https://doi.org/10.1037/cdp0000070

Dobbin, F., & Kalev, A. (2016). Why diversity programs fail. Harvard Business Review, 94(7-8), 52–60. https://hbr.org/2016/07/why-diversity-programs-fail

Leslie, L. M. (2019). Diversity initiative effectiveness: A typological theory of unintended consequences. Academy of Management Review, 44(3), 538–563. https://doi.org/10.5465/amr.2017.0087

Paluck, E. L., Porat, R., Clark, C. S., & Green, D. P. (2021). Prejudice reduction: Progress and challenges. Annual Review of Psychology, 72, 533–560. https://doi.org/10.1146/annurev-psych-071620-030619

Iyer, A. (2022). Understanding advantaged groups’ opposition to diversity, equity, and inclusion (DEI) policies: The role of perceived threat. Social and Personality Psychology Compass, 16(4), e12666. https://doi.org/10.1111/spc3.12666

Burnett, L., & Aguinis, H. (2024). How to prevent and minimize DEI backfire. Organizational Dynamics, 53(2), 100981. https://doi.org/10.1016/j.orgdyn.2023.100981

Fanshawe, S. (2021). The power of difference: Where the complexities of diversity and inclusion meet practical solutions. Kogan Page Publishers.


Saturday, November 23, 2024

Acronyms, alliteration and absurdity: the sad truth about Organisational 'Values'

I had an epiphany some years back. In a brief conversation with a young woman, in the queue for lunch at a corporate ‘values’ day (I was a Director), opened my eyes up to the whole values thing in organisations. 

“I have my values,” she said, “and they’re not going to be changed by a HR department.... I’ll be leaving in a couple of years and no doubt their HR will have a different set of values… which I’ll also ignore”. Wisest thing I heard all day.

You’ve probably had the ‘values’ treatment. Suddenly, parachuted out of HR, comes a few abstract nouns, or worse, an acronym, stating that the organisation now has some really important ‘values’. Even worse, an expensive external agency may have juiced them up. I genuinely like organisations that have a strategy, purpose, even a mission. But the obsession with organisational values I just don't buy.

Not long afterwards my fears were confirmed. I chaired a Skills Summit, where innumerable HR folk pompously paraded their company values in a series of talks. An endless stream of abstract nouns, all of which seemed like things any normal human being would want in any context, in or out of work - you know the words - integrity, innovation, honesty, customer-focus, community....  After a full day of this stuff I was impressed by the guy who ran a small, very successful software company, who stood at the podium, and claimed that his company didn't really have any stated values and felt that the whole 'values' thing could be replaced by one phrase 'Don't be a dick!". That became a meme some years back and touched a nerve because it cut through the hubris.

Why HR-driven values are out of touch with reality?

Having dealt with hundreds of large organisations for more almost 40 years, I have yet to find one whose values were anything more than platitudes. They are invariably a crude mixture of reactive PR, HR overreach even a marketing ploy. Usually a crude selection from a list of abstract nouns, often forced into an idiotic acronym, they bear no relation to reality. Even when masked by complex consultancy reports and training - it's almost always bullshit Bingo.

Why would we imagine that HR have any skills in this area? In what sense are they 'experts' in values? For me, it is a utopian view of work and organisations. I can remember the day when organisational 'value' lists never existed. People were more honest and realistic about expectations. They came in when HR suddenly decided that they had to look after our emotional and moral welfare - always a ridiculous idea.

Values-washing

The banks were full of this 'values' culture – that was before the financial crash. I worked with many of them. It was all puff and PR. People do not, and don't, buy into this stuff. They can barely recall what the values are. I have values and I'm not interested in what HR, or some external consultant, says my values should be. The even more ridiculous idea that people who don't adopt those values should be forced out is wrong and illegal.

They shove them on the website but few remember them and even fewer care.... The really interesting thing about 'values' is that those companies who feel most compelled to get them identified - banks, accountancies, consultancies, tech companies, pharma companies etc - are the very companies where they were most ignored. They are blatant attempts at value-washing, appearing to be value driven when you are not.

Try these authenticity tests to your company values. Sniff out the hubris and bullshit.

Test 1: Bad acronyms - values created to fit word

If your values set is an acronym, they’re almost certainly inauthentic. The net result of fuzzy HR thinking is so often the ‘bad acronym’. Chances are that someone has shoehorned some abstract nouns into a word that sounds vaguely positive, completely losing sight of the original intention. Are they telling you that their values ‘just happened’ to fall into that acronym? Actually, what happens is that at least some of the values emerge from the acronym. That's bullshit.

How about this from a Cheshire voluntary group: FLUID - Freedom 2 Love Ur Identity. Or another real example of a crap acronym: VALUE - this HR person actually went online as she could only think of Value Added….. and wanted others to fill the acronym out! They did, and she was delighted with, Value Added Local, User friendly Experience. What a load of puff. 

When values are created to fit a word you are engaging in an infantile exercise that treats employees like children. Even worse is the use of middle letters, rendering the acronym, as an aide memoire, completely useless. Here’s a real example. It’s a cracker. PEOPLE: Positive Spirit and Fun, HonEsty and Integrity, Opportunities Based on Merit, Putting the Team first, Lasting value for Clients and People, Excellence through Professionalism. One overlong, impossible to remember acronym with eleven nouns, and I love the way they have to use the ‘E’ in the middle of HonEsty to make it work. This, by the way, is from an HR consultancy.

It’s not that I hate acronyms. They’re great as memorable cues. For example, I rather like ABC (Airways, Breathing, Circulation) in first aid. I also have a soft spot for funny acronyms, such as ALITALIA (Airplane Lands In Turin And Luggage In Ancona), BAAPS (British Association of Aesthetic Plastic Surgeon) unbelievably a real organisation, and DIMWIT (Don't Interrupt Me While I'm Talking).… it’s just that I’m a fully paid up member of the AAAA, the Association Against Acronym Abuse. And let's just quietly forget Microsoft's 'Critical Update Notification Tool'.

Test 2: Alliteration test

You hear alliterative value lists all the time  - 'Imagination, Integrity, Innovation' (two organisations I know have this one set, clearly having cribbed it from the internet, or lists of 'C' words such as creativity, curiosity and collaborative. These are far too conveniently alliterative for my liking. The world is not intrinsically alliterative and if your list of values all start with the same letter - it's forced nonsense. 

Test 3: Negative test

Lists of values are often so obvious that they are hardly worth mentioning. Sure, you can say we all need to be 'Customer friendly' and so on. But who would say that being Customer unfriendly was ever on the cards? The ‘negation’ test is a useful filter. Ask whether any normal human being would deny having the stated opposite or negative value. If the answer is invariably NO, as it’s not a value but a basic, common sense belief. Human nature is a complex thing and people are too different to be corralled into value sets. Beware of BIG words like integrity, imagination, creativity, innovation…… if your values are abstract platitudes – no one will care.

Test 4: Are they really values?

A value is something that determines a moral decision. Yet many organizational ‘values’ are not values at all. ‘Imagination’, for example, is not a moral value, neither I would argue is 'creativity'. I’m not sure that ‘Leadership’ is an intrinsic value, in the sense that Pol Pot was a leader. So, for this test, look at each value in turn and ask whether it really is a value or activity, competence or some other thing? 

Test 5: Diversity problem

There’s something odd about having diversity as a value within a non-diverse, fixed value set. Empirically, people have different sets of values. We know this from large-scale studies, such as the World Values' Survey, going since 1989, in over 100 countries. An organisation is likely to have a mix of nationalities and cultures; religious, secular, liberal, conservative, individualistic, communal. Imposing a single set of values from above may not fit with this diversity of cultures and values. If diversity of values matters, the imposition of a set of fixed values makes little sense. 

To practice diversity is to live with a diversity of values. At the Skill Summit, some companies seemed to imply that if you didn't fit in with their imposed values, they'd try to get you out. Really? When values become reasons to sack people, you've got to worry. Even the phrase 'Don't be a dick' worries me. Companies often have dicks in the workplace. So what? Lots of very competent and talented people are 'dicks'. Elon Musk is a dick. Steve Jobs was a dick. Gates was a dick. Get over it. We're all different.

Test 6: Sniff test

It is usually quite easy to expose the hypocrisy of corporate values, namely the

hypocrisy of an organisation that exhorts ‘values’ by looking at its a) tax affairs b) senior staff salaries, c) senior staff bonuses d) customer list e) behaviours. If the company plays the tax avoidance game using offshore tax arrangements, or transfer pricing – that’s almost every large tech company, Google, Apple, Amazon, Starbucks etc. etc. then add hypocrisy to their values. If the CEO earns a ridiculous amount of money but doesn’t pay a living wage to the people at the bottom, the value of their values is nil. To be more precise, if your company pays the CEO way more than x10 the salary of the lowest member of staff – question the values. If, as a bank or other organisation, you’ve mis-sold, ripped people off and generally fiddled the markets, ripped off suppliers, don’t pay on time - don’t even mention values. 

Read Nagel's Equality and Partiality. It doesn't take long to work out that stated public values are often different from personal values. The same with organisations. You get the idea. Subject your organisation to a sniff test. Take the values and really ask – of the people who have told you that they matter – whether they’re applied at the top of the organisation and in its financial dealings. 

Conclusion

In truth, everyone knows that values are actually marketing exercises, used by organisations as slogans. They have little to do with actual behaviour in organisations. They infantilise people, reduce them to ciphers. Ask the person in the street if large organisations have served society well in terms of values? Banks? Supermarket chains? Tax dodging tech companies? Tax dodging retailers? Football organisations like FIFA? Sports organisations? Political parties? Energy companies? No. 

We have a crisis of trust in institutions because people parrot values which they don’t then practice. The ‘values’ obsession is just another example of overreach by HR. It keeps them occupied and gives everyone the sense that moral purpose has been served. It may even mask the reality of controlling behaviour. When I hear people discuss values, or see ‘values’ training, I hear moralising. Lots of back-slapping and ‘aren’t we great’ type platitudes. We’re all different. It is the workplace not a moral crusade. 

Forget the buzzwords, the brutal truth about company values is that a select group at the top come up with 'values' and we all have to march in step to those values, even though, as most of us know, the further up an organisation you go, the more rarified values become.

Groucho Marx said "I have values and if you don't like them.... I have other values" and if asked whether I change my values if they are not the same as my employer, I have argued with many people for many years that employers and HR have no right to do this. The answer is NO. It's an excuse for one group to impose their personal views on another and is causing untold damage in organisations.

People have values, organisations don’t. 

Sunday, November 17, 2024

20 ways to use NEW Google Video analysis

Google’s Gemini-Exp-1114 has launched and is getting rave reviews. It beats OpenAIs GPT-4o model hands down.

VIDEO

The BIG new feature is the analysis of video. You can upload video for analysis from your Google drive and off it goes.

Examples include analysis of:

STUDENT PERFORMANCE

Presentation performance

Eye tracking in teaching surgery etc

Coaching on sports performance

Feedback on musical performance

Teacher/lecturer/trainer performance

Compare present with past performance

STUDENT LEARNING

Lecture summarisation

Image, graph, diagram analysis into notes

Generate practical tutorials from tasks/procedures

Vocational tutorials

Pick out key clips

Create branched scenarios from video

TASK ANALYSIS

Behavioural tasks analysis from real world actions

Workflow efficiency - redundant steps in task execution

Performance analysis of meetings

Highlighting health & safety in real world environments

ASSESSMENTS

Assessments created from a lecture

Create video-based assessments

Assessment of student presentation of assignments

Identify specific errors made when doing something

You get the idea.

BREAKTHROUGH

It may lift us out of the current world of teaching and learning, incredibly limited by its focus on ‘text’. We are drowning in a sea of text, yet we know it does little in teaching an assessing actual skills and performance.

MODEL PERFORMANCE

Gemini-Exp-1114 has knocked OpenAI off the top of the Chatbot Arena leaderboard, the de facto goto place for benchmarking. It scored particularly well in Maths, Creative Writing, Longer Query, Instruction Following, Multi-turn and Hard Prompts. This will also help with the video analysis.

LONGER QUERY and Instruction following is a query, prompt or instruction that includes a detailed and extended description of what the user wants the AI to do or generate, allowing you to get the model to follows specific steps, formats, or guidelines in its response.

MULTI-TURN is dialogue with the AI that spans multiple exchanges (turns), where each response builds on the previous ones, remembering prior questions and answers to provide more coherent responses.

HARD PROMPTS are prompts that are intentionally complex, ambiguous, or challenging for the AI to interpret or execute correctly. It is here you expose the model's limitations, so useful in evaluating its boundaries.

CONCLUSION

The competitive race is producing faster better models but more importantly, a new layer of multimodal functionality is making AI far more usable. All of the above show that the new models are improving practical usefulness. AI is moving away from just a model release to something that gives better practical results, especially in education and learning.

Thursday, November 14, 2024

Is AI generated video as GOOD as real video in learning?

There are many surprising things we can learn from research into video and learning. I have given many talks on the subject showing research on video and memory (the transience effect), does learning at x1.5 or x2 affect learning? Do segmentation, length, perspective, picture quality, audio and so on affect learning? Here are 15 THINGS that may shock you from the research… some will surprise you!

But is AI generated video as good as real video in learning?

Leiker et al (2023) in Generative AI for learning looked at this hypothesis.

The study took 83 adult learnersn randomly assigning them into 2 groups:
1.    Traditionally produced instructor video
2.    Video with realistic AI generated character
Pre and post learning assessment and survey data were used to determine what was learnt and how learners perceived the two types of video.

NO SIGNIFICANT DIFFERENCES
No significant differences were found in either learning or how the videos were perceived. They suggest that AI-generated synthetic, talking head learning videos (limited) are a viable substitute for videos.

This doesn't surprise me. I’ve been creating avatars of myself at increasing levels of fidelity in appearance, movement, lip-synch & voice, speaking many languages from Chinese to Zulu. This involved going into a studio for video capture and separate audio studio for voice capture. A range of services are available from Synthesia to Heygen. These avatars can be used as employees in management training, patients in healthcare training and customers in retail training.

SIGNIFICANT DIFFERENCES
Any form of human interaction can use this technique for training; in instructional videos, trigger videos, branched scenario videos and videos with additional AI generated learning experiences and assessment. In fact, the use of AI can lead to significant UPLIFTS in learning outcomes. In one trial with a client, before GenAI appeared, in 2020, AI enhanced learning resulted in a 61% increase in assessed learning.

INTERACTIVE CHARACTERS
We now have avatars that one can converse with using AI chatbot technology taking it to another level through scenarios and simulations, using real dialogue. We can expect tons of these to appear in computer games (OpenAI have dealings with GTA). But it is in training that they have huge potential. It has been impossible to create high fidelity simulations for soft skills in the past. I created a lot using fixed video clips in interviewing skills, conflict, language training and so on. They took a lot of time to design write and produce. These are about to get a lot quicker and cheaper.

CONCLUSION
The use of AI generated video is already here and will continue to evolve. We are not yet at the level of full drama but the direction of travel is clear.