AI will not take your job but someone using AI will – it may well replace Doctors?

This paper (Influence of a Large Language Model on Diagnostic Reasoning: A Randomized Clinical Vignette Study by Goh et al.) on ‘diagnostic reasoning’ hasn’t had enough attention. The authors fully expected Doctors plus GenAI to win. But GPT 4 on its own beat Doctors hands down.

One of the authors made the point that the surprise was that the results broke that oft quoted trope that “AI will not take your job but someone using AI will”.

GenAI, for some time, has been beating medical students hands down on clinical exams. But can it outperform real Doctors?

They used a randomised design, with 50 physicians from various medical institutions. Three approaches were compared. The Doctors were randomised into two groups, then compared to GPT4 used on its own:

1. Docs + conventional resources
2. Docs + GPT-4 & conventional resources
3. GPT-4 alone

Each had 60 mins to complete up to 6 clinical problems. Their diagnostic reasoning was measured on differential diagnosis accuracy, supporting/opposing factors and their next diagnostic steps.

SHOCK RESULTS

When used WITHOUT human input, GPT4 scored 15.5% higher than the conventional resources group, outperforming both physicians and hybrid methods.

1. Docs + conventional resources only (73.7%)
2. Docs + GPT4 & conventional resource (76.3%)
3. GPT4 alone (89.2%)

Doctors using GPT4 alongside conventional resources showed only a marginal improvement in diagnostic accuracy over the conventional resources group. The GPT4 group also took less time per case.

It showed that GPT4 not only excels at real-world diagnostic reasoning, it also measured diagnostic reasoning through structured reflection, giving richer insights than simple accuracy. Remember those complaints about transparency and AI. Well, here we have it.

Sure, a limited sample at 50, also puzzling that GPT4 is better on its own that when used as an aid by the Doctors. Suggest that the Doctors are the confounding factor here? Turns out they were often doing is using GPT4 as a search engine.

GenAI is here to stay in medicine and may surpass that of trained Doctors.

CONCLUSION

Diagnosis rates among General Physicians stand at around 5%, sounds worse when you say 1 in 20. Let’s suppose AI, on its own, a UNIVERSAL DOCTOR has a misdiagnosis rate of only less than 1%. I'm sure this will happen, now that reasoning has arrived. At this point you’d be a damn fool to go to your Doctor.

Straight from the imagination to the screen.... Meta's video release

This is the promise of AI. And we’re getting tHere faster than anyone could have imagined. You become a video director, without the eye watering production costs. GenAI has moved way beyond text to the image, video, audio space.

Meta’s new ‘cast’ of models allow you to create but also edit and personalise images, videos and audio. Personalised means based on your or anyone’s face.

This is another astounding milestone... models that generate:

·     1080p HD

·     Up to 16 secs (16FPS)

·     Different aspect ratios

·     Synchronized audio 48 kHz

·     Instruction-based video editing

·     Personalised videos based on a user’s image

Note that 16 seconds doesn’t sound long but call them takes and they’re very long but TikTok built a global business on videos with a 15 second limit (now longer).

All from text prompts.

https://ai.meta.com/research/movie-gen/

LEARNING

Any teacher can create videos of them showing students how to do anything. That’s surely useful, especially in vocational learning but also in science, art, music and many other subjects. Call this personalised teaching.

 

It gives anyone the ability to create short (here’s 10 for startes):

Instructional step-by-step procedures

Explainer videos

Trigger videos

Branched scenario-based

Simulation videos

Animations

Video flashcards

Scenes too risky to film

Scenes impossible to film

Short messages from CEO etc

Ads for new initiatives

Long-form video suffers from the transience effect, but there are ways to make it more relevant and effective in learning, beyond the lecture and talking head.

I’ve been involved in many scenario-based video sims and this has just made them attainable on a low budget. Your video production costs on management training sims, such as interviewing, having difficult conversations and so on has just plumetted.

In performance support, huge numbers of short videos can be created ready for delivery at any moment of need in the workflow.

CONCLUSION

I can see this move towards longer takes, even drama. To be honest that may already be here. You’ll see a lot of creative fun stuff, like personalised birthday messages. The personalisations is interesting. Thought also has to go into impersonation and the avoidence of explicit material.

Realistic dialogue is here....

Advanced Voice Mode on the OpenAI app is now avaialble in the UK. I’m getting pretty used to advanced releases and was hugely impressed by ChatGPT o1, with its open thinking and reasoning capabilities. Then got pretty excited by NotebookLM with its automatic podcast feature and other functionality. But this was a user experience like no other.

Fluid

Once again, I feel as though I’m being swept up and away, as AI trailblazes forward at a blistering speed. Once again I’d emphasise that you really do have to ‘try it to get it’. An interesting paper just published showed that much of the scepticism on GenAI comes from those who have not tried it. Lesson - for any real critical thinking or analysis, some use is necessary.

Immediate impression – conversations are just so much more fluid, natural and therefore real. This is Alexa and Siri on steroids. Much better on interruptions, which matters, as as voice dialogue can be a little odd when there’s clumsy turn taking. Here, you feel you are in control and not getting dragged along by the ‘machine’. It feels more like talking to a human. Nass & Reeves were right, the more you eliminate non-human cues, such as odd pauses, clumsy interruptions and misfires, the more you are fooled into thinking the machine is a person. I keep saying this but interfaces matter in tech.

It copes superbly well with pauses or interruptions and pick up where you left off with ease. It also recognises when you are finished speaking. No need to tap button when you stop – that was badly needed. Also better on filtering out background noise.

Accents 

It also picks up on conversational cues better – recognises intonation, speed of speech, even if it is a question. A variety of voices are available – different genders, accents, British American or Scottish, different speaking styles are also available. Choosing a voice just makes things feel better. I can express myself more freely as it has no problem with my strong Scottish accent. I tossed an entire caber at it on this one. Yet it coped with my odd vernacular and very fast speech. Believe me you wouldn’t have understood what I said  - it did. Asking it to speak back to me in a Scottish accent was fun.

Memory

As it remembers coversations, subsequent conversations are more meaningful over time. It keeps track of preferences, interests or details about you to make the chats more relevant. It then took me by surprise. More of that later. Memory matters and if I had a disability or specific preferences it will remember and tailor the conversion to my needs. This is a boon for accessibility and inclusion.

Accessibility

On that point, I’ve been doing some work in DEI and accessibility in AI, where people have truly underestimated, not only hoiw much has already been achieved but also its potetial to give agency to people, who find education and the workplace difficult. This is of immediate use for the blind, dyslexics and to be honest – anyone. We all have learning difficulties as learning is difficult. This gives access and agancy to so many who find education and learning difficult.

Learning

Tried it for learning a second language – astonishing on pronunciation, like talking to a good teacher. We ar egtting ever closer to this being a tutor, trainer, coach, mentor, counsellor and so on. I think we can see how any training that requires role playing – managerial training, sales training, customer care, dealing with patients and so on. You can prompt it to play roles – that’s very cool. When the dialogue and pedagogic paths are tighter it will be a UNIVERSAL teacher, on demand, anytime, anyplace on anything, at any level, for anyone in any language. 

What’s coming next?

My guess is that future enhancements are likely to be even better at understanding and emotion detection, as emotional cues allow more nuanced reactions. More customisation is inevitable. But what we’ll also see is is more contextual awareness, to improve relevance and make converstions more intuitive. Integratoipn with other applications and devices will come in time.Eventually we will be speaking fluently to avatars, images of people htat look like real people. This is inevitable. The integration of this realistic dialogue into robots also seems certain.

Conclusion

Honestly. This was amazing. Maybe the first time I’ve ever felt that I was speaking to another human but actually a machine. This is going places – whether it is help with learning, getting things done in the workplace or getting some help if you’re feeling down. 

One moment stood out. She dropped the fact that I had a dog into the converstion, not just that, she knew it was a Schnauzer. I love my dog. He was snuggling up to me at that very moment on the sofa. I stoked his back, he looked over his shoulder at me. All three of us had a moment.

 

Is SaaS in learning dead?

Our LMS seem antiquated, a bare learning landscape of frustration, overloaded with outdated content a stockpile of cartoonish modules and tedious slideshows dotted with MCQs.

The market seems weary of simply renewing licences for software that has low use figures, often just slabbing out compliance content with no actual measurement of its productivity or impact. Measuring bums, even on virtual seats is to measure the wrong end of the learner. 

These systems get locked in as a matter of habit not impact. But dreams don’t last forever and pricing on LMS software, content libraries and other learning services has got fiercely competitive. Many are seeing revenues plateau or fall. Management is being swapped out, share-prices falling, revenues no longer predictable, profits still showing overhang on 3 year licence fees. It has all become fiercely competitive.

AI is disruptor

Having been involved in a lot of AI projects for over a decade, since early 2014, the assumption has always been that people need to build some sort of AI SaaS product, sit back and watch the money pour in. A recent Tweet questioning this, by the much admired John Rush, went viral on this point.

Yet AI is already bringing services directly to your users, be it learners or employees in any sector at any level. With ChatGPTo1 reason has now appeared. AI now gives massive agancy to the individual separate from your corporate systems. Like a supercar at saloon prices it has overtaken many services on the fast lane. SaaS services get locked into fixed depreciating code. These inflexible code sets and fixed products make progress impossible, whereas AI seems to release new astounding updates on a weekly basis, so quick, it is hard to keep up.

AI is not really a business change, it is a change in mindset. You have to think of of power being plugged directoy into actual learners and employees. It releases agancy and motivation. Once you realise that everyone is using it, you need to see that its direct impact bypassing your existing systems. 

AI on the SLY

The studies on productivity and impact are clear, that’s why most are using AI on the SLY. Individuals are increasing their productivity, making their lives easier, while organisations, locked into legacy systems are trying to ignore it or fighting it wit negativity. That’s because it needs a shift in mindset, to accept the probability game, understanding that everything has flaws, everyone is biased, especially people and processes. Sure AI may occasionally come up with the wrong answer but so do people and at least AI is fast! In fact, for most education, learning and business uses, hallucination is not a problem at all, as it has been pushed away out to the periphery. The appearance of reasoned ‘thinking’ has meant a whole load of new educational and organisational uses have been folded in to the omelette. And you don’t make an omelette without breaking eggs.

Productivity

Traditional Saas product in learning may be struggling but it also means that straight to user services in real time are alive and kicking ass.

One group of companies that are doing well are those that use AI to simply increase internal productivity. This gives them edge and the financial room to increase productivity or respond to pricing pressure. In one case we’ve seen astounding increases in productivity on process, both massive savings in time but also increases in quality. This gives you time to focus on customers and quality. How do you make your learning company more valuable? Get lean. Get agile. AI allows you build software fast. AI allows you to fast-track productivity increases. AI gives you quality increases fast. AI unleashes life in an organisation.

Performance support

Performance support, which is what 99% of AI is, is now in the hands of users. There’s tons of rooms for good delivery here, as organisations need to deliver performance support based on their own motherlode of content. In steps AI with RAG and other techniques to do precisely that - superbly well. Suddenly, you’re not relying on generic content, that everyone’s using, giving your organisation no edge at all. You can focus and lever what you already own. 

Conclusion

We’re still locked into the lumbering LMS, often a dark dungeon of disappointment, also over-engineered content, often out of date and hopelessly cliched and generic, with its cartoon libraries and slideshows peppered with MCQs. There is a sense that the old era is dying in the vine, coming to an end.

 

Good survey (1045 parents & teens 13-18) showing high use of GenAI in schools (with short podcast)

Yet another good survey (1045 parents and teens 13-18) showing high use of GenAI in US schools. Now the predictable story of learners using AI on the SLY.

Use

First, we have to recognise that they’re using it and that their use is across a number of tools for different purposes. But homework and assignments are the most common.

• 7/10 teens, 13 to 18 used at least one type of AI
• Using AI for a variety of purposes
• Help with homework most common 
• 40% report using AI for school assignments 
• 63% of AI users used chatbots or text generators
• 57% used search engines with AI generated results

Permission

It would appear that about half have had no guidance, or don’t know of such guidance from schools. This is worrying and shows how institutions lagging in their professional duties, to create the right context for learning and behaviour. Parents, of course, are usually kept in the dark

• Even split on using AI with/without teacher’s permission 
• 6/10 teens say school has no rules/don’t know for AI use

Parents

Most parents are in dark about their child’s generative AI use. They say schools have not communicated about AI policies and only 37% of parents whose teen had used GenAI platform thought their child had used it.

Accuracy

With around half using other sources to check accuracy, this is to be expected from teens. However, as hallucination errors rapidly decline, and capabilities in maths and science increase, this is a diminishing problem. This is less of a problem with high school content, compared to say Universities.

• 49% checked other sources to verify accuracy of AI  for assignments
• 39% using AI detected problems and inaccuracies

Discussions

Teens who’ve had class discussions about AI more likely to have nuanced views about its usefulness and challenges. More often they say it has changed the skills, educational path, or job they plan to pursue. That’s fascinating. The Harvard review at College levels said exactly the same thing, that these experiences are shaping hteir view of their own futures, even resulting in rethinks on study paths and careers.

Conclusion

Use is normalised, with a range of tools for a range of purposes but still mostly for homework and assignments. Schools tardy in having policies and not much use within schools. Parents in the dark about all of this.

Here's an AI generated podcast about the report:

There’s lots of other interesting detail in the report. Remember this is a US study.
https://www.commonsensemedia.org/research/teen-and-young-adult-perspectives-on-generative-ai-patterns-of-use-excitements-and-concerns

Another great teaching and learning tool. NotebookLM's automated podcasts will blow your mind

NotebookLM shows how AI can be used right now in learning. The automatic podcasts will blow your mind.

This is a monumental achievement, as anyone can now take a motherlode of documents and get AI to extract summaries, answer questions, give a timeline, even create a podcast.

We have worked on note taking software adding AI features, and seen it work in practice. This really is an area that is ripe for application, as note taking is an established practice, the bridge between the teacher and learner, the active learning that allows the learner to move forward.

With NotebookLM, I uploaded a large book on learning theorists, at 300,000 words, saved as a 700+ page PDF. It covers 2500 years, with 300 learning theorists and has taken me years to write. Here's the list of Learning theorists and topics covered:

Chat

It gives a pretty good summary, briefing and timeline, century by century, across 2500 years. 

Then, I started to ask it questions, using the chjat function, engaging in dialogue with my own work. It is superb. If every teacher, department, course had its curated content in this context, this would be a boon for students. If you want to know some specifics about a particular learning theorists or more geneal view of a group, it does a credible job.

Learners can ask general questions, then, when engaging in a class or assignment, specific questions, getting carefully considered answers. I asked it some general questions about historical trends, why so few women in learning and so on. Pretty good…

What is impressive are the provenance numbers that link to the source text in hte document for most components in the answer.

Podcast

But the big hit comes at the end. I asked it to do an ‘Audio overview’ of all 300 learning theorists in 700 pages. This was way beyond my expectations. A dialogue podcast, at around 11.5 minutes, was amazingly realistic, with a male and female commentator bounding off each other. The language was informal and sounded like real dialogue. I have since asked people to listen – they ALL thought these were real people.

It covered the material well, pulling out relevant theorists and making insightful observations as they took us through 2500 years of learning theory. 

For novice learners who want a friendly introduction, those with poor literacy skills, English as a second language, but also dyslexic learners, this is a wondrous feature. I can see this being a great introduction to any topic, subject, course or assignment. 

Fronting a learning journey with an introductory podcasts seems like a fine idea to me. A brief and accessible introduction to a subject, super quick to produce and prepares the learners for deeper learning.

I can see this getting real traction with learners at school and college.

Take notes

You can also create and manage notes. You select the box in the upper-right corner of one or more notes, and NotebookLM displays text-buttons with actions to take on the notes. These include:

Summarise the selected notes.

Suggest related ideas.

Create a study guide.

Create an outline.

Combine the selected notes into a single note.

I am always astonished at how many learning professionals do not take notes at conference talks I give. It's almost endemic. Yet, notes, especially in school and college are the bedrock for subsequent learning. It is not the notes in themselves but what you do with the notes afterwards that matter. That's why the debate over handwriting v typing is irrelevant - the learning takes place largely in what follows the act of writing - the retrieval process, reflection, manipulation, subsequent analysis and expansion of those notes. Notes launch good learning journeys. This encourages a more structured approach to notes, with more tools and help.

Sharing notebooks

You can share notebooks by opening a notebook and clicking the Share icon in the top-right corner of the screen. You can grant either Viewer or Editor access to another user by adding their email address.

A viewer will have read-only access to all the source documents and notes you shared with them in the shared notebook. An editor will be able to view, add, or remove sources and notes in your shared notebook as well as share it further with other users. Personal gmail accounts can share a notebook with up to 50 other individual users

PS

If you want REAL podcasts and deep dives into specific topics in this this content, there are 36 1 hour podcasts in the 'Great Minds on Learning' series here…
https://greatmindsonlearning.libsyn.com/

Further experiment

Experimenting further, here's the NotebookLM AI-generate summary and 10 min podcast from four large articles I wrote on Heidegger, Foucault, Lyotard and Derrida. Just one small section from the huge 700 pager.

I loved how it created jokes, even analogies. Absolutely brilliant. Who said AI couldn't be innovative!

And here's the AI generated summary

Summary

This text provides an overview of the philosophical and educational theories of four prominent figures in the 20th century: Heidegger, Foucault, Lyotard, and Derrida. While exploring their individual contributions to areas such as ontology, power relations, and language, the text also criticizes their theories, particularly their tendencies towards relativism, cynicism, and obscurity, arguing that these thinkers often detached their ideas from the real world. The text highlights the influence these figures have had on critical pedagogy, postmodernism, and contemporary educational practices.

Now here's the real long-form podcast by myself and John Helmer...
https://greatmindsonlearning.libsyn.com/gmols6e36-continental-theorists-with-donald-clark