OpenAI drummed up a great PR campaign with leaks around the movie ‘Her’ and Altman hints. What many missed was another launch by Google. That’s how fast this stuff moves.
More than this they published a paper we in the learning game should all read. Towards Responsible Development of Generative AI for Education: An Evaluation-Driven Approach. It has 75 authors!
What the paper does is address a BIG issue – getting learning insights into GenAI.
Excitement is brewing over the potential of GenAI to revolutionise learning by providing a personal tutor for every student and a teaching assistant for every teacher. The OpenAI launch was fantastic and showed real opportunities and promise. They out Appled Apple with their focus on user experience and functionality.
However, this vision is still not reality, mainly due to the complexity of translating learning insights into effective AI prompts and the lack of robust ways to measure AI's teaching effectiveness. SO Google teamed up with learners and educators to turn broad educational theories into practical evaluation benchmarks. This is a mix of quantitative and qualitative measures, both automated and human-led, and they have developed new training datasets to enhance the educational skills of their GenAI system, LearnLM-Tutor.
They claim to be achieving great results from LearnLM-Tutor over other models because of its superior teaching abilities. They would say that, wouldn’t they. But I’m impressed, as this work lays the groundwork for a comprehensive framework to assess educational AI, potentially accelerating GenAI's positive impact on learning.
What this report does is look at the use of GenAI to enhance education through AI-driven tutoring, particularly focusing on conversational models. Their approach utilises Supervised Fine-Tuning (SFT) with data informed by educational principles, improving the AI tutor, LearnLM-Tutor, beyond the Gemini 1.0 model.
They are honest about the challenges, especially in defining and achieving true pedagogical mastery with AI. Collecting a diverse range of high-quality pedagogical data is costly and labour-intensive, and it is unclear how many examples are necessary to comprehensively cover pedagogical behaviours.
So they have established a set of benchmarks to evaluate their progress, although these too have their limitations—especially the expensive nature of human evaluations. To address these issues, they have put together a multidisciplinary team, including AI scientists, engineers, pedagogical experts, and cognitive scientists, to work on refining these evaluation methods. This collaborative effort aims to not only enhance the current AI models but also invites the broader AI and learning science communities to join in enhancing and utilising these pedagogical benchmarks. Their ultimate goal is to leverage AI effectively to benefit learners, pushing forward the boundaries of learning technology. More power tyo their elbow.
Google is Google. They don’t do product launches like OpenAI, they integrate things into their world. They have a global view of technology, which means integrated. Don’t write them off.
No comments:
Post a Comment