Multimodal means many more moments for learning

Sora from OpenAI and Videopoet from Google shows that the LLM revolution will be televised. Multimodal models mean quick and cheap input, amending and output in any medium. The consequences for teaching and learning are huge. 

1. Free from tyranny of text

We can free ourselves from the tyranny of ‘text’. Far too much learning is text based. Schooling and Universities are almost obsessed by reading and wring text, thereby excluding most useful skills in life. E-learning with its blocks of text and graphics seems like a pale imitation of life. Education, in particular, is fundamentally text based, limiting teaching and learning opportunities.

2. Optimise media for learning

Each medium and its combinations have affordances in learning, in terms of retention and transfer.  Each has a set of DOs and DON’Ts. I wrote about this. in detail, in my book Learning Experience Design. These are backed up with research which shows how to use the right tools for the right job. The important point is to use the optimal medium for the learner and learning task at hand. We now have those options... and they are quick and cheap.

3. Quick resources for teachers

For teachers, the fast production of resources will be a boon to their craft. They are now literally able to create almost anything in any medium in seconds. An image from history, for geography, famous person from the past saying something. Teachers can breathe life into almost anything they teach in any medium. This freedom allows teachers to be both creative and more effective in the classroom and online.

4. Learners can do it for themselves

Perhaps the biggest lesson is the pendulum swing from teaching to learning. That is made easier. The user now has these tools at their fingertips. Generative learning, explored by Wittrock and others is one of the most powerful forms of learning. It can now be used as a generative tool by the learner. If you want to summarise a book, an image as a mnemonic to remember something, flashcards with images, a self-quiz, a checklist, job aid… do it yourself.

5. Context and cultural adaptation

If you want resources that are more appropriate for your context and culture, simply ask for it. This can be as simply as text in the appropriate language up to translating text into your first language. It can be images that show relevant, even local, elements or video relevant to your place and culture. All of this makes the learning more relevant to the learner.

6. Accessibility

This has already had profound beneficial effects in accessibility for those with sight impairments, with text, image and video to speech. Similarly for hearing impairments, with speech to text, images and video. Dyslexia, autism and ADHD, and other learning difficulties will also be aided by multimodal capabilities in the hand of both teachers and learners. Much more to be done on this front.

7. Media mix

Blended learning is so often just blended teaching, a bit of offline with online. At last we have the tools to move towards really blending teaching and learning experiences, using the right media mix for the right teaching and learning tasks. But solid, measured pedagogic rules should be applied. These are my top three:

Less is usually more.

Learning is a process not an event.

Doing really does matter.

Get to know

Get to know this list and consider all of these in your future use of media in teaching and learning. It is quite extraordinary how much is now possible through generative AI.

Text generation
Text-to-video
Text-to-image
Text to audio
Text editing
Text translation
Image generation
image editing
Image-to-video
Image to text
Image to audio
Image editing
Audio generation
Audio to text
Audio to image
Audio to video
Audio editing
Audio stylisation
Video generation 
Video-to-audio
Video frame continuation
Video inpainting
Video outpainting
Video stylisation
Data critiqued
Data to text
Data to image
Data to audio
Data to video

Finally - a warning

One must also be careful in over-producing and over-production. Just because one can do it, doesn’t mean you should do it! Combinations of media must also be considered. Again, research is clear on issues such as over-writing with text, using the wrong style of language, use of text in images, the danger of the transience effect in video, usefulness of audio only in podcasts. There are literally hundreds of things one needs to know to make media effective in learning, use them carefully.

Final thought

Above all this gives opportunities to teachers and learners in places where resources and money is tight. It gives power to the hands of teachers and learners in poorer countries.