Future of learning – mind boggling

Jaron Lanier, the VR expert, vividly describes his work in You Are Not A Gadget, where the brain starts to believe in the virtual world instead of the physical one. Lanier takes this one stage further describing the experience of extending your body through morphing. In other words, you can become a lobster with extra limbs or have an arm that is ten feet long. This is because of the homuncular plasticity flexibility of the body image.

We experience virtual worlds every day when we get caught up in a TV programme, movie, computer game or theatre. This is involuntary. In other words our perceptual systems and brains are hard-wired to believe that the simulated is real. Even at very low levels of fidelity and interaction, for example simple e-learning, Nass and Reeves from Stanford University, showed through a series of ingenious experiments, that we regard the computer and the people presented in such programmes, as real.

I’m labouring this a little, because there are still plenty of people out there who seem to regard our minds as not having this ability (this includes many educators and trainers). Time and time again you see statements to the effect that cognitive and behavioural change can only take place through real world face-to-face interactions with real people. The most obvious objection to this is the success of flight simulators, where complex cognitive and behavioural change is effected in a way that is far superior to any classroom experience, and there are many others.

Patrick Dunn, a respected learning expert, had an epiphany recently. In his blog, he described experiencing the power of the mind in learning through a mind control demonstration. He told us how Ian Glassock had demonstrated of a game at a recent Learning Games Conference that allowed you to control a fish and get it to descend to pick up a coin. Trivial you may think, but the potential is huge. In terms of training the mind (and in the end that’s all education and training is) this technology has the potential to allow us to control our emotions in simulated scenarios, cure phobias, tackle post-traumatic stress disorder.

The important point here is that we have to literally ‘change our minds’ and practice ‘actual belief’ in simulated situations. Anything is possible in this most flexible of all possible worlds. In fact, it is more flexible than any real world situation, as any unusual and dangerous situation can be simulated. Clive Shepherd and I used to sell this stuff way back in the 80s where we had a PC stress game that measured skin resistance through a headband, but things have certainly moved on.

The games world is now producing a slew of technical inventions that we should take note of, for this is a glimpse into the future of learning. This stuff is huge in the games world. Nintendo have sold more Wiis than the Playstation and Xbox put together, all on the back of a new interface controller – the Wii Chuck. However, both Sony (Playstation) and Microsoft (Xbox) are fighting back with astonishing new devices. They fall into three main categories:

1. Handheld controller

2. Look no handhelds

3. Mind blowers

Handheld controllers

Wii Motion

Nintendo’s Wii Motion Plus is an enhanced Wii controller that fits onto the existing controller and, through a gyroscope and rate sensor, measures motion much more accurately for sports games and other physical acts. This type of sensor really comes into its own in sports simulations such as tennis and golf, where tiny variations in wrist movement result in hooks, slices and mishits. The Wii with its health sensors and Wii Fit board has already moved firmly in the direction of home health and learning.

Jonathon Flynn at the University of Huddersfield has been doing great work in physiotherapy with the Wii Fit. From sprained ankles to broken bones, osteoporosis and strokes, it really can help with rehabilitation, by retraining the mind. The advantages are clear; you can do it at home, it’s cheap, it’s fun increasing compliance. As usual, the resistance he’s had has not been from patients but senior managers in his institution!

Johnny Lee has used the Wii remote to create 3D VR. He uses the WII remote backwards to head track. This video is truly fantastic. The effect is stunning.

Playstation Move

Sony’s has its own Playstation Move, using the PlaystationEye technology and a controller with motion sensing. The sphere on top lights up and allows the sensors to accurately track position and distance. Internally motion sensors track acceleration and motion. A magnetometer also measures movement relative to the earth’s magnetic field. The advantage of the physical controller is that haptic dimensions are still present, including feedback. The orb itself will provide light feedback, such as muzzle flashes. This is pretty smart stuff.

The difference between the Wii and Playstation is largely in terms of the fidelity of the graphics, but we can expect controller and console advances to reach a level where the application will hold ‘ideal’ model as and you as a learners will be taught to mimic those models e.g. Federer’s forehand, Beckham’s free kicks and Tiger Woods missionary position. In the non-sporting world, any physical task, and there are thousands of them from manual handling to precision surgery, can be taught using these devices.

Look no hands

Natal it’s cracked up to be?

Afraid it is! Natal (named after a city in Brazil), launched at E3, is one of many technical initiatives that are literally game-changing. Imagine interacting with NO controller, keyboard or mouse. It knows how many people are standing there, at what distance and so on. The compute will know your every movement and allow you to drive, exercise, or whatever. No more searching for the lost TV remote; just a relevant hand gesture to sweep to the next channel, stop the action or increase the volume. Facial gestures are recognised for the reading of wishes, reactions and emotions. It also has voice recognition that can distinguish between voices and the voice of the application. None of this is science fiction – this is exactly what the future XBox Natal interface promises.

When Stephen Spielberg says, "This is a pivotal moment that will carry with it a wave of change, the ripples of which will reach far beyond video games" he’s more prophetic than even he realises. This may herald a new era, not only in games but in simulations. Peter Molyneux, genius games designer, rightly says that this means a slew of new genres but if we take this a little further we can also speculate that it leads to technical advances in learning. The very idea that one can just step into a world and experience meaningful interaction that allows you to experience and learn is literally mind blowing. Human interaction where all of your movements, gestures, facial expressions and language are understood is now starting to be realised.

Mind blowing

But there’s more, and this is literally mind blowing. We are now seeing signs that mind control is entering games and simulations.

Mind Flex

Matel launched a game in January 2008 called Mind Flex (with slogan Think it. Believe it. Move it). This is literally mind over matter, using power of your mind through three sensors (one on your head two on your earlobes) Your mind controls the speed of a little fan which controls the height of a ball (like blow football). You turn a set of obstacles on the course. One could call it mind-eye co-ordination.

It uses Neurosky’s ‘ThinkGear’ EEG technology (Electroencephalography) to read theta brain waves with a wireless headset. In effect, it’s reading the ‘complexity’ of the signal. EEG is busy and noisy when you focus but less noisy when you relax, so it’s not the level of data that is produced but its randomness that matters.

I also like this spin on the famous Milgram electric shock psychological experiment. These guys hacked Mind Flex to give you an electric shock if you concentrated too hard. What a wonderful way to raise learners’ attention!

Force Trainer

The Star Wars Force Trainer is a £45 toy available on Amazon now that allows you to train yourself in the Jedi ways, well, to raise a ball in a tube. It has some great sound effects and works

Other spins on EEG technology include the Neural Impulse Actuator from computer supplier OCZ, that allows your brain to control the pointer on your PC screen and Neurosky have their own Mindset pack.

Future promise

It’s too early to even grasp the possibilities of mind technology but it’s not too early to speculate that this will have significant impact at some time in the future. It makes 3D cinema look decidedly dull. I, personally, think that the next generation of games controllers has much more to offer learning than games.

Role playing for real

This is role playing for real, where you really are being someone in an environment, not just manipulating an avatar like a puppet on a couple of awkward strings. You’ll be able to walk the walk and talk the talk.

Softskills

Model behaviours can be taught by giving you corrective feedback on variants from the model e.g. presentation skills, coaching skills, interviewing skills and so on.

Attention Deficit Disorder

One can easily imagine how children, or adults, with attention difficulties, could be trained to focus more on learning, or at least recognise the difference between their attentive and non-attentive mental states.

Physiotherapy

Recovery from physical injury and illness can include everything from the rehabilitation of amputees to basic physiotherapy. The new interfaces can provide amazingly accurate feedback on position and movement.

Sports coaching

The technology is now reaching levels of fidelity that bring them close to training people in the optimal use of their body, club, racquet or bat. No surprise then that the first crop of games using these newer interfaces are in sports simulations.

Tools and instruments

The physical use of tools in trades and engineering can be measured along with most sophisticated use of scalpels and surgical instruments, where keyhole surgery had demanded higher and higher levels of dexterity.

Conclusion

These different technologies, separately or together, offer mind boggling possibilities in learning. They will become smaller, faster (less latency) and cheaper over the next few years. In practice a ‘Bird in hand, may be better than bird in bush?’ Retaining the physical controller may not be such a bad idea, as many games and simulations have the manipulation of an object as their goal. If you want to perfect that golf swing, then the enhanced Wii controller or Playstation Move may be better in terms of accuracy and feel. In terms of learning the manipulation of tools, instruments in surgery, sports racquets, bats, manual handling, button pressing and so on may be betters served by this approach. Recent interface advances on the WII, Natal, sensors and cognitive controllers (using just your brain to do things), all point towards a future where learning by doing trumps the hokey, training world of role playing and breakout groups. Another relevant advance is the recent introduction of real time brain scanners which can (to a degree) evaluate the success of such training. Mind control and mind reading are now firmly on the horizon.