Mitra – Hole in the wall… but holes in the research…

Sugata Mitra has become famous for his hole-in-the wall projects and has gone on to evangelise his SOLE (Self Organised Learning Environments), the granny and now the school in the cloud. His TED talk is well known.

Hole-in-the-wall

Mitra placed an online computer in a wall in New Delhi in 1999, available to anyone to use. Mitra then claimed that children not only used the computer but began to navigate and learn, even deep conceptual learning, without instruction. The experiment was repeated in many locations in different countries and termed MIE by Mitra – minimally Invasive Education.

SOLE

Self Organised Learning Environments are structured, three-phase, learning sessions – question (5 minutes), investigation (40 minutes), review (10-20 minutes). It involves group work (groups of four), each group with access to a computer, students can move groups at any time and it is OK to share. It is not clear how this is a particularly new approach to inquiry-based learning, other than the prescriptive nature of the groups and computers. The School in the Cloud is a structured environment for safe search and tools for curation. An addition feature is the use of grannies to support the learning. The rather misleading term ‘Grannies’ refers to any online facilitator, male or female, young or old, who mediate SOLE.  They operate from schools, community centres or private homes, using Skype. 

Criticism

“What we see is the idea of free learning going into free fall” said Payal Arora. When Arora came across these two ‘hole-in-the-wall’ sites (Almora and Hawalbagh in northern India), she discovered not the positive tales of self-directed learning but failure. One was vandalised and closed down within two months, the other abandoned and, apparently, had been mostly used by boys to play games. A real problem was sustainability, as no one seemed responsible for the electricity and maintenance bills. My own research into a hole-in-the-wall project uncovered the same story - empty holes in the wall, resentful teachers and testimonies that claimed the whole project was a failure from start to finish. People arrived at their school, knocked holes in their walls, inserted computers and left. They felt violated. The computers rarely worked, as the DSL line was often down, and when it did work, the larger boys dominated them, playing games.

Little independent evidence

As Arora (2010) points out, there is little real independent evidence, other than that provided by HiWEL itself and one must always question research funded by those who would benefit from a positive outcome. The lack of independent research on the sites is astonishing, something noted by Mark Warschauer, one of the few critics who have actually visited a site. Most of it comes from Mitra himself, or those in his team, almost all from one Journal. Control groups were given questionnaires at the start and end of the period, but those in the experimental group were tested every month. The obvious problem here is the polluting effect of effect the regular assessments. Indeed, as De Bruyckere et al. (2015) say, there is ample research, from Reedier & Karpicke and others on the positive effect of testing.

Arora exposed a glaring weakness in the design of the experiment. The 75 days of learning (with a mediator) was compared to the same period in the local school but like was not being compared to like, so the comparison was meaningless. It was not comparing the amount of time spent on the hole-in-the-wall material with the same or similar amount of time in school.

Role of schools

“Schools are obsolete” said Mitra. Far from being sited in open places, HiWEL sites were later invariably in school compounds. By being in the school it is difficult to do research that isolates the experience from the school, difficult to disentangle the role of the school (teachers, books etc.) and the hole-in-the-wall computers. Indeed, as HiWEL has explained, they involve ‘teachers’ in their implementation and mediation, making it almost impossible to isolate the causes of educational improvement. One could say, with Arora, that this has become “self-defeating”. The ‘hole-in-the-wall’ has become the ‘computer-in-the-school’. This is a subtle switch - evangelise on one premise, deliver on another.

Role of teachers

As HiWEL makes extensive use of mediators (teachers), the real lesson of the hole in the wall experiments is that teachers, or at least mediators, seem to be a necessary condition for learning to combat exclusion, mediate learning and avoid the vagaries of child-centred behaviour. Yet this is not what the TED talks and hole-in-the-wall evangelism suggests. Another problem is that by seeing teachers as ‘invasive’, such initiatives can antagonise teachers and educators, leading to poor-support.  I found this in my research in Africa, where the teachers were resentful. Arora concludes are that these experiments do not work when not linked to the local schools and that, far from being self-directed, the children need mediation by adults. Arora goes further and claims that disassociating learning from adult guidance can lead to uncritical acceptance of bad content and bad learning habits.

Low level learning

Warschauer (2003) is even more critical than Arora. He claims that “overall the project was not very effective”, with low-level learning and not challenging. In addition, he found that some of the many problems were the fact that the internet rarely functioned, no content was provided in Hindi, the only language the children knew, and many parents thought that the paucity of relevant content rendered it irrelevant and criticised the kiosks as distracting the children from their homework. Sure they learned how to use menus, drag and drop but most of the time they were “using paint programs or playing games”. This is hardly surprising and seems to confirm the rather banal Influence that when you give kids shiny new things, they play with them. 

Influence

It is not clear that this is anything other than traditional project or inquiry based teaching. The TED effect has given Mitra a celebrity status but little independent research has been done on his work and when researchers do look, they find a rather different story. What we can say is that his work has stimulated interest in how far we can push independent learning, given the power of new technology.

Bibliography

https://donaldclarkplanb.blogspot.com/search?q=mitra

Arora, P. (2010), Hope-in-the-Wall? A digital promise for free learning. British Journal of Educational Technology, 41: 689–702. http://www.payalarora.com/Publications/Arora-HopeintheWall.pdf

Koseoglu, S. (2011). The hole in the wall experiments: Learning from self-organizing systems. Retrieved from http://umn.academia.edu/SuzanKoseoglu

Mitra, S., & Rana, V. (2001). Children and the Internet: Experiments with minimally invasive education in India. British Journal of Educational Technology, 32 (2), 221-232.

Mitra, S. (2003). Minimally invasive education: A progress report on the ‘Hole-in-the-wall’ experiments. The British Journal of Educational Technology, 34 (3), 367-371.

Mitra, S. (2005). Self organising systems for mass computer literacy: Findings from the 'hole in the wall' experiments.  International Journal of Development Issues, 4 (1), 71-81.

Mitra, S. (2006). The Hole in the Wall: Self-organising systems in education. Noida, UP: TataMcGraw Hill.

Mitra, S. (2009). Remote presence: ‘Beaming’ teachers where they cannot go. Journal of  Emerging Technology and Web Intelligence, 1 (1), 55-59.

Mitra, S., Dangwal, R., & Thadani, L. (2008). Effects of remoteness on the quality of education:A case study from North Indian schools. Australasian Journal of Educational Technology, 24 (2), 168-180.

Mitra, S., & Dangwal, R. (2010). Limits to self-organising systems of learning - The Kalikuppamexperiment. British Journal of Educational Technology, 41 (5), 672-688.

Roediger H.L. & Karpicke J.D. (2006) Test enhanced Learning, Psychological Science
Warschauer, M. 2003. Technology and Social Inclusion: Rethinking the Digital Divide. Cambridge, Massachusetts: MIT Press.

Warschauer, M. 2009. ‘Digital literacy studies: Progress and prospects’. In The Future of Literacy Studies, edited by Baynham, M; Prinsloo, M. London: Palgrave Macmillan: 123–140.

Khan's HOLES IN CHEESE trounces Mitra’s HOLES IN WALLS , as he talks about mastery & motivation

Good to see Salman Khan speaking at BETT, albeit by video link, rather than the all-too-predictable and fanciful Mitra. Salman Khan trounced Sugata Mitra’s gloriously irrelevant, utopian nonsense, with a level of relevance, detail and understanding that is always impressive. While Mitra talks a lot about education in the developing world with holes in walls, grannies and the hopelessly naive SOLE, Khan really has provided a free, world-class education for anyone, anywhere for hundreds of millions of learners.

Master - Swiss Cheese Problem

Khan's diagnosis of problems in some subjects, especially maths, has become known as the ‘Swiss Cheese Problem’. In a linear curriculum, with a large cohort of students, an assessment system that ignores mastery and demotivates through the wrong mindset in teaching and learning - large-scale failure is inevitable. He identified the problem precisely when he said that too many students in maths get moved on to the next topic without mastering what they NEED to know in order to progress. Most students are moved on with varying degrees of knowledge and skills, even the best performing students, with 80-90% scores still have a 10-20% shortfall. This shortfall varies from student to student, with often significant misconceptions and absences dotted around their mnds like conceptual holes in a Swiss Cheese.

Khan’s solution, and I agree, is to close these personal holes and move forward through the subject in a peronalised fashion catching and closing the holes, one by one, leaving no holes left behind. In practice, this means less focus on summative testing and fixed grades. The emphasis, he believes, should be on far more formative assessment and adaptive learning that leads towards complete mastery.

Motivation also

In addition, and I also agree, he also sees the need for motivational factors to be added, so that the student is not derailed by fixed mindsets around ability and talent. The hideous 'gifted and talented' movement is typical but much of the language around maths is about talent and ability, not perseverance and resilience. Dweck is his theoretical source for improving the motivational component. (for more on Dweck see here.)

Technology

The only way to do this on the necessary scale, he thinks, is to use technology. We have never, in the past, had the technology that could deliver this level of adaptivity, while maintaining motivation. Now we have, he claims. There are systems available that deliver personalized mastery through smart formative assessment, formative motivation and smart realtime use of algorithms and gathered data, to both eliminate the holes in the cheese, as well as maintain motivation. I have been involved in such work, have seen the data and the impact. He is right.

Conclusion

Like Wikipedia and MOOCs, whenever you mention Khan, you get that wrinkled face look and the educator’s favourite word ‘but….”. This is the Khan Academy’s tenth year of operation and it has been groundbreaking. Khan’s combination of an acute pedagogical analysis of the two key problems (mastery and motivation), both in the minds of learners and methods of teaching, are a real wake-up call. Technology is not about shoving devices into walls but using devices to close down the conceptual holes when you teach – that is what adaptive, personalized learning offers. Khan, for me, is way smarter than Mitra (or Robinson) and way further down the road in terms of real solutions.

Mitra’s SOLE – 10 reasons on why it is ‘not even wrong’

It saddens me when learning methods are  reduced to simplistic, ill-informed acronyms and practices. SOLE is one such acronym, or as one teacher trainer called it a ‘pseudo-framework’. SOLE is not a method based on sound learning theory, it’s not new and at worst a destructive force in education, doing, in my opinion, more harm than good. Mitra most recent pronuncement “knowledge is dead” is what the physicist Wolfgang Pauli described as “not even wrong”. In other words, so wrong it is barely worth critical comment. But here goes….

What is SOLE?

Self Organised Learning Environments (see here for method) are structured, three-phase, learning sessions – question (5 minutes), investigation (40 minutes), review (10-20 minutes). It involves group work (groups of four), each with access to a computer, students can move groups at any time and it’s OK to share. There’s some advice on choosing questions that will spark curiosity. So far, so good, but that’s about it.

1. Not new

What is not clear how this is a particularly new approach to inquiry-based learning, other than the prescriptive nature of the groups and computers. Even Socrates 2500 years ago would ask carefully chosen questions to groups of young men in the agora, let them think, discuss, then come back with a critique. Indeed, the Socratic method, when unpacked, is a good deal more sophisticated than Mitra’s SOLE method, as it was based on uncovering misconceptions and superficial knowledge. Since then there has been centuries of teachers working with groups by answering questions. Socratic Circles and seminars have been used for centuries. Question-led group work was also a strong feature of Worker’s Education Societies and many other adult education initiatives. In the early 1900s Kurt Koffka explored this area, then Kurt Lewin in the 20s-40s, Morton Deutsch in the 40s-50s, David and Roger Johnson in the 60s. Then we have the whole Vygotsky, Bandura, Murray, Tjosvold… I could go on and on. On top of this is the simple fact that almost every teacher I’ve ever known employs this or something similar in their teaching strategies (note the plural). It is quite simply old news.

2. Optimal groups

I’d take issue with Mitra’s optimal group size at ‘four’. Research has been done on this. Indeed, I was involved in just such as study on language learning (French), where the researcher tried various groups sizes, measuring attainment. The optimal number was ‘three’ not ‘four’. There were several reasons for this. First, a computer comfortably sits one in the middle with one on either side. The fourth, when added, was often marginalised. Then, in terms of social interaction, the good news was that they started speaking French, not only to the screen but to each other (rare in traditional teaching) but side to side in dialogue. More participants diluted the collective effort. Mitra’s model is not researched and, in my opinion, not optimised. A more general problem with this type of groupwork, as any teacher knows, is social-loafing, with one student ‘driving’ the computer, the others being more passive, even excluded.

3. Questions

The SOLE website is remarkably simplistic on what types of questions are to be asked in the SOLE method. This is not a trivial issue. In the recent Wired article, the question was “Why do dogs chase cats?”. As is so often the case, it is not clear, how these questions relate to context – curricula, teaching, level of existing knowledge and so on. Black and Wiliam have written extensively on this subject with far more insight and many more concrete suggestions on what constitutes good questions. Mitra’s suggestions are basic, almost trite.

4. Knowledge is dead?

Most worrying of all is Mitra’s statement in the recent Wired article, that “Knowledge is dead”. Mitra has form here. He has said on various occasions that schools and teachers are dead. He did, of course, backtrack as soon as he found that the Hole-in-the-wall methods were failures without teacher support.

You don’t have to be a Hirsche or Willingham to see how muddled Mitra’s views of knowledge, teaching and learning really are when he pops out these platitudes. Critical skills need a body of knowledge to act upon, if critical inquiry is to be fruitful. What is often ignored is the danger of this in terms of social equality. A complete pendulum swing away from knowledge to critical thinking is likely to led to increasing levels of inequality, as the poor receive an inferior education while the wealthy retain this more valuable mix of educational strategies. Far from aiding the poor, Mitra may be doing untold damage.

Let me illustrate this using the “Why do cats chase dogs?” example. It’s a reasonable question that requires a complex answer, including some knowledge of evolution, genetics, artificial breeding, selection and domestication. Do the children we see really understand any of these concepts? Will they pick up these complex ideas without teacher intervention?

5. Schools are obsolete!

As I said earlier, Mitra has form here. Remember his “Schools are obsolete”statement. Far from being sited in open places, HiWEL sites are now invariably in school compounds. By being in the school it is difficult to do research that isolates the experience from the school, difficult to disentangle the role of the school (teachers, books etc.) and the hole-in-the-wall computers. Indeed, as HiWEL has explained, they involve ‘teachers’ in their implementation and mediation, making it almost impossible to isolate the causes of educational improvement. One could say, with Arora, that this has become “self-defeating”. The ‘hole-in-the-wall’ has become the ‘computer-in-the-school’. This is a subtle bait and switch - evangelise on one premise, deliver on another.

6. Criticism

On this idea of SOLE, hole-in-the-wall strategies, let’s look at the evidence. “What we see is the idea of free learning going into free fall,” said Payal Arora. When Arora came across these two ‘hole-in-the-wall’ sites (Almora and Hawalbagh in northern India), she discovered not the positive tales of self-directed learning but failure. One was vandalised and closed down within two months, the other abandoned and, apparently, had been mostly used by boys to play games. A real problem was sustainability, as no one seemed responsible for the electricity and maintenance bills. 
My own research into a hole-in-the-wall project uncovered the same story - empty holes in the wall, resentful teachers and testimonies that claimed the whole project was a failure from start to finish. People arrived at their school. knocked holes in their walls, inserted computers and left. They felt violated. The computers rarely worked, as the DSL line was often down, and when it did work, the larger boys dominated them, playing games.

7. Little independent evidence

As Arora (2010) points out, there is little real independent evidence, other than that provided by HiWEL itself and one must always question research funded by those who would benefit from a positive outcome. The lack of independent research on the sites is astonishing, something noted by Mark Warschauer, one of the few critics who have actually visited a site. Most of it comes from Mitra himself, or those in his team, almost all from one Journal. Control groups were given questionnaires at the start and end of the period, but those in the experimental group were tested every month. The obvious problem here is the polluting effect of effect the regular assessments. Indeed, as De Bruyckere et al. (2015) say, there is ample research, from Reedier & Karpicke and others on the positive effect of testing.

8. Not like with like

Arora exposed a glaring weakness in the design of the experiment. The 75 days of learning (with a mediator) was compared to the same period in the local school but like was not being compared to like, so the comparison was meaningless. It was not comparing the amount of time spent on the hole-in-the-wall material with the same or similar amount of time in school.

9. Role of teachers

As HiWEL makes extensive use of mediators (teachers), the real lesson of the hole in the wall experiments is that teachers, or at least mediators, seem to be a necessary condition for learning to combat exclusion, mediate learning and avoid the vagaries of child-centred behaviour. Yet this is not what the TED talks and hole-in-the-wall evangelism suggests. Another problem is that by seeing teachers as ‘invasive’, such initiatives can antagonise teachers and educators, leading to poor-support.  I found this in my research in Africa, where the teachers were resentful. Arora concludes are that these experiments do not work when not linked to the local schools and that, far from being self-directed, the children need mediation by adults. Arora goes further and claims that disassociating learning from adult guidance can lead to uncritical acceptance of bad content and bad learning habits.

10. Low level learning

Warschauer (2003) is even more critical than Arora. He claims that “overall the project was not very effective”, with low level learning and not challenging. In addition, he found that some of the many problems were the fact that the internet rarely functioned, no content was provided in Hindi, the only language the children knew, and many parents thought that the paucity of relevant content rendered it irrelevant and criticised the kiosks as distracting the children from their homework. Sure they learned how to use menus, drag and drop but most of the time they were “using paint programs or playing games”. This is hardly surprising and seems to confirm the rather banal conclusion that when you give kids shiny new things, they play with them. The danger with SOLE learning is the classroom is the illusion of deep learning. Without formative feedback and guidance, the learners may well be thrashing about to no great effect.

Conclusion

It is not clear that this is anything other than traditional project or inquiry based teaching. The TED effect has given Mitra a celebrity status but little independent research has been done on his work and when researchers do look, they find a rather different story. It’s all too easy to tell a few anecdotes, with some well-selected pictures of kids at a screen, even slum kids, and present this as a polished, keynote narrative. Teachers need to be more critical than to fall for this stuff. ‘I’m a Celebrity let me fix your education system’ should not be a substitute for careful thinking, scrutiny, evidence and critical analysis.