Caplan - The Case Against Education – some inconvenient truths….

Bryan Caplan has dared to ask an almost taboo question, ‘Could it be that we have too much education?’ Where is the economic evidence. part from the correlation between qualifications and salaries, which he explains through 'signalling. This is a brave question.

Education uber alles

In what is a deeply researched and comprehensive book, he concludes that education, especially Higher Education, is around 80% ‘signalling’, therefore much can be seen as of little value to society or even the students themselves. Not all, he still thinks there is essential value in the 20% proportion. A degree for many, however, has become a sticker on your forehead saying ‘hire me’. More people are getting ‘schooled’ for longer and longer and the percentage of one’s life being schooled is increasing. But to what end? Lots of people are now being prompted and pushed into being academic, when they are not, prolonging their schooling, when the evidence suggest that it “neither raises their productivity nor enriches their lives”. His ‘signalling’ theory explains some odd phenomena, such as prevalence of cheating, the final year being worth more than all previous years, rising graduate underemployment and so on. Signalling also raises salaries but not necessarily skills, through credential inflation, that is why he thinks it is so wasteful.

This leads him to the heretical claim that we should spend less on education, allowing that money to be spent elsewhere, such as health. He argues that arguments for lowering education spend tend to be politically unacceptable but “At what point would education spending be excessive?” - a reasonable question. His recommendation is that we need less spend as that will deflate runaway creditialism, without reducing skills.

Time and time again he uncovers key studies that are rather shocking, such as those on the lack of evidence for education improving critical thinking. He gets rather tired of educators telling him that what they do ‘can’t be measured’ – it is a cop out he claims. 

Schools

Why teach so much academic stuff for so long, for almost two decades, when they’re going to forget it anyway. Reading, writing and maths are necessary basic skills but much of the otherworldly curriculum, he argues, is outdated. Millions learning a foreign language in the US, merely for college admission, something they never use, and in any case couldn’t use, as they don’t gain even a basic competence. Schools have the odd and catastrophic result of making almost no one fluent in a language. As proof of futility of signalling he points out that we’re still teaching a dead language – Latin. Much of maths, especially algebra, abstruse number theory and geometry, and other subjects, are notoriously irrelevant, with little transfer, and even if useful, the knowledge is largely forgotten. The existing system seems unable to change, even when there is overwhelming evidence of failure – for example forgetting during the long summer recess.

Vocational learning

Caplan thinks we should Reboot vocational learning. This, he thinks, is a more worthwhile spend. We have evidence that it works from Germany, Austria and Switzerland. ‘Signalling’ theory tilts us towards rebalancing the system towards vocational. It raises pay, reduces unemployment and increases school completion.  The fact that it carries a stigma, reinforces his ‘signalling’ theory, where middle-class parents and employers rank vocational learners as inferior, is proof of his general proposition. However, the social return of vocational learning is clear and proven. In an interesting comparison, he notes that on-the-job training ‘internships’ are regarded as admirable for academic students but looked down upon in vocational learning. We have lost the admirable idea that early exposure to work, even Saturday jobs, are valuable, as “Early jobs are good for kids and good for society”. Unfortunately, we have a system that crowd-pleases the middle-classes, while the disaffected and school drop-outs become embittered. Forcing bored kids through a relentless diet of academic work for 13 or more years makes them resentful – it backfires. 

Higher education

Like Roger Schank, Caplan thinks there’s two things wrong with Higher Education, what we teach and how we teach it. Research on the idea that academia broadens horizons in also rather bleak. Post-testing, even among garduates in the US shows a woeful lack of skills and knowledge. Only around 60% of students attend lectures in the first place, no one takes courses in their University unless they are mandated (though it’s easy and free) showing a puzzling disinterest in wider intellectual development. One fewer degree, it would appear, would make little difference as credential inflation has been rampant. The answer to not having a good view at a concert may for the individual to stand up, but if everyone stands up, no one gains. Worse still, to push an academic track on the “failure prone majority is cruelly misleading”. 

International evidence

At the national level, he shows that research about the economic benefits of education seem to “vanish”. The effects when found, seem “puny” and do not seem to justify the vast sums spent on education, making it more of an act of faith than evidence based policy. There is even evidence that reverse causation may be at work here. It is not that schooling creates prosperity but prosperity leads to more schooling. The richer a country becomes, the more it spends on politically appealing education. Yet Harvard’s Lant Pritchett, formerly of the World Bank, did the data crunching and in a now famous article ‘Where has all the education gone?’ found little evidence between education and higher economic growth. Cambridge economist, Ha-Jon Chang refutes the idea that ‘more education in itself is not going to make a country richer’ and there are plenty of counter examples.

Inequalities

Even worse, could education spend be increasing inequalities in society? Challenging question. Inequalities are certainly rising, especially in countries with large education spends. Caplan argues that massive subsidies for education hurt the poor through credential inflation, which reshapes the job market to their detriment. The economy has not changed, we just have more graduates. The Road to Somewhere by David Goodwin, unpacks this phenomenon in the UK, in terms of the emergence of a 'graduate' class. A similar thing may have happened in the US and elsewhere. To continue on this treadmill, he thinks, is a mistake.

Online education

His signalling theory also explains why online education, that simply apes the current system, MOOCs for example, are bound to fail. No matter how successful they are, they don’t provide the ‘signal’  (the degree) and that’s what students are really paying for. Students are buying signals, not human capital. Online learning makes perfect sense in terms of access, flexibility, cost and convenience - yet all of that goes out the window if the signalling is absent.

Influence

As Caplan says, he is really a whistleblower. One of the reasons that education doesn’t really get put through the economic, sociological, political and pedagogic wringer, is that most of the people responsible for policy, have been through or work in its institutions. Confirmation bias is a powerful thing. His recommendations are that we focus on literacy and numeracy but cut back on spend to counter the non-productive ‘signalling’ and ‘credential inflation’ and spend more on vocational learning or other social goods, such as health, social care, whatever we decide. 

Bibliography

Caplan, B., 2018. The Case Against Education: Why the education system is a waste of time and money. Princeton University Press.

The Case Against Education – some inconvenient truths…. a must read

Bryan Caplan has dared to ask an almost taboo question, ‘Could it be that we have too much education?’ Brave question.

Education uber alles

In what is a deeply researched and comprehensive book, he concludes that education, especially Higher Education, is around 80% ‘signaling’, therefore much can be seen as of little value to society or even the students themselves. Not all, he still thinks there is essential value in the 20% proportion. A degree for many, however, has become a sticker on your forehead saying ‘hire me’. More people are getting ‘schooled’ for longer and longer and the percentage of your life being schooled is increasing. But to what end? Lots of people are now being prompted and pushed into being academic, when they’re not, prolonging their schooling, when the evidence suggest that it “neither raises their productivity nor enriches their lives”. His ‘signaling’ theory is persuasive, as it explains some odd phenomena, such as prevalence of cheating, the final year being worth more than all previous years, rising graduate underemployment and so on. Signalling raises salaries but not necessarily skills, through credential inflation, that’s why he thinks it’s so wasteful.

This leads him to the heretical claim that we should spend less on education, allowing that money to be spent elsewhere. He argues that arguments for lowering education spend tend to be politically unacceptable but “At what point would education spending be excessive?” - a reasonable question. In the US it’s $1.1 trillion. His recommendation is that we need less spend as that will deflate runaway creditialism, without reducing skills.

You’ll be thinking of all sorts of arguments against this, as I was, but just as you think you have an objection, Caplan has an answer, backed up by research. Time and time again he uncovers key studies that are rather shocking, such as those on the lack of evidence for education improving critical thinking. He gets rather tired of educators telling him that what they do ‘can’t be measured’ – it’s a cop out he claims. What I did like about this economist’s arguments was that he wasn’t scared to tackle educational issues, like the curriculum, quality of teaching and actual (as opposed to claimed) outcomes. 

Schools

Why teach so much academic stuff for so long, for almost two decades, when they’re going to forget it anyway. Reading, writing and maths are necessary basic skills but much of the otherworldly curriculum, he argues, is outdated. Millions learning a foreign language in the US, merely for college admission, something they never use, and in any case couldn’t use, as they don’t gain even a basic competence. Schools have the odd and catastrophic result of making almost no one fluent in a language. As proof of futility of signaling he points out that we’re still teaching a dead language – Latin. Much of maths, especially algebra, abstruse number theory and geometry, and other subjects, are notoriously irrelevant, with little transfer, and even if useful, the knowledge is largely forgotten. The existing system seems unable to change, even when there’s overwhelming evidence of failure – for example forgetting during the long summer recess.

Vocational learning

Caplan thinks we should Reboot vocational learning. This, he thinks, is a more worthwhile spend. We have evidence that it works from Germany, Austria and Switzerland. ‘Signaling’ theory tilts us towards rebalancing the system towards vocational. It raises pay, reduces unemployment and increases school completion.  The fact that it carries a stigma, reinforces his ‘signaling’ theory, where middle class parents and employers rank vocational learners as inferior, is proof of his general proposition. However, the social return of vocational learning is clear and proven. In an interesting comparison, he notes that on-the-job training ‘internships’ are regarded as admirable for academic students but looked down upon in vocational learning. We have lost the admirable idea that early exposure to work, even Saturday jobs, are valuable, as “Early jobs are good for kids and good for society”. Unfortunately, we have a system that crowd-pleases the middle classes, while the disaffected and school drop-outs become embittered. Forcing bored kids through a relentless diet of academic work for 13 or more years makes them resentful – it backfires. 

Higher education

Like Roger Schank, Caplan thinks there’s two things wrong with Higher Education, what we teach and how we teach it. Research on the idea that academia broadens horizons in also rather bleak. Post-testing, even among garduates in the US shows a woeful lack of skills and knowledge. Only around 60% of students attend lectures in the first place, no one takes courses in their University unless they are mandated (though it’s easy and free) showing a puzzling disinterest in wider intellectual development. One fewer degree, it would appear, would make little difference as credential inflation has been rampant. The answer to not having a good view at a concert may for the individual to stand up, but if everyone stands up, no one gains. Worse still, to push an academic track on the “failure prone majority is cruelly misleading”. 

International evidence

At the national level, he shows that research about the economic benefits of education seem to “vanish”. The effects when found, seem “puny” and do not seem to justify the vast sums spent on education, making it more of an act of faith than evidence based policy. There is even evidence that reverse causation may be at work here. It is not that schooling creates prosperity but prosperity leads to more schooling. The richer a country becomes, the more it spends on politically appealing education. Yet Harvard’s Lant Pritchett, formerly of the World Bank, did the data crunching and in a now famous article ‘Where has all the education gone?’ found little evidence between education and higher economic growth. Cambridge economist, Ha-Jon Chang refutes the idea that ‘more education in itself is not going to make a country richer’ and there are plenty of counter examples.

Inequalities

Even worse, could education spend be increasing inequalities in society? Challenging question. Inequalities are certainly rising, especially in countries with large education spends. Caplan argues that massive subsidies for education hurt the poor through credential inflation, which reshapes the job market to their detriment. The economy hasn’t changed, we just have more graduates. On that note I highly recommend The Road to Somewhere by David Goodwin, who unpacks the Brexit phenomenon in the UK in terms of the emergence of a 'graduate' class. A similar thing may have happened in the US and elsewhere. To continue on this treadmill, he thinks, is a mistake.

Online education

His signalling theory also explains why online education, that simply apes the current system, MOOCs for example, are bound to fail. No matter how successful they are, they don’t provide the ‘signal’  (the degree) and that’s what students are really paying for. Students are buying signals, not human capital. Online learning makes perfect sense in terms of access, flexibility, cost and convenience - yet all of that goes out the window if the signalling is absent.

Conclusion

Anyone who is serious about education policy and open minded enough to consider the idea, not that education is bad, but that we’re spending far too much on the wrong things, should read this book. As Caplan says, he’s really a whistleblower. One of the reasons that education doesn’t really get put through the economic, sociological, political and pedagogic wringer, is that most of the people responsible for policy, have been through or work in its institutions. Confirmation bias is a powerful thing. His recommendations are that we focus on literacy and numeracy but cut back on spend to counter the non-productive ‘signalling’ and ‘credential inflation’ and spend more on vocational learning or other social goods, such as health, social care, whatever we decide. This is not just a good book, it is essential reading.

Humboldt (1767-1825) – Prussian schooling and Higher Education… hugely influential…

After Napoleon had defeated Prussia in 1806, the Prussian state reflected on what had caused this catastrophe and out of that trauma came a reformed system of schooling designed by Wilhelm von Humboldt. The aim was not to produce rounded, autonomous citizens but young men obedient to a military culture and, oddly as he never attended school himself, he set about standardising many of the features of schooling we still see today.

Influenced by the ancient Greeks, his vision of education revolved around ‘character’ and the Platonic idea that education develops and leads one towards a fruitful life. This lde to him defining a school and University system that persists to this day.

Systematic schooling

The schooling that is still prevalent today is largely based on this Prussian model; a set school day, set syllabus, several subjects taught in periods across the day, rows of seats and bells to signal class changes. This stretched to standardised textbooks, state exams and inspections. Although his plans for reform were not published until after his death, along with a piece from a planned ‘Treatise of Human Education’. He saw education as providing a means to an end, the student’s interaction with the world, and not as an end-in-itself. The individual must do more than just satisfy their personal curiosity, they have personal, social and state responsibilities. He was, essentially, an Enlightenment thinker who saw progress of the individual, state and state of man, as the purpose of education.

However, his views on academic versus vocational education strongly influenced education today. He thought of education as a preparation for a life of skills and saw individuals as being prepared for vocations by receiving an academic education. First, vocational skills being easily acquired later. This was to have a profound effect on education globally, diminishing the vocational for the academic approach. Horace Mann, an American, who visited Prussia in 1843, brought this system back to the US and in 1852 Massachusetts adopted the system statewide.

Higher education

His greater influence was on Higher Education, where his combined model of research and teaching became and is now the global norm. This ‘Humboldtian Model’ was an Enlightenment idea, based on the freedom, cultivation and autonomy of the individual. To this end teaching should be informed by research.

Criticism

Unfortunately, it has led to the downgrading of vocational learning, apart, of course, from those Universities teach, like medicine, engineering and, oddly, dentistry. Bryan Caplan has shown the that this approach is wasteful for many students and society as a whole. As much of it is signalling for the employment market, it does not conform to the Humboldtian ideal of intellectual inquiry. Few students ever attend any other lectures than those mandated and even then drop-out is substantial. The cost to society is great in terms of cost and national debt. Caplan argues that much of this money could be better spent on other areas of social good, such as healthcare and vocational education. Even Germany, a strong economy, made sure that the academic and vocational systems were both robust.

Another criticism is the link between research and teaching. The evidence suggests that teaching skills, such as communications, presentation skills, interpersonal skills are not often high in those whose speciality is systematic research. The modern academic has two jobs research and teaching, their passion, training and skills are in the former, with little in the way of training in the latter.

Influence

Humboldt’s influence has been profound, in both schools and Universities. He set the tone for a system that aims to prepare and get young people into University, as a grounding for good citizenship. This has become the model for most countries in the developed world. No other figure matches him for direct causal influence on the structures and methods for education. Yet that influence may be the problem. It has become a mantra which Schank, Caplan, Thiel and many others consider wasteful.

Taming the AI & Assessment Conflict: 21 Practical Resolutions

There is a battle underway, with assessors in education and institutions on one side and Generative AI on the other. It often descends into a cat and mouse game but there are too many mice, and the mice are winning. Worse still is when it descends into a tug-of war, tech showdown or brutal legal collision.

We clearly need to harmonise assessment with our evolving technology and navigate our way through AI and assessment to avoid friction, with some practical solutions to minimise disruption. Assessment is at a crossroads.

Legal confrontations between assessors and the assessed will do great damage to institutions. When Sandra Borch, Norway’s Minister of research and higher education cracked down on plagiarism, taking a student to the Supreme Court, another student uncovered serious plagiarism on her Master’s theses – she had to resign. A more distasteful case was Claudine Gay, President of Harvard, who had to resign after an attack by a right-wing politician, Bill Ackerman. The whole thing turned toxic as people uncovered plagiarism in the work of his academic wife. As many US kids have lawyers as parents, a slew of cases are hitting the courts putting the reputations of schools and colleges at risk. This is not the way forward.

Problem

Assessment stands right in the middle of the earthquake zone as the two tectonic plates of AI and traditional education collide and rub up against each other. This always happens with new technology; printing, photocopying, calculators, internet, smartphones… now AI.

We are currently in the position where the mass use of AI is common, because it is a fabulous tool, but is being used on the SLY. There is widespread use of unsanctioned AI tools to save time. Learners and employees are using it in their hundreds of millions, yet educational institutions and organisations are holding out by ignoring the issue, or doing little more than issuing a policy document. AI is therefore seeping into organisations like water, a rising tide that never ebbs.

The problem has just gotten way more challenging as AI agents are here (automation). The Claude Sonnet 3.5 upgrade has just gone 'agentic', in a very practical way. It can use computers the way people do. It can look at your screen, go find stuff, analyse stuff, complete a chain of tasks, move your cursor, click on buttons and type text. To be clear, it understands and interacts with your computer just as you are doing now. This shift is in ability means it can to do open-ended functions like; sit tests, do assignments, even open research.

In truth…

We need to be more honest about these problems and stop shouting down from moral high horses, such as ‘academic integrity’. Human nature determines that if people find they are being played, or have an imperative that is different, they will take paths of least resistance. E.O. Wilson’s observation that we have “ Palaeolithic minds, Medieval institutions & Godlike technology” leads people to take shortcuts, through fear of failure, financial consequences, even panic. It is pointless to continually say 'They shall not pass' if the system has below par teaching and assessment is poorly constructed. We cannot simply blame students for the systemic failures of a system. Education must surely be a domian where teachers and learners are in some sort of harmony.

It doesn't help that we delude ourselves about the past. Cheating has been and still is common. We’re kidding ourselves if we think parents don’t do stuff for their kids at school and in universities. Essay mills and individuals writing assessments, even Master’s theses, exist in their tens of thousands in Nairobi and other places, feeding European and US middle-class kids with essays. Silk cloths with full essays go back to Confucian times. Technology can be bougt on the internet with button cameras and full comms into invisible earpieces. Cheating is everywhere. That's not to condone it, just to recognise tat it is ALWAYS a problem, nout just an AI problem.

In truth, we also have to be honest as accept that assessment is far too ‘text’ based. Much of it does not assess real skills or performance – even critical thinking. Writing an essay in an exam does not test critical thinking. No one writes critically starting top left, finishing bottom right – that’s why students memorise essays and regurgitate them in exams. Essay setting is easy, actual assessment is hard. We also have to be honest and accept that most educators designing and delivering assessment know little about it.

In the workplace, few take assessment seriously. At best it is multiple choice or e-learning thinly peppered with MCQs. L&D doesn’t take assessment seriously because they are not driven by credentials, nor do make much effort to evaluate training. With MCQs, you can guess, (1 in 4), distractors are often poor or simply distract, are difficult to write, easy to design badly, often too factual or unreal, require little cognitive effort and can be cheated, (longest, opposites etc.). An additional problem is that online authoring tools lock us into MCQs.

Assessments are odd. People settle for 70-80% (often an arbitrary threshold) as tests are seen as an end-point. They should have pedagogic import and give the learners momentum, yet there is nothing meaningful on improvement in most assessment and marking. Even with high scorers, full competence rarely the aim as a high mark is seen as enough, not full competence. The aim is to pass the test not master the subject. 

Plagiarism checkers do not work, so DO NOT use detectors. Neither should you depend on your gut – that is just as bad, if not worse. There are too many false positives and they consistently accuse non-native speakers of cheating. Students KNOW this tech better than you. They will always be one step ahead and even if they are not, there will be places and tools they can use to get round you.

Neither does setting traps for the mice, like "Include in your work icitations from <fictional name>", as when employed, once the trap is revealed, the learners can use the tech to reveal the fictional trap.

In a study by Scarfe et al., GenAI submissions were seeded into the exam system for five undergraduate modules across all years of BSc Psychology in a UK University. 94% of the AI submissions went undetected, with the AI submissions getting grades a half grade boundary higher than real students.

In a recent survey of Harvard students, some had made different course choice because of AI, other reported a sense of purposelessness in their education. They were struck by the fact that they were often learning what will be done differently in a world of AI. It is not just assessment that needs to change but also what is taught to be assessed. Assessment is a means to an end, assessing what is known, learnt or taught. If what we need to know, learn to teach changes, so should assessment.

Solutions

Calculators generate numbers, GenAI generates text. We now live in a post-generative AI world where this is the norm. Most writing is also now digital, so why are so many exams written? 

Most writing in the workplace is not a postmodern critique of Macbeth but fairly brief, bureaucratic and banal, getting things done by email, comms, plans, docs and reports. Management is currently laden with admin and GenAI promises to free us from admin to focus on what matters - the goal. It is here to stay because there is a massive need in the real world to raise productivity on speed and quality and not nget bogged down on redrafting or pretending you are Proust.  Why expect everyone to be writers of brilliant prose, when the goal is to simply get things done.

1. We have to move beyond text-only assessment into more performance-based assessments. Kids go to school at 5 and come out up to 20 years later having did little else other than read, write and comment on text. There is this illusion that one can assess skills through text – which is plainly ridiculous. Accept that people use GenAI to improve their writing beyond their own threshold. Encourage them to use AI to help them make their writing morce concise through summarisations. Allow them to critique their own work through AI.

2. Build 'pedagogy' into creating assessments with AI. We have done this by taking the research, on say transfer and action, then building that into the AI assessment creation process. You get better, more relevant assessment items, along with relevant rubrics.

3. Also build good assessment design practices into creating assessments with AI. There are clear DOs and DON’Ts in the design of assessment items. Build these into the AI creation process. Go further and match assessments to quality assessments standards. Believe me, this can be done in AI.

4. Match assessments more closely to what was actually taught. This alignment can be done using AI, including the identification of gaps, representative coverage, weaknesses on emphasis identified. The documents and transcripts used in teaching and/or the curriculum, can be used by AI to create better quality assessments.

5. Do more pre-assessments. David Asubel said “The most important single factor influencing learning is what the learner already knows.” I totally agree, yet it is rarely done. This gives assessment real pedagogic import – it propels or feeds forward into the learning process and helps teachers. These can be created quickly by AI.

6. Let's have more retrieval practice. This can be created quickly by teachers, even learners themsleves. We know that this works better than underlining notes and highlighting. Making leatrners or learners themselves making the effort to recall ideas and solutions intheir own minds helps get stuff into long-term memory.

7. Move away from MCQs towards short open text, assessed by AI. Open text is intrinsically superior as it demands recall from memory rather than identification and discrimination (correct answer in MCQs is there on the paper or screen). Open response more accurately reflects actual knowledge and skills.

8. Move away from authoring tools that lock you into fixed, templated MCQ assessment items. They also template things into rather annoying cartoon content with speech bubbles etc. Here's 25 ways I think bad design makes content and assessments suck.

9. Use more scenario and simulation assessment. They match real world skills, have more relevance and can set more sophisticated assessments on decision making and other skills. AI can create such scenario and sims assessment and the content you need to populate the scenarios.

10. On formative assessment, 'test out' more. Testing out means allowing people to progress if they pass the text. they may be able to skip modules, even the entire course. This should be the norm in compliance training, or training full stop, where people get the same courses year after year.

11. Get AI to create mnemonics and question-based flashcards for learners to self-assess, practice and revise and create personalised spaced-practice assessment, so they can get learning embedded. 

12. On formative assessment, use more AI designed and delivered adaptive and personalised learning. Adaptive learning can be 1) PRE-COURSE: Student data or pre-tests define pathways. (Never use learning styles or preferences). 2) IN-COURSE: Continuous adaption which needs specialised AI software (this is difficult but systems can do it). 3) POST-COURSE: With shared data across courses/programmes, also Adaptive assessment, Adaptive retention such a personalised spaced practice and performance support.

13. AI created Avatars can be built into assessments. These can be customers in sales or customer care training; employees in leadership. management and specific skills such as recruitment interviewing; or patients in medical education, where you can interact with people in assessments to provide realism.

14. Automate marking. Most lecturers, teachers and trainers have heavy workloads, many rightly complain about this, so focus on teaching not marking. Automated marking will also give you insights into individual performance and gaps. The Henkel study (2024), in a series of experiments in different domains at Grade levels 5-16 (Key Stage 2/3/4), showed that AI was as good at marking as humans.

15. Use audio to deliver assessment results, along with feedback and encouragement. This is more personal and motivating for the learner. It also forces the assessor to be more articulate and precise.

16. Use AI post-assignment or post-assessment techniques, such as generated audio questions that interrogate the learners understanding of their own work. Their audio answers could be transcribed by AI, even assessed by AI. 

17. Situates GenAi chatbots alongside the student assignment draft. Teachers have visibility of student-GenAi chatbot conversations when the work is submitted. Teacher prompts, mark schemes and other contextual information inform GenAi chatbot responses.

18. Make assessments more accessible on language and content. Far too many assessments have overly academic language or assessment items that are to abstract and can be turned into better expressed and relevant prose and problems. Translate overly-academic language to readable and vocational using AI. Critique and translate into more readable prose.

19. AI has revolutionised accessibility thorough text-to-speech and speech to text. It has now provided text and speech chatbots and automatically created podcasts (free NotebookLM) AI has also given us live captioning and real-time transcription. For dyslexics (5–15% of population), T2S & S2T, spell checks, Grammar Assistants, Predictive Text and voice dictation have been incredibly useful in reducing fear and embarrassment. AI can do wonders in making assessment more accessible.

20. Use AI for data analysis on your assessment data. It is as simple as loading up a spreadsheet and asking questions you want answered.

21. Stop being so utopian. Most people at school and University will not become researchers and academics. Don’t assess them as if that is ‘their’ goal.

22. What are the skills left over for education to focus on after you get GenAI into common use both in education, the workplace and life? The pat answer is too often - soft skills. I disagree. Move deeper into hard expertise and skills, with a broad perspective, that can be enhanced, magnified, even executed and automated by AI.

Radical thought

AI could disrupt the current system of higher education, which often prioritises signaling over actual learning, as argued in Bryan Caplan's The Case Against Education. Caplan suggests that many students pursue degrees not to gain knowledge or skills (human capital) but to signal qualities like conscientiousness and dedication to employers.

If building human capital were the primary goal, cheating would make little sense—it undermines the purpose of learning. However, in a signal-focused system, cheating becomes a way to appear diligent and capable without actually possessing those traits. AI, by making mass cheating more accessible, could devalue degrees, particularly those in softer disciplines, by stripping away their signaling power. Without this value, students might reconsider the utility of pursuing such education and shift toward activities that genuinely enhance their knowledge and skills.

Universities will resist this shift. As institutions that benefit from the current model, they are likely to find ways to preserve the signaling value of degrees rather than embrace a reset. Despite this, students should reflect on how to maximise the real value of their education. Four years dedicated to study is a rare chance to gain lasting knowledge and skills. By focusing on genuine learning instead of merely signaling, students can gain a meaningful advantage over peers who treat education as a hollow credential.

Conclusion

AI is moving steadily away from prompting to automation. This is happening in writing, coding spreadsheet analysis, image creation even moving image creation. It is happening with avatars and in real-time advanced dialogue as speech.

Just like calculators generated numbers GenAI generates text. We need to recognise this and change what we expect of learners. Turning it into a cat and mouse game will not work, there are too many mice and they're winning.

PS
This is a sort of summary of my talk in Berlin at the ATP European Assessment Conference.

 

Knowles (1913 - 1997) - Adult learning… Andragogy… lifelong learning...

We learn most of what we know, not as children but as adults, yet adult learning, sometimes called ‘Andragogy’ receives a tiny amount of attention in learning theory. Malcolm Knowles didn't give is the word, which goes back to the 19th C but he did give us a set of adult learning principles, a good counter to the clearly false idea that education and learning is confined to one’s youth. Knowles was not dogmatic about the word or his principles, but thought they were an adequate framework for dealing with adult learners. His theories have endured.

Six adult learning principles

Andragogy, meaning ‘man-led’ is to be distinguished from Pedagogy, meaning ‘child-led’. His core adult learning principles (andragogy) are:

1 Need to know – why, what, how

2 Self-concept– autonomous, self-directing

3 Prior experience – resource, mental models

4 Readiness to learn – life related, developmental task

5 Orientation to learning – problem-centred, contextual

6 Motivation to learn – intrinsic value, personal payoff

Adults differ in that they need to be more self-directed, driving their own learning with a focus on connecting new learning to prior knowledge. They need to see the relevance to their one work or life, with an emphasis on problem-solving not memorisation, with internal motivation being a more important factor.

Lifelong learning

What Knowles does reveal is the weakness of lifelong learning when seen as extended schooling. The answer to bad schooling is always more schooling. We may even want less learning. Yet for adults we may want less schooling and courses. As Bryan Caplan has argued, more people are getting ‘schooled’ for longer and longer. But to what end? Signalling. Credential inflation is the wasteful result. Academic qualifications are not the solution to adult learning and the dismantling of vocational learning has meant a surfeit of such courses at the expense of actual skills.

Life, for most, is for living, not learning. We learn to learn without formal structures, following our interests and curiosity, that was Knowles insight. Lifelong Learning is a phrase that appears in lofty reports, grant applications or by organisations that no one has even heard of. Nobody calls themselves a Lifelong Learner. It is a sort of educational conceit, often a hollow appeal by professional ‘educators’. Adults do not want to be infantilised by this sort of jargon. They are adults not learners. The older you get the less inclined you are to want to cram and sit exams, as you know you’ve forgotten most of what you previously learnt. People remain curious throughout their lives but life is not a course.

Criticism

Some question whether adult learning really is so separate that it deserves a separate learning theory. Knowles actually presents very little evidence for this bold assertion, Darbyshire (1993). Indeed, he compares adult learning with a rather outdated account of children’s learning, seeing children as resistant learners, when they’re actually just in a different context with different goals. It is not at all clear that adults are different in having the qualities Knowles claims. It seems odd that there is some boundary between childhood and adulthood where a different set of rules suddenly apply. Adults can be as stubborn, inactive and procrastinate as children. 

The recent emphasis on cognitive psychology suggests that efficient learning may not be that very different, even in adults. Others think that the assumptions are too strong and structured and that directed learning is still necessary. It may be the case that adults do not have the drive and motivation to learn without structured help.

Influence

The word Andragogy has gained some currency but mostly at an academic level. Adult learning still tends to be an orphan in the learning community, despite the fact that the vast majority of learning takes place by adults, a huge amount of it in the workplace. Knowles provides a welcome balance to learning theory. With an increase in the need for reskilling, a focus on lifelong learning, perhaps more leisure time due to automation and an ageing population, we may well have more time as adults to learn.

Bibliography

Knowles, M.S., Holton III, E.F. and Swanson, R.A., 2012. The adult learner. Routledge.

Knowles, M.S., 1980. The modern practice of adult education.

Tennant, M., 1986. An evaluation of Knowles’ theory of adult learning. International Journal of Lifelong Education, 5(2), pp.113-122.

Facebook causes cancer! The Big Debate

Fireworks in Berlin at 'The Big Debate', where Aric Sigman and I locked horns on the question:

“The increasing use of technology and social software is damaging students' minds and undermining the benefits of traditional methods of learning”.

I argued that it improved students’ minds and enhanced the benefits of traditional education.

'Facebook causes cancer' was a headline from the Daily Mail this year, sparked off by a paper written by Aric Sigman, in a peer reviewed journal called ‘Biologist’ (Well connected? The biological implications of ‘social networking’). Ben Godacre, Doctor and award winning journalist, author of Bad Science, and a debunker of some renown, took Sigman to task on Newsnight. It’s as good a demolition job as I’ve ever seen on Newsnight and I’ve seen a few. Even Paxman thought he was a nutter! (Also watch out for Susan Greenfield's admission that there is NO EVIDENCE.)

Sigman's Cherry picking

Back to the debate. I followed Goldacre’s line and attacked the original paper on the grounds that the papers Sigman cited did NOT mention social networking and were largely about medical effects in people over the age of fifty, in some cases even older.

Cole SW et al (2007) Social regulation of gene expression in human leukocytes

No mention of ‘social networking’

Tiny sample aged 50-67

Lamkin D M (2008) Positive psychosocial factors and NKT cells in ovarian cancer patients

No mention of ‘social networking’

Study of women over 65

Rutledge T et al (2004) Social networks are associated with lower mortality rates among women with suspected coronary disease

No mention of ‘social networking’

Mean age was 59

Cohen S et al (1997) Social ties and susceptibility to the common cold

No mention of ‘social networking’

1997 – way before social networks!

Ertel K A et al (2008) Effects of Social Integration on Preserving Memory Function in a Nationally Representative US Elderly Population

No mention of ‘social networking’

US sample of elderly adults

Deception

On top of this, on one citation, he deliberately failed to mention that the authors Kraut R et al (1998) (Internet Paradox: A Social Technology That Reduces Social Involvement and Psychological Well-Being?) who had discovered small negative effects of using Internet on measures of social involvement and psychological well-being among Pittsburgh families in 1995-1996, had in Kraut R et al (2001) (Internet Paradox Revisited) had changed their minds, “In a 3-year follow-up of the original sample, we find that negative effects dissipated over the total period. We also report findings from a longitudinal study in 1998-99 of new computer and television purchasers. This new sample experienced overall positive effects of using the Internet on communication, social involvement, and well-being.” That is more than cherry-picking by Sigman, it’s deception.

In fact the evidence, that Sigman knew about, but deliberately ignored points to the opposite:

1. Caplan SE (2007) “Relations among loneliness, social anxiety, and problematic Internet use.”

“The results support the hypothesis that the relationship between loneliness and preference for online social interaction is spurious.”

2. Sum et al (2008) “Internet use and loneliness in older adults“.

“greater use of the Internet as a communication tool was associated with a lower level of social loneliness.”

3. Subrahmanyam et al (2007) “Adolescents on the net: Internet use and well-being.”

“loneliness was not related to the total time spent online, nor to the time spent on e-mail”

Byron review

Tanya Byron was commissioned to look specifically at these issues by the UK government and in a well conducted and level-headed research project, collected a” vast array of evidence…commissioned three literature reviews:

• up to date research evidence on children’s brain development – Prof. Mark Johnson Birkbeck University

• comprehensive review on the vast body of child development research - Professor Usha Goswami Cambridge University

• current media effects literature in relation to video games and the internet – Prof. David Buckingham Institute of Education"

Annexes F, G, and H and at www.dcsf.gov.uk/byronreview

Some of her conclusions, relevant to this debate, were that, “there is no clear evidence of desensitisation in children”, “children actively involved in sport play on consoles for same amount of time as those who are not” and “technology specifically useful; for those with learning difficulties and disabilities”.

US Department of Education Study

In support of my proposition that technology enhanced education I then quoted from the US Department of Education’s study ‘Evaluation of Evidence-Based Practices in Online Learning: Meta-Analysis and Review of Online Learning Studies’ which looked at data from 1996 to 2008, selecting rigorous, measurable effects, random assignment and the existence of controls, “The meta-analysis found that, on average, students in online learning conditions performed better than those receiving traditional face-to-face instruction.” and “Instruction combining online and face-to-face elements had a larger advantage relative to purely face-to-face instruction than did purely online instruction.”

Finally I pointed toYouTube EDU, iTUNES U, Open Learn, MITOPENCOURSEWARE, Project Gutenberg and the Hole in the Wall project to show that there are some wonderful examples of enhancement.

Conclusion

Aric Sigman is the academic version of Sue Palmer, cherry-picking luddites who have books to sell, with titles like ‘Toxic Childhood’ and The Spoilt Child’. They’re part of a ‘parenting industry’ that creates and thrives on fear. It’s people like them that are promoting helicopter parenting and risk averse attitudes that lead to kids being locked up indoors, not the technology.

That's was pretty much my case. I only had 10 minutes, so summed up with a quote from Douglas Adams,

“everything that’s already in the world when you’re born is just normal;anything that gets invented after you’re thirty is against the natural order of things and the beginning of the end of civilisation as we know it until it’s been around for about ten years when it gradually turns out to be alright really. Apply this list to movies, rock music, TV, word processors and mobile phones to work out how old you are.”

We won the vote. I have to say that it was a great format and really got the juices flowing. Conferences should have more of this.

One last point. Sigman claimed that kids spend on average 7.5 hours a day online. I challenged this but he stuck to his guns. Now I don't know about you, but but mine would have to switch on the minute they got back from school and stay focussed until midnight every night without going to the toilet, eating etc. This figure alone makes him look ridiculous.

Appendix

Here’s some tweets and a blog post on the debate:

Why blame technology for something that depends on the home environment, parents must take responsibility for childrens learning

read his book on 'bad science' and then you'll see through people like aric

by the way, nasty of Aric to slag Ben Goldacre..he's not a journalist but a doctor and specialises in statistical misuse

Good fuel for a hot debate - extremely well selected speakers

Donald Clark : technologies helps inclusion. Very important

The Brits are demonstrating how to run a controversial debate. Fun.

Lectures on YouTubeEdu are improving education. Teachers get a larger audience

Donald Clark: USDE meta study found good support of e-learning

Donald Clark: Aric's studies based on the elderly, not using social networking

Sigman uses sources for his theory that are not about social networking

Sigman does not understand that social software is very social

listening to Aric Sigman I start to think we should call it OFFLINE Educa next year.

Aric Sigman: North Korea as the model for modern education - teachers get respect!

The sessions were rounded off with a 'debate' on the proposal that the internet is destroying our children's minds. A motion led by Aric Sigman who shouted and attempted to scare everyone. His extremely aggressive style offended some, particularly those unfamiliar with him (the vast majority of the 2,000+ international delegates), but for others gradually seemed like a raving madman. He attacked the audience as being pushers of this mind-rotting technology..not a great debating tactic, but he gives the impression of a man who cares about nothing other than his ego which was bloated by the use of the video projection screens, sadly.

He then was robustly challenged by Donald Clark who did a great job and was happy enough to show some passion and contempt for the scaremongering. The next two speakers were less effective. Bruce 'the Brute' (see Private Eye) Anderson, a veritable caricature of a fleet street hack, his tie slung askew muttered along the lines of trying to support the motion but being 'reasonable' (the old good cop/bad cop pairing), then some guy 'from Silicon Valley,' Jerry Michalski gave a fairly anodyne response to that...his analogy of the development of the 'automobile' with the net currently being at Model T wasn't a good one for a European audience, as a bicycling Dutchman commented!

Anyway, what needs to be said to those unfamiliar with Dr. Sigman is that cherry-picking (ie selective use of some reports and wilfully ignoring of other contradictory findings) seems to be his speciality, as pointed out by Ben Goldacre who he seemed to have a pop at during the session. If you want more on this aspect and some examples then visit this link.

OEB debate on Higher Education... 7 mins each... here's my 7 mins worth...

 ‘This House Believes Universities, in Their Current Form, Are Unsustainable as Mass-participant Institutions’

We each had 7 minutes to put our case… here’s mine…

My 1st argument is PERSONAL

I have two sons – twins, now 27 years old – interesting experiment – one went to University and has a degree in AI, one did NOT. He hated school and University, sitting in lectures would have sucked then life out of him. The one who did not has done as well as the other, runs his own business, and employs graduates. And that’s my point. We need to create a system that works for everyone not just the academic kids. 

We have an estimated shortage of 40,000 nurses in the UK, so let me tell you another personal story, both my mother and sister were nurses. They would not have been nurses now. The degree qualification excluded working class women and men from that profession. It increases class inequalities. We have to stop seeing University, as some sort of religious imperative. Like Catholic indulgences, where you have to pay to get the piece of paper or you fall into the HELL of low status and precarious employment. We need a Reformation and this motion suggests that. 

We need to REBALANCE our education system more towards people like my son, sister and mother.

2nd  argument is POLITICAL

In 2010 Peter Turchin, a scientist at the University of Connecticut who uses maths to model historical change, made a startling prediction in Nature. He said, “overproduction of young graduates with advanced degrees” would lead to social unrest due to rising inequalities. TTurned out he was right. The evidence… Trump, Brexit, the Gilet Jaunes in France… The graduate class has not only screwed over the rest of the working population over, it actually looks down on them – calls them deplorables…  The two best books I’ve read on Brexit by Stephen Davies and David Goodhart argue that this is what is behind such social discontent – the rising inequalities between the professional, urban, middle-class graduate classes and the rest. It was the graduates in Wall Street who caused the financial crash, nor ordinary working people.

We need to REBALANCE the inequalities in society and unite our DIVIDED societies, divided now by education…and not look down on people just because they didn’t go to University.

3rd argument is ECONOMIC

We have demoted vocational skills, outsourced manufacturing skills to China and wonder why our economies are faltering. Yet the graduate barista is now the norm. The destruction of vocational education has been a moral disaster and disgrace. But not everywhere. And I tip my hat to Germany, Austria and Switzerland who have more balanced systems and, unsurprisingly the strongest economies in Europe. If you think that Universities drive economic wealth think again. Taiwan, South Korea, and China grew their economies in spite of having low University infrastructure. 

As Bryan Caplan here at Online Educa, 80% of this is SIGNALLING but the cost of that 80% is needed elsewhere for healthcare and spending on those in real need.

In the US the cost as of now is a staggering $1.67 trillion to the national debt and don’t think that forgiving student debt is the answer. Forgiveness would be a highly regressive policy. Full cancellation would put $192 bn into the pockets of the top 20% of earners and a measly sum to the bottom. It does not rebalance, it increases inequalities.

We need to REBALANCE the system economically for growth and prosperity, as it costs too much.

4th is PEDAGOGIC

My son did a degree in AI at a European University. Few of his lectures were recorded (despite the students pleading for them to be so). Two hours of high end calculus is not easy and actually a bad way to teach calculus. Online resources were rare. Universities, globally, are still trapped in the 1 hour lecture (only because the Sumerians has a 60-base number system), the essay is still a dominant form of assignment and assessment. Yet look at the scale of essay mills. You can buy a freshly written Masters thesis for less than €1000. And when it came to Covid the rush to the university of Zoom – exactly what you should NOT do in online learning was bizarre. This is ONLINE EDUCA folks we should have done better…

We need to REBALANCE towards online delivery. All courses should be Blended Learning and digital by default.

5th is CULTURAL

Lastly a cultural argument. Universities, as Jonathon Haidt, in his excellent book The Coddling of the American Mind shows, our universities seem to be producing people with less, rather than more, resilience in the real world. We had the unedifying spectacle of junior employees of a global publisher Random House/Penguin literally crying their little eyes out when they heard that they employer was publishing a book by Jordan Peterson. I was raised in a world where crying, when you couldn’t get your own way, was abandoned with diapers - not whenever a work colleague thought differently from you. Debate is being cancelled. Diversity of though no longer respected.

We need to REBALANCE towards freedom of speech and diversity of thought.

Finally…

We’re in this debate but how many of us did NOT go to University? This, I think, is a problem. We have produced a graduate class that speaks largely to itself, in bubble conferences for graduates, talking about US. But this DEBATE is not about US!  

This is ONLINE EDUCA, so let us  pedagogically REBALANCE towards vocational, towards online and blended learning, to prevent further political, economic, pedagogic and cultural damage. With my friend Brian, I recommend this motion to the House.

OEB Berlin… AI, video, learning analytics, data, and schnapps!

Been going for many years… and like the vibe… fruity mixture of tech, academic, government from 73 countries. There's the added attraction of Berlin, a Christmas market right across the street and free drinks at the Marlene Bar!

Gave three talks, all on AI theme… AI and ethics (the overblown hysteria), Learning Analytics (how to and real examples), Video and AI (research and how to). A number of the sponsors were companies that use AI and it was a solid theme this year, rightly, as it has already changed why, what and how we learn.

As is usually the case at conferences I found the smaller presentations and conversations more useful than the keynotes. Great start with Julian Stodd, who was his usual articulate and incisive self. He talked about the weirdness of HR trying to ‘impose’ values and compliance training on people, attacking people’s sense of self and agency. But one phrase that really resonated with me was the ‘humility to listen’. There’s a lot of depth to those three words…

The opening keynotes were a trio of very different fruits. The Max Planc/MIT guy gave a solid talk, showed the Frey and Osborne report (2013), but got the date wrong – it wasn’t 2016 – this matters as it was a paper that predicted the 47% jobs at risk of automation over a decade in the US. We are 6 years in and there is pretty much full employment in the US. Toby Walsh eviscerated this report and talked at this conference two years ago – so we seemed to be going backwards. The Chinese guy was clearly giving a sales pitch but at least he had data and citations to back up his case. Audrey Watters gave her standard  ‘it’s largely agitprop, ideology and propaganda’ replete with soviet posters. Oddly she mentioned being jeered at a summit in Iceland. I was there - it was a very small audience and the first question she was asked (by a woman) was whether she was throwing the baby out with the bathwater, neither was there a 3D cat. She rightly showed some claims that were unsubstantiated but out of context and actually several are evidence-based but Audrey’s was so keen to show that everyone else was ideological that she missed the fact that hers was the most ideological talk of the three. But oh how academe clapped.

The keynotes on the second day (HE session) tackled the future of HE. Professor Shirley Alexander showed the shocking costs, debts and default rates of HE – it is basically out of control on costs. But her solution, literally on the next slide was a huge, spanking new building they’ve just erected and some writing feedback software. I was convinced by nether the erection nor software, which has been around for decades. Bryan Alexander is always up for some fun and opened his talk in a Death Metal voice. Had a great conversation with Bryan about AI afterwards and he did the futurist thing – 3D printing, drones etc…didn’t really see any scalable solutions that tackled the cost issue.

One feature of learning conferences is a general refusal to face up to political issues such as cost and inequality. It is assumed that education is an intrinsic good, no matter what the cost.  No reflection on WHY Brexit, Trump, Gilet Jaunes and other political upheavals are happening, only a firm belief that we keep on doing what we do, no matter the cost. This is myopic. Bryan Caplan tried with his keynote last year, with real evidence, but once again we seemed to have gone backwards. I had a ton of conversations in the bar, in restaurants and over coffees on these issues. A refreshingly straight talk with Mirjam Neelen was one of many.

I liked the practical sessions on learning analytics. It is complex subject but offers a way forward that builds on a platform of data that can be used to describe, analyse, predict and prescribe learning solutions. With smart software (AI), it frees us from the fairly static delivery of media, which online learning has done for over 30 years. Speaking with the wonderfully named Thor and Christian Glahn, we opened up the world of xAPI, LXPs, LRSs and adaptive learning. Here lies some real solutions to the problems posed by the keynotes. 

Sure there are ethical issues and I gave a session explaining that AI is not as good as you think and not as bad as you fear. We went through a menu of ethical issues: Existential, employment, bias, race, gender and transparency. Every man, woman and their dog is setting up and ethics and AI committee, pouring out recommendations and edicts, often based on a thin understanding of ethics and the technology. Many seem designed to give people an excuse to avoid it and do nothing.

Enjoyed Mathew Day’s session on the use of video which is uploaded to the International Space Station, which they use just before they do a task. That’s what I call cosmic, performance support. I was on just before him and showed the evidence in learning theory on why video on its own is rarely enough for deep learning, as well as key evidence on what makes a good learning video, much of it counterintuitive – POV, slower pace, edit points, not so much talking heads, maximum length, adding active learning and so on.

So many interesting chats with people I knew and met for the first time. What I did walk away with was a sense that people are waking up to the possibilities of AI in learning, especially for teaching, Henri Palmer of TUI gave a great case study, showing how one can deliver a large project, super-fast at a fraction of the cost using AI created online content. Great to hear that her team won a Gold Award for that project the night before in London. 

Final dinner in Lutter and Wegner, an old German restaurant was great. Harold did his pitch-perfect Ian Paisley impression at full volume with much clinking of glasses… wine and schnapps. When you’re sitting next to people from Norway, Poland, France, Belgium and Trinidad – you can’t go far wrong.

BIG thanks to Channa, Astrid, Rosa, Rebecca, Harold and the team for inviting me… open people who not only do a great job organising this event but are also open-minded enough to encourage critical thinking…