Monday, September 26, 2011

7 reasons why Kafka would have loved assessment

I’m in Prague this week, speaking to the world’s test providers . Prague was Franz Kafka’s city, whose unfinished masterpiece The Trial tells of the accusation, arrest and relentless pursuit of Josef K. He doesn’t know what he’s done wrong but the whole world seems determined to put him to trial and find him guilty. This is not far removed from the modern obsession with testing. Young people are in a perpetual world of exams and are not sure why the world is so determined to accuse them and find them guilty of not knowing huge tranches of weird stuff. Here’s just seven Kafkaesque features of modern assessment:
Sell ‘cheating’
We tell people tests are merictocratic but the test community sells ‘tests’ and also courses on how to ‘cheat’ those tests. Companies like Kaplan and others sell expensive courses that tell you literally, how to cheat. Imagine a business school that runs courses on banking, and at the same time sells courses on robbing banks! (Then again….) This is immoral and a sign that the tests are not what they claim to be, immune from improvement through tutoring and practice.
Mindless maths
We a dull & irrelevant curriculum, teach it badly, test it endlessly and wonder why they hate it or fail. PISA set the wrong pace, with STEM close behind, so politicians have become obsessed with the weird world of abstract maths, despite the fact that the vast majority of students will NEVER use the quadratic formula, surds, trigonometry or any algebra at any point in their lives. Abstract maths is easy to test, so the tests drive the curriculum. The majority of learners actually need ‘functional’ maths, fit for practical living, not the tiny minority that go on to do STEM degrees.
Summative is too late
The educational system is structured like horizontal layers of impermeable rock. The learner has to punch vertically upwards through these strata, with exams at every stratum, designed to halt progress for the majority. This obsession with summative assessment also means that formative assessment suffers. Teachers teach to the test. In short, we test too late, when the damage is done.
Tests measure failure
Precious few people get 100% in any test, so testing almost always tests below full competence. Why don’t we test until full competence is achieved, rather than accepting second-best? This is what simulations and games approaches do and therefore offer. Why can’t we go for systems of smart, adaptive assessment that assure competence at every stage?
Test and forget
Most exams test knowledge that will be forgotten within days or at most weeks. Ebbinghaus proved this in 1885, yet we still operate a system that follows the ‘cram, test, forget’ method. Part of the problem is the fact that we have abandoned ‘learning by doing’. Tests favour ‘knowing that’ as opposed to ‘knowing how’. Imagine an Olympics with only a few medals available for a few, pure athletics races and the rest are rubbish.
Luddites
Most tests are still done using pens. Two points: 1) few students and workers use pens in the real world, they use keyboards; 2) not giving students the chance to restructure and rewrite essay answers leads students to memorise and regurgitate set essays and answers. Critical thinking through writing is all about rewriting, so why not give them the ability to word process? We test using primitive technology that actually stops them from showing competence.
Abysmal quality control
Recent A-level exams contained impossible questions in a range of subjects from major test organisations such as AQA, OCR and Edexcel. Their quality control was so abysmal that they hadn’t tested their papers with even a SINGLE student. They claim to have statistically eliminated the problem by adjusting marks. Just how did they measure the distress and distraction in trying to answer an impossible question?
Conclusion
We test to blame, whether its students, schools even entire educational systems, which at times has led to a pathological view of education, and the demonization of state schooling. There’s so much testing going on, that relevance, innovation, skills, honesty and quality have gone down the plughole. We’re stuck with a Kafkaesque approach that is relentless, bureaucratic, accusatory and often tests the wrong things for the wrong reasons, killing the desire to learn. We’ve turned our children into a generation of Josef Ks.

 Subscribe to RSS

Tuesday, September 20, 2011

Higher Education? Myths, mantras, illusions and delusions by Hacker & Dreifus


It’s all in the ‘question mark’. Hacker & Dreifus are academics who have no interest in destroying higher education. They do, however, think it has gone badly wrong, in the US at least, straying from its real purpose. Their charge is that Universities have become insular, inward-looking and self-serving, abandoning teaching in favour of research. Politicians, parents, students and external commentators are waking up to the parallel universe that is the modern university, a feather-bedded, faculty culture that seeks to avoids teaching, with high costs, sabbaticals, endless conferences and dubious research. Does their analysis apply to the UK? Let’s see.
Teaching
Their cardinal charge is the high cost students pay, through debt, for low quantity and quality teaching. Now that people pay top dollar they are asking what they get in return. The reality, uncovered in detail,  is that academics seek to abandon teaching as their careers progress. The bulk of undergraduate teaching (70%) is now done by low paid, part-time ‘contingent’ staff. In some cases foreign students with English so poor they can’t be understood, especially in maths and science (a complaint I’ve heard many times from students in the UK).
The stronger charge is that academics don’t care much for teaching undergraduates “nor do they feel the need to”. They witness appalling attitudes towards students, and in particular, poor teaching with low preparation, poor skills, little eye contact, and therefore little attention by the class, also a propensity to blame students for lack of attention. Why? Because teaching ability is not valued. Research is what leads to permanent employment and promotion.
“Good teaching is only possible if professors are also active in research” is the mantra the hear time and time again. But the pressure to shovel more and more research into more and more journals, with fewer and fewer readers and attend more conferences, with fewer attendees, is a road of diminishing returns. Research, through the publishing virus, is now a reputation and resume issue, largely divorced from teaching. Indeed, newer teaching institutions have massively expanded research, at the expense, they say, of teaching. This research-led teaching myth also promotes the teaching of inappropriate, esoteric topics. In short, there’s a need to de-link or further disengage research from teaching.
Having attended both a US and UK universities and seen lots of academics present, I would say that, if anything it is worse in the UK. We have a more reserved culture, where highly analytic researchers find it difficult to face up audiences. Post-92 we also had a massive uplift in teachers who do research. I’m not convinced that this has been to the benefit of either research or teaching in the UK.
Dropout rates
Shocking figures are presented on dropout rates. Estimates for those who drop out in the first semester lie between one quarter to one third of all freshmen. Engineering courses have huge dropout rates, with faculty holding the view that their job is to ‘weed out’ poor performance, an approach they refuse to apply to themselves. This lack of interest from faculty astounds the likes of Eric Mazur, a physicist and teaching expert at Harvard, who claims that a meeting on falling teaching standards had the, “lowest turnout he could recall”.
We also have a problem with increasing dropout rates, with huge variations between institutions, but although less than the US. However, like the US, poor teaching and student experiences seems to be one cause, especially in the newer Universities.
Bureaucratic behemoths
Universities have also loaded up with odd (they give job titles), and in their view superfluous, administrative jobs and show that the ratio of administrators to students has doubled in 30 years. With these jobs comes office space and buildings, adding significant costs to students’ fees. This war of words between faculty and administrators is familiar in the UK.
Extravagant facilities
The costs of huge sports facilities and teams, and the distortions this places on finances and learning are explored in detail. This is not such a problem in the UK, but our propensity to throw up buildings, that are empty most of the time, is just as bad. At least in the US they have more commitment to summer schools and keeping campuses open for students year round.
Poor performance
Failing academics are almost impossible to fire, so poor teaching and lazy research abounds, with ‘academic freedom’ used as the shield to defend poor performance. On top of this academia clones itself, leading to groupthink and a lack of academic diversity. Indeed, the evidence they cite shows that an academic, like Ward Churchill, is far more likely to be dismissed for exercising academic freedom, something well documented by Lionel Lewis’s Cold War on Campus, where Universities caved into McCarthyite demands for sackings.
We’ve had no McCarthyism in the UK, but the sclerotic and over-protectionist policies towards faculty staff are possible more entrenched. As the evaluation in the performance in teaching is low, it is never a serious cause for the necessary weeding and feeding that Universities need to stay vibrant. Even when low research output is delivered, attempts to remove faculty are greeted with hysterical responses e.g. Sussex and Middlesex.
Skewed admission
Lazy admission policies lead to middle-class parents paying for personal essays (in the UK personal statements). On SATS, the rich pay thousands of dollars for courses from Kaplan and Princeton Review to ‘cheat’ the exam. Is there any other area of human endeavour where respectable educational institutions offer courses on ‘cheating’? It’s like a Business School offering a course on ‘Robbing banks’. (Mmmmm…)
Vocational/academic
The authors unashamedly support a return to a Liberal Arts model with undergraduate courses leading to vocational specialisms. They are rather snooty about engineering, MIT and CalTech. Although they have a pop at the Ivy League and rankings that reflect a “premium on prestige” and brands not teaching and learning. The Golden Dozen in the US come in for some heavyweight criticism for below par teaching.
This is mirrored in the UK, where vocational degrees (apart from medicine) are regarded as below par. On the other hand we have some worthy Universities, such as Heriot-Watt and Manchester and UCL that still carry the banner for solid, professional vocational courses. In may ways we’ve never adopted the Liberal Arts BA then Medschool/Lawyer model from across the pond. That’s to our credit.
Recommendations
They’re a bit lightweight on technology in learning (they call it techno-learning) but smart enough to see that it’s a solution, with a detailed case study on Humanities 2510 at a Florida  College, where the costs have been halved with increased attainment. Great to see the wonderful Carol Twigg research (LINK) being used as the inspiration for this course. There’s nothing quite like the OU in the US, then again, the great failing in the UK was the failure to apply the OU model, even for high-volume theoretical courses, where it clearly works.
“The future is here…it’s just not evenly distributed”, said William Gibson, so price and product are not aligned, so shop around. They name a whole raft of small colleges offering excellent degrees at a fraction of the price of bigger institutions. Some colleges have simply dropped excessive spending on sports teams and stadia, others are wholly committed to students and teaching, not research, even free tuition. Braver still is Evergreen College, where grades and fixed curricula have been abandoned; students get a long written evaluation.
Good advice in the UK. We have similar snobbishness around prestige and brands. The ranking tables are a disgrace, ignoring teaching. This, I feel, is set to change.
Toxic debt & bubble
The core problem is the indirect subsidy of 2nd and 3rd rate research. Reduce research and you improve teaching and reduce costs. It’s that simple. So they call for more student-centred institutions, less sabbaticals, less research, less administrators, more scrutiny of bad teaching and more leadership. How many people can actually name the Vice Chancellor of their local University? They have become anonymous apparatchiks, obsessed with research not teaching, building not technology, chasing gongs not glory. Couldn’t agree more.
It’s the debt that’s attracting attention. As costs have risen by 250% in real terms in the US and much higher levels in the UK, we should be worried. This has all the signs of a bubble. We have deferred payment through loans (debt) to a generation who may not be able to pay and bankruptcy is no escape. What if these debts turn toxic?
The bottom line is that students and parents are being short-changed by entrenched values that have sacrificed learning for research. Academia has adopted the same stance as the finance sector, refusing to hold itself to account, adopting a prickly, defensive posture. The worry, for the authors, is that we’re creating an “indentured educated class”.
This may be less of an issue in the UK, as loans are recovered from tax with a one third expected loss guaranteed by government (still a loss). But the debt bubble is still a possibility, through a combination of evasion and inability to pay.
In many ways the arguments in this book can be applied to UK Universities, the notable differences being the US commitment to campus sport and a much larger number of private institutions. I’d suspect, therefore, that the debate has already reached these shores with fees looming, as we are now third in the world on the cost of Higher Education to students, with only S Korea and the US ahead of us. We would do well to listen to what these researchers have to say.

 Subscribe to RSS

Sunday, September 18, 2011

Brilliant student tool that saves tons of time


If I had to name just one time-saving tool for students, researchers and academics, it would be CITEME, a simple Facebook application. How many hours do students and researchers waste in, first working out what format should be used for citations; second writing and formatting them?

CITEME – how it works
Simply type in the name of the author, title, subject or ISBN and it will drop down a list of properly formatted citations, which you cut and paste. It’s that easy. You can also choose from a range of formats (Harvard, Chicago, APA, Turabian or MLA) as required by your institution and/or tutor.
The first problem is that academics are often vague about what format they require from their students. Is it Harvard, Chicago, APA, Turabian or MLA? To be honest it’s nit-picking, as the differences between these are miniscule, such as position of publication date and whether the date should have brackets. One thing you have to do is check the policy for format.

Saves tons of time
This tool will save you hours of time and allow you to focus on the content, rather than the searching for, writing and formatting of citations.  It also means you spend more time learning how to use proper citations. Simply write that essay or paper then go through adding the citations one by one in your chosen format. I’ve yet to meet a student who wasn’t grateful for this tip.

PS Thanks to Millie thesocialnetworkingteacher, who put me on to this tool.

 Subscribe to RSS

Saturday, September 17, 2011

Recording can improve a bad lecture! 7 surprising facts about recorded lectures


Universities churn out lectures by the thousands, yet academics, even academics who support the use of technology in education, often go completely gaga when you even dare to question their pedagogic worth. ‘Lecturers’ will go to any lengths, apart from actual research or data, to defend ’lecturing’, confusing a channel of teaching with learning, and form with function.
You don’t have to know much about the psychology of learning to realise that a series of once-only, delivered lectures is pedagogic nonsense. We learn next to nothing from once-only experiences like unrecorded lectures. Indeed, everything we know about learning shows that repeated access to content is necessary for learning.
I am always delighted, therefore, when yet another set of data confirms the obvious fact that students gain when they are given access to recorded lectures.  In this wonderful little study by Pierre Gorrisen, delivered at the ALT conference, they cleverly combined usage data with some survey and interview data to come to some clear conclusions. Their student-centred approach to the problem is refreshing. So what did they discover?
1. Students watch lots of recorded lectures at home
Surprise number one; students watch lots of recorded lectures in their own time. Learners look at content, do what is necessary for learning, reflect, take notes, elaborate and deep process – all the things that are necessary for learning yet extremely difficult, if not impossible, in a long, uninterrupted lecture.
2. No technical problems
Surprise number two; no technical problems. We now live in YouTube world where computers handle video and audio with ease. In another ALT presentation, Dragos Ciobanu, Neil Morris and Alina Secara showed how, at the University of Leeds, they are experimenting with Adobe Connect Pro. All you need is a URL to access their Adobe Connect room, and anyone with a laptop, iPad, iPhone, Android device could participate. It’s not just access to recorded lectures that is easy technically, it’s their delivery.
3. Watch lectures multiple times
Surprise number three; students watch lectures many times. This really should come as no surprise. Repeated, spaced practice is essential for real learning, deep processing, elaboration and therefore recall. Students, if not lecturers, understand this. When you read an academic text or paper, don’t you read things more than once? Of course you do. That’s how reading works. So why deliver a lecture once only?
4. Watch <75% of lecture
Surprise number four; they don’t watch the whole lecture. This is interesting as they found that not many students watch more than 75% of a lecture. Why is this? Well, let’s be honest, most lecturers are not that great at lecturing and tend to have overlong introductions, ramble, go off on tangents and include inappropriate and irrelevant material. Students are smart, so they fast forward. The standard one hour lecture is artificial time period that has nothing to do with the psychology of learning and exists only because the Babylonians had a base-60 number system. Lecturers therefore pad out content to fill the hour.
5. Would like to see ALL lectures recorded
Surprise number five; students want ALL lectures recorded. Damn right. No one would dream of just giving students access to a single recording of a book or paper. That would be lunacy. So why give them unpublished lectures? They want them to learn and revise for exams. It’s that simple. To deny them recording is to deny them the opportunity for learning. It’s what they’re paying for.
6. Improves pass rate
Surprise number six; improves pass rates. Now this is important. The researchers claim, although I saw no data in the presentation, that recorded lectures increase the pass rate. This was confirmed by studies I’ve cited previously from the Institute of Theoretical Physics in Trieste and others. If true, then let’s just get on with it and record them all.
7. Recording lectures improves bad lectures!
Surprise number seven; a recording of a bad lecture can actually improve that lecture….. What a wonderful observation. The fact that you can skip the bad bits, replay and watch it several times actually improve the experience. Now I don’t believe that lectures should hold such pedagogic sway in education. And even if you were to show me that lectures were a powerful teaching technique, I’d still claim that the majority of lecturers are not up to the task. But if you still insist on lectures, why not record them on this one principle only, that bad lectures can be improved by recording?

 Subscribe to RSS