I’m increasingly convinced that HE is being pulled in the
wrong with its obsession with data analytics, at the expense of more fruitful
uses of AI in learning. Sure it has some efficacy but the money being spent at
present, may be mostly wasted.
1. Bears in woods
Much of what is being paid for here is what I’d say was answers to the question, ‘Do bears shit in the woods?’ What insights are being
uncovered here? That drop-out is being caused by poor teaching and poor student
support? That students with English as a second language struggle? Ask yourself
whether these insights really are insights or whether they’re something
everyone knew in the first place.
2. You call that
data?
The problem here is the paucity of data. Most Universities
don’t even know how many students attend lectures (few record attendance), as
they’re scared of the results. I can tell you that the actual data, when
collected, paints a picture of catastrophic absence. That’s the first problem –
poor data. Other data sources are similarly flawed, as there's little in the way of fine-grained feedback. It's small data sets, often
messy, poorly structured and not understood.
3. Easier ways
Much of this so-called use of AI is like going over top of
head with your right hand to scratch your left ear. Complex algorithmic approaches
are likely to be more expensive and far less reliable and verifiable than
simple measures like using a spreadsheet or making what little data you have
available, in a digestible form, to faculty.
4. Better uses of
resources
The problem with spending all of your money on diagnosis,
especially when the diagnosis is an obvious limited set of possible causes, that were probably already known, is
that the money is usually better spent on treatment. Look at improving student
support, teaching and learning, not dodgy diagnosis.
5. Action not
analytics
In practice, when those amazing insights come through, what
do institutions actually do? Do they record lectures because students with English
as a foreign language find some lecturers difficult and the psychology of learning
screams at us to let students have repeated access to resources? Do they tackle
the issue of poor teaching by specific lecturers? Do they question the use of
lectures? (Easily the most important intervention, as the research shows) is the shift to active learning. Do they
increase response times on feedback to students? Do they drop the essay as a
lazy and monolithic form of assessment? Or do they waffle on about improving
the ‘student experience’ where nothing much changes?
6. Evaluation
I see a lot of presentations about why one should do data
analytics - mostly around preventing
drop-out. I don’t see much in the way of verifiable analysis that data
analytics has been the actual causal factor in preventing future drop-out. I
mean a cost-effectiveness analysis. This is not easy but it would convince me,
7. Myopic view of AI
AI is many things and a far better use of AI in HE, is, in
my opinion, to improve teaching through personalised, adaptive learning,
better feedback, student support, active learning, content creation and and
assessment. All of these are available right now. They address the REAL problem
– teaching and learning.
Conclusion
To be fair I applaud efforts from the likes of JISC to offer
a data locker, so that institutions can store, share and use bigger data sets.
This solves some legal problems but looks at addressing the issue of small
data. But this is, as yet, a wholly unproven approach.
I work in AI in learning, have an AI learning company,
invest in AI EdTech companies, am on the board of an AI learning company, speak
on the subject all over the world, write constantly on the subject . You’d
expect me to be a big fan of data analytics in HE – I’m not. Not yet. I’d never
say never but so much of this seems like playing around with the problem, rather than facing up to solving the problem.
