Do bears shit in the woods? 7 reasons why data analytics a misleading, myopic use of AI in HE

I’m increasingly convinced that HE is being pulled in the wrong with its obsession with data analytics, at the expense of more fruitful uses of AI in learning. Sure it has some efficacy but the money being spent at present, may be mostly wasted.

1. Bears in woods

Much of what is being paid for here is what I’d say was answers to the question, ‘Do bears shit in the woods?’ What insights are being uncovered here? That drop-out is being caused by poor teaching and poor student support? That students with English as a second language struggle? Ask yourself whether these insights really are insights or whether they’re something everyone knew in the first place.

2. You call that data?

The problem here is the paucity of data. Most Universities don’t even know how many students attend lectures (few record attendance), as they’re scared of the results. I can tell you that the actual data, when collected, paints a picture of catastrophic absence. That’s the first problem – poor data. Other data sources are similarly flawed, as there's little in the way of fine-grained feedback. It's small data sets, often messy, poorly structured and not understood.

3. Easier ways

Much of this so-called use of AI is like going over top of head with your right hand to scratch your left ear. Complex algorithmic approaches are likely to be more expensive and far less reliable and verifiable than simple measures like using a spreadsheet or making what little data you have available, in a digestible form, to faculty.

4. Better uses of resources

The problem with spending all of your money on diagnosis, especially when the diagnosis is an obvious limited set of possible causes, that were probably already known, is that the money is usually better spent on treatment. Look at improving student support, teaching and learning, not dodgy diagnosis.

5. Action not analytics

In practice, when those amazing insights come through, what do institutions actually do? Do they record lectures because students with English as a foreign language find some lecturers difficult and the psychology of learning screams at us to let students have repeated access to resources? Do they tackle the issue of poor teaching by specific lecturers? Do they question the use of lectures? (Easily the most important intervention, as the research shows)  is the shift to active learning. Do they increase response times on feedback to students? Do they drop the essay as a lazy and monolithic form of assessment? Or do they waffle on about improving the ‘student experience’ where nothing much changes?

6. Evaluation

I see a lot of presentations about why one should do data analytics  - mostly around preventing drop-out. I don’t see much in the way of verifiable analysis that data analytics has been the actual causal factor in preventing future drop-out. I mean a cost-effectiveness analysis. This is not easy but it would convince me,

7.  Myopic view of AI

AI is many things and a far better use of AI in HE, is, in my opinion, to improve teaching through personalised, adaptive learning, better feedback, student support, active learning, content creation and and assessment. All of these are available right now. They address the REAL problem – teaching and learning.

Conclusion

To be fair I applaud efforts from the likes of JISC to offer a data locker, so that institutions can store, share and use bigger data sets. This solves some legal problems but looks at addressing the issue of small data. But this is, as yet, a wholly unproven approach.

I work in AI in learning, have an AI learning company, invest in AI EdTech companies, am on the board of an AI learning company, speak on the subject all over the world, write constantly on the subject . You’d expect me to be a big fan of data analytics in HE – I’m not. Not yet. I’d never say never but so much of this seems like playing around with the problem, rather than facing up to solving the problem.