Monday, July 22, 2024

Future of AI is here but not evenly distributed - some real SURPRISES on use, gender, age, expectations & ethics

A few surprises in this excellent paper on use of ChatGPT on ‘The Adoption of ChatGPT’ by Anders Humlum from the University of Chicago (July 9, 2024).

It was a large-scale survey (Nov 2023 - Jan 2024) of 100,000 workers from 11 exposed occupations, linked to labour market histories, earnings, wealth, education and demographics to characterise the nature of adoption of ChatGPT. Note that this was before release of certain improved AI models such as ChatGPT4o and Claude 3.

The 11 occupations, a good selection, were; 

HR professionals, Teachers, Office clerks, IT support, Software developers, Journalists, Legal professionals, Marketing professionals, Accountants, Customer Service and Financial advisors.

Use


almost everyone is aware of it 

50% had used technology

adoption rates from 79% for software developers to 34% for financial advisors

differed in their intensity of use


As expected and the authors confirm the “widespread adoption of ChatGPT, only a year after its first launch, solidifies it as a landmark event in technology history.”

Age a factor

younger & less experienced workers more likely to use ChatGPT

every year of age has 1.0 percentage point lower likelihood of using ChatGPT 

similarly with every year of experience at 0.7 lower

Gender gap

Women less likely to use tool:

20 percentage points less likely to have used the tool

pervasive in all occupations

in various adoption measures

persists when comparing within same workplace

also controlling for workers’ task mixes

Expectations

Employees perceive:

substantial productivity potential in ChatGPT

average estimates that ChatGPT can halve working times in a third of job tasks

smaller rather than larger time savings for workers with greater expertise

38% say they will not perform more of the tasks ChatGPT saves time completing

Interesting experiment

In a clever experiment, inserting exposure to expert assessments of the time savings from ChatGPT in their job tasks, they found, “despite understanding the potential time savings from ChatGPT, treated workers are not more likely to use ChatGPT in the following two weeks”. This is rather worrying for those who see training, frameworks and documents as the solution to increasing use.

Finally, and this really matters, they investigated what prevents workers from transferring the potential productivity gains from ChatGPT into actual adoption. 

Workers reported barriers:

restrictions on use

needing training as the primary barriers to adoption

need for firm policies (guidelines for use or facilitating employee training) 

few reported existential fears, becoming dependent on technology or redundant in their jobs, as reasons for not using ChatGPT

This last point is important. Employees are not too concerned with ethical issues and do not seem to fear dependency or redundancy.

Conclusion

There's a ton of other useful stuff and detail in this paper but the Surprises that stood out for me were the lower uses by women, strong barriers to adoption by employers, despite workers agreement htat is will increase productivity and indifference to ethical concerns.

Bibliography

Humlum, A. and Vestergaard, E., 2024. The Adoption of ChatGPT. University of Chicago, Becker Friedman Institute for Economics Working Paper, (2024-50).


AI projects: What to do? How to do it? Let me introduce you to Nickle LaMoreaux, the CHRO (Chief Human Resources Officer) at IBM

For all the ‘big’ reports from Deloitte, BCG, PWC etc, I prefer the more measured views from people within organisations who are doing smart stuff and learning from it. One I particularly liked, as it chimed exactly with my own experience on implementing AI within small and large organisations comes from IBM.

Let me introduce you to Nickle LaMoreaux, the CHRO (Chief Human Resources Offices) at IBM.

She sees HR as ‘client zero’ for lots of initiatives. I like this up front statement, as HR is so often simply reactive or sidelined into initiatives that are not focussed on business improvement.

It just seemed reasonable from someone who is actually doing stuff, not just talking about it of doing surveys. She’s practical and has to deal with the bottlenecks, cultural resistance and hierarchies within a large organisation. We should listen.

What to do? The 3 cs!

1. Consumer-grade experiences: delightful experiences enabled & enhanced by technology. 

I loved this. Go for time saving and increasing quality but if you want to effect change, go for something visible, that touches people. GenAI clearly does this as the billions of uses per month show that dialogue works. It is simple, engaging and delivers what people want. It gives them agency.

2. Cost efficiencies 

Cost efficiencies are so 2024, so she also went for a solid project and measured the efficiencies in terms of time saved and quality. “HiRo saved 60,000 hours for Consulting managers in a year”. That’s what I want to hear, a solid project with measurable outputs. She unashamedly wants to see a high return on investment. This is refreshingly businesslike.

3. Compliance  

Spot on. This is the one area where AI can be leveraged to reduce time and, increase quality in processes and training. As she rightly says, it is “perfect fit for AI”. Complaince has become a nightmare at the national and international level. AI can really help in terms of processes, support, learning and keeping everything up to date. This is one area where the data side of AI can really help.

Identify Some Quick AI Victories

Don’t procrastinate – that’s the easy choice. AI is the here and now. The tools are available, use cases identifiable so don’t give into to the ‘wait and see’ attitude. She explains how to proceed by winning some smaller battles, not turning into a strategic war. This is correct. This is new, powerful technology, it has huge potency but also some vagaries. We need to proceed clearly but also with caution. What’s the point of creating an unnecessarily negative climate for future progress on the back of some failed and over-ambitious masterplan?

Choose “High-volume, repetitive tasks. Processes employees don’t enjoy. Moments that matter for employees.” This is so right. Large organisations are full of processes, which are tedious, bureaucratic, have bottlenecks, old technology solutions and are ripe for automation. They are often deeply embedded practices. Focus on these and the quick wins will come.

How to do it?

1. Start small and experiment 

This will vary across organisations but I doubt there’s a single organisation, small, medium or large that will not benefit from the use of AI across a range of uses. These need not be grandiose or abstract, such as skills recommendation prediction or large data analysis projects. They can be simple and manageable.

2. Learn as you go, fail fast & be agile 

Can’t emphasise this enough. This technology can be implemented fas=st against a goal. Don’t get bogged down in huge project plans – get on and do mit as the technology will get better as you process. It is not that you may have to pivot in some way on the technology and approach – you WILL have to pivot.

3. Lead with use case, not technology

This often comes down to finding a pain point, bottleneck or hated process for a quick win. It doesn’t have to be big just impactful. Make sure your goal is clear. 

4. Cover off data and security issues

You have to establish trust and allay fears in the projects chosen. This often comes down choosing a technology partner whose vision, goals and purpose align with your own. But some simple FAQs to vapourise the myths and calm fears is wise.

5. Build advocates out of your employees

I have seen this for real. Young employees, let loose to use this technology and show their worth. People who do process know what’s needed to redefine, short-circuit and improve those processes. Give them the agency to suggest projects.

Clear goals

I’ve seen projects founder when it was not clear what the final goal was. AI projects tend to shapeshift and that is fine in terms of swapping out LLM models used and changing tactics – this is normal and easier than people think. But don’t founder on the rock of vagueness. Be clear about goals

I’m not a fan of old-school SMART objectives, as what is needed is a solid goal and sub-goals. This FAST framework, from MIT, sums it up perfectly:

Conclusion

We are in the first phase of seeing the benefits of GenAI in organisations. Before we head off to climb that great single peak in the distance, take time to conquer a few foothills. Choose your hills carefully and once chosen, make sure you focus on them. Stay ambitious, measure that ambition and shout it from the hill top when you get there! Thanks Nickle.

Sunday, July 21, 2024

Collaborative Overload Harvard Business Review - why collaboration often sucks and what to do about it!

Neat study in Harvard Business Review on the disturbing impact of increased collaboration in the workplace on employees and organisational efficiency. 

I literally gave a sigh of relief when I saw and read this. We’ve all been there – in meetings where we know many should not be there,, Everyone these days wants to schedule a Zoom meeting, when an email would probably suffice. Overlong zoom calls with far too many attendees, some paying little attention. Poorly chaired meetings, everything scheduled for an hour, sometimes longer. Too many presentations and not enough decision making. 

I’ve long had the suspicion that collaboration, especially in meetings but also in learning, is over-egged and often results in poor productivity. That’s what my recent post on ‘quiet learning’ was about.

The study, based on data collected over two decades and research conducted across more than 300 organisations, found that time spent in collaborative activities by managers and employees has increased by 50% or more but here’s the rub, 20% to 35% of value-added collaborations come from only 3% to 5% of employees. These high levels of collaboration consume valuable time and resources, often leading to low performance and stress. The result is actually overburdened employees who become bottlenecks, leading to decreased personal effectiveness and higher turnover rates.

In truth, time and energy are finite, and knowledge can often be shared asynchronously, without over-long meetings. Smaller teams, more focus on goals, moving fast – shared resources – that’s the way to increase productivity

They urge us to:

Leverage technology to make informational and social resources more accessible & transparent

Encourage employees to stop, filter & prioritise requests

Educate employees to Just Say No or allocate ‘limited’ time for requests

Promote use of informational and social resources over personal resources

Implement practical rules for email requests and meeting invitations

Use tools to monitor and report time spent on collaborative activities

Promote natural F2F collaborations by co-locating interdependent employees

Include collaborative efficiency in performance reviews, promotions, and pay raises

Neat... but it all goes a bit off beam with the idea of hiring Chief Collaboration Officers to manage and promote effective collaboration within organisations. Davide Graeber’s brilliant book ‘Bullshit Jobs’ comes to mind.

Was done some time back and of anything I feel things have got a lot worse.

 

Recruitment is a cat and mouse game. AI gives the mice real edge! 30 ways candidates use AI to get new jobs…

I gave a talk on AI for recruitment in Scandinavia this year, showing how ‘recruitment’ was one of the first things to be hit by AI. For all the abstract talk on AI, let's get practical.

There was the famous interviewee who got a job as a space engineer by taking the online questions from the interviewer, applying speech to text along with insertion into Chat GPT, then coming back with answers, which he read in his own words. He had no knowledge in the domain but was offered the job! This was unusual but candidates are almost invariably using AI to increase their chances.

In truth, anyone who recruits needs to know about what is happening here. My talk was largely about how recruiters can use AI to improve the process of recruitment but first you need to know how potential candidates are using AI. Believe me – they are!

So here’s my Cliffnotes for job seekers….

30 ways candidates use AI to get new jobs

General

Prompt for jobs you may not have thought of that match your quals/CV and experience

Write and refine speculative email/letter

Use Ad, Job description, org website, anything you can find in your prompts

CV/Letter

Personalise CV for every new job

Identify key skills for that job then adjust your CV

Rewrite CV with key points at top

Rewrite with keywords from ad/Job Description in CV

Make CV more concise/turn into bullet points

Summarise down to 1 or 2 pages

Continue to create/adjust/rewrite your CV to match Ad/Job Description

Summarise your CV for the covering letter

Create the covering letter

Critique/proofread your final CV/covering letter

In online applications rewrite entries before inputting

Create your own avatar and send with CV!

Interview preparation

Ask for top ten interview questions (interviewers are lazy)

Ask for likely questions from Ad/Job Description

Use CV to create great answers for each predicted question

Make your achievement answers more memorable

Create answers that showcase specific skills from your CV

Include keywords from Ad/Job Description do I need to include in my answer

Create questions you may want to ask

Create credible answers for gaps in CV

Go to web site and get AI to summarise what ‘organisation’ does

What challenges/opportunities does the organisation have with solutions

Interview practice

Prompt with your Job description and CV and get AI to interview/role play for the job

Try speech dialogue in ChatGPT4o for realistic spoken dialogue

Negotiation

Create a negotiation strategy for salary

Create follow up email if you haven’t heard

Create thank you email even when you don’t get job

PS

If you’re feeling uneasy with this list, then I recommend that famous quote from Catch22…

From now on I'm thinking only of me. 

But, Yossarian, suppose everyone felt that way?

Then, said Yossarian, I'd certainly be a damned fool to feel any other way, wouldn't I?

Next post will be 30 things you can do as a recruiter…


Saturday, July 20, 2024

A plea for quiet learning for adults against the cacophony of the training room or e-learning...

David Foster Wallace’s wrote ‘Infinite Jest’ a huge sprawling novel that is free from the usual earnestness of the novel. You need to find the time and quiet to read its 1000+ pages. It requires effort. Here he explains why reading is now often cursory, if people read at all.

I’m not as fanatical as most about reading, especially the ‘reading Taliban’, kids spending hours and hours reading Harry Potter. I’m with Plato on this and think that being out doing stuff is often better when you’re very young. Nevertheless, he has a point about solitary silence. I want to apply this to learning.

All my life I’ve been told that learning is ‘social’, ‘collaborative’ and should be done in ‘groups’ or ‘cohorts’. Most education is assumed to benefit from being in a ‘classroom’, ‘lecture hall’ or ‘training room’. I’m not so sure. It strikes me that most learning is done afterwards in the quiet of your bedroom with homework, in the library as a student or when reading and reflecting on your own as an adult or doing your job.

Teaching and training tends to be fuelled by people who think that you need to be in their company to learn anything. This is rarely true. I much prefer being free from the tyranny of time, tyranny of location and the tyranny of transience. That means the tyranny of ‘courses’.

My least favourite and one I refused to attend or having anything to do with, are those round table sessions where you choose a chair, get some vague question, discuss and feed back with flipchart sheets stuck on the wall. It's simply an exercise in groupthink. Nothing good comes from it as it's just a lazy, embedded practice.

I learn best when it is at a time and place I choose and not at some arbitrary pace set by a teacher, lecturer or trainer. I’m not wholly against the classroom, lecture or course, as it is clearly necessary to bring structure to young people’s lives for sustained attention in schools and for critical tasks. But for most learning and especially adult learning I’m not convinced.

One cognitive phenomenon that confirmed my views is the ‘transience effect’. Perhaps the least known but most important effect in learning. When you watch a video, television or a movie, listen to a lecture or get talked to for a long period, you will be under the illusion that you are learning. In fact, due to your working memory’s inability to hold, manipulate and process them enough to get into long-term memory, you forget most of it. It’s like a shooting star  - it shines bright in your mind at the time but your memories burn up behind you.

This is why I am suspicious of using a lot of non-recorded lectures, even video in learning. Even note taking is hard in a fixed narrative flow, as you’re likely to miss stuff while you take the notes and still don’t really have time to deeply process the information. I have no problem with recorded video or podcasts as I can stop and start, as I often do, to reflect and take notes.

What is annoying can be the cacophony of bad e-learning, with its noisy, cartoonish animations, visual and sound effects, especially in gamified content. I find it almost unbearable to page through this stuff, with its speech bubbles and often pointless interactions.  I posted this six years ago but you get my point…
https://youtu.be/BlCXgpozrZg

Being alone in the quiet (even music has a negative effect, especially if it has lyrics) actively reading, taking notes, stopping, reflecting and doing stuff works well because you are in the flow of your choosing, not someone else’s narrative flow. Agency matters as learning, the actual presence of knowledge and skills in your long-term memory, is always a personal experience. It requires your attention, your effort and your motivation.

For thousands of years the loudest things we would have heard was the background noise of nature, something that we barely register now. The church bell was perhaps the loudest non-natural thing one would have heard for centuries. Now, it’s the din of traffic, sirens, TV on, radio on, computer on… something’s always there. The cacophony of music in pubs, shops and public spaces drowns out thought or worse stimulates annoyance, even anger. 



US-EU digital divide deepens as Meta and Apple hold back product – it’s a mess

I warned about this over a year ago in Is the new Digital Divide in AI between the 'EU' and 'Rest of the World'? We saw Italy stupidly ban ChatGPT, then a similar rush to get a the Digital Markets Act and EU AI Act on top of the very odd GDPR legislation. As Taylor Swift says ‘players gonna play, haters gonna hate, fakers gonna fake’. That’s what this is largely about. Technocrats and legislators feel the need to play their regulatory game, fuelled by an intrinsic dislike of technology they’re going to rush it and because they’ve rushed it they make up things to legislate against.

AI has not proved to be a threat to anything or anybody. We’ve had several huge elections, including the EU parliament and even the bogeyman of deepfakes did not materialise. Meanwhile, the US economy is shooting ahead of Europe. The real issue is one of productivity and economic growth, not moral hazard.

This shows how the market is dominated by the US and China, not a single EU company is in this list:

Google

Three out of the top four EU fines have been against Google. To be fair these were good anti-competition fines that were right, not the more recent laws. It is here that the EU could play a strong and worthy role. However, it has not corrected the outright theft of revenues from tax havens within its own countries, most notably Ireland, where tax revenues are literally stolen on sales from other countries.

Facebook

The negativity and legislation in the EU is now coming home to roost as Meta have decided not to release their open-source ‘multimodal’ version of its Llama models in the EU. This is a crushing blow to both research and start-ups who use these models for product. This was in reaction to the setting of deadlines this month set by the EU. Note that this was actually in response to the opacity of GDPR not the new EU AI Act which will bring on a whole other set of problems.

Meta have made this bold move as the EU has not provided clarity over GDPR in relation to the use of customer data to train its AI models. My guess is that this is Facebook posts.

Apple

Apple have already stated that it is likely that Apple Intelligence will NOT be released in the EU later this year. Your iPhone and Mac will therefore be severely restricted within the EU. In fact, Apple Intelligence, is a clever attempt to protect your private data through edge computing. Apple's objection is, not against the EU AI Act but the Digital Markets Act. Apple claim it puts personal data and privacy at more risk. This is the danger of premature legislation. You cut off potential research and solutions to the very problems you are trying to solve.

This is worth some reflection. The aim is to increase competition. That is admirable, but despite decades of legislation there is no evidence that restricting innovation in US companies has actually led to home-grown solutions and companies. Note that this also means that any company using multimodal Llama and Apple Intelligence cannot be sold in the EU. It has a ripple effect that affects consumers and companies within the EU. The overall dampening effect of the legislation seems to do as much harm to Europe and the US. Meanwhile the Chinese make hay.

The threat for AI organisations, both public and private, is massive fines if they do not comply by August 2026 – the problem is, compliance to what? The legislation, in terms of huge putative fines, is simply too odd and vague for these companies to take the risk. 

X

Musk is suing the EU over their claim that the X blue tick breaches EU rules. He claims to have evidence that they asked for a secret deal. It's all very messy.

Conclusion

The EU AI Act was ratified by the European Parliament in March 2024 but a shadow lies over the whole processas it needs a ton of guidelines. Legislation like this is necessarily vague. On top of this there is a raft of legislation beyond this that is similarly vague. This shadow will dampen researcha dne commercial activity. I know this, as I get a ton of questions on this on almost every project.

As European influence diminishes in the world (it is only 5.8% of world’s population), the EU seems to be shooting itself in both feet, repeatedly. Rather than encourage research and the practical application of this technology to increase productivity and growth, it seems to want to play cops and robbers.

All of this is BEFORE the awful AI AU Act is enforced.


Thursday, July 11, 2024

Good discussion paper on the Role and Expertise of AI Ethicists: Bottom line – it’s a mess!

 Good discussion paper on the Role and Expertise of AI Ethicists

Who is an AI Ethicist? An Empirical Study of Expertise, Skills, and Profiles to Build a Competency Framework Mariangela Zoe Cocchiaro et al.

Bottom line – it’s a mess! 

In the less than 2 years, AI Ethicists have become common. You see it in social media profiles, speaker profiles, especially in academia. Where did they all come from, what is their role and what is their actual expertise?

Few studies have looked at what skills and knowledge these professionals need. This article aims to fill that gap by discussing the specific moral expertise of AI Ethicists, comparing them to Health Care Ethics Consultants (HC Ethicists) in clinical settings. As the paper shows, this isn’t very clear, leading to vastly different interpretations of the role.

It’s a mess! A ton of varied positions that lack consensus on professional identity and roles, a lack of experience in the relevant areas of expertise, especially technical, lack of experience in real-world applications and projects and a lack of established practical norms, standards and best practices. 

As people who have as their primary role the bridging the gap between ethical frameworks and real-world AI applications, relevant expertise, experience, skills and objectivity are required. The danger is that they remain too theoretical and can be bottlenecks if they do not have the background to deliver objective and practical advice. There is a real problem of shallow and missing expertise along with the ability to deliver practical outcomes and credibility. 

Problem with the paper

The paper focus on job roles, as advertised, but misses the mass of self-proclaimed, internally appointed and simply identified as doing the role without much in the way of competence-based selection. Another feature of the debate is the common appearance of ‘activists’ within the field, with very strong political views. They are often expressing their own political beefs, as opposed to paying attention to the law and reasonable stances on ethics – I call this moralising, not ethics.

However, it’s a start. To understand what AI Ethicists do, they looked at LinkedIn profiles to see how many people in Europe identify as AI Ethicists. They also reviewed job postings to figure out the main responsibilities and skills needed, using the expertise of HC Ethicists as a reference to propose a framework for AI Ethicists. Core tasks for AI Ethicists were also identified.

Ten key knowledge areas

Ten key knowledge areas were outlined, such as moral reasoning, understanding AI systems, knowing legal regulations, and teaching.

K-1 Moral reasoning and ethical theory  

● Consequentialist and non-consequentialist approaches (e.g., utilitarian, deontological approaches, natural law, communitarian, and rights theories). 

● Virtue and feminist approaches. 

● Principle-based reasoning and case-based approaches. 

● Related theories of justice. 

● Non-Western theories (Ubuntu, Buddhism, etc.). 

K-2 Common issues and concepts from AI Ethics 

● Familiarity with applied ethics (such as business ethics, ecology, medical ethics and so on).

● Familiarity with ethical frameworks, guidelines, and principles in AI, such as beneficence, non-maleficence, autonomy, justice and explicability (Floridi & Cowls, 2019). 

K-3 Companies and business’s structure and organisation 

● Wide understanding of the internal structure, processes, systems, and dynamics of companies and businesses operating in the private and public sectors. 

K-4 Local organisation (the one advised by the AI Ethicist) 

● Terms of reference. 

● Structure, including departmental, organisational, governance and committee structure.  

● Decision-making processes or framework. 

● Range of services.  

● AI Ethics’ resources include how the AI Ethics work is financed and the working relationship between the AI Ethics service and other departments, particularly legal counsel, risk management, and development.  

● Knowledge of how to locate specific types of information. 

K-5 AI Systems  

● Wide understanding of AI+ML technology’s current state and future directions: Theory of ML (such as causality and ethical algorithms) OR of mathematics on social dynamics, behavioural economics, and game theory 

● Good understanding of other advanced digital technologies such as IoT, DLT, and Immersive.  

● Understanding of Language Models – e.g., LLMs – and multi-modal models. 

● Understanding of global markets and the impact of AI worldwide.  Employer’s policies  

● Technical awareness of AI/ML technologies (such as the ability to read code rather than write it). 

● Familiarity with statistical measures of fairness and their relationship with sociotechnical concerns.  

K-6 Employer’s policies 

● Informed consent. 

K-7 Beliefs and perspectives of the stakeholders 

● Understanding of societal and cultural contexts and values.  

● Familiarity with stakeholders’ needs, values, and priorities.  

● Familiarity with stakeholders’ important beliefs and perspectives.  

● Resource persons for understanding and interpreting cultural communities.

K-8 Relevant codes of ethics, professional conduct, and best practices  

● Existing codes of ethics and policies from relevant professional organisations (e.g. game developers, software developers, and so on), if any.

● Employer’s code of professional conduct (if available).

● Industry best practices in data management, privacy, and security. 

K-9 Relevant AI and Data Laws 

● Data protection laws such as GDPR, The Data Protection Act and so on. 

● Privacy standards.  

● Relevant domestic and global regulation and policy developments such as ISO 31000 on risk.  

● AI standards, regulations, and guidelines from all over the world.  

● Policy-making process (e.g., EU laws governance and enforcement). 

K-10 Pedagogy  

● Familiarity with learning theories.  

● Familiarity with various teaching methods. 

Five major problems

They rightly argue that AI Ethicists should be recognized as experts who can bridge ethical principles with practical applications in AI development and deployment. Unfortunately this is rare on the ground. It is a confusing field with a lots of thinly qualified, low level commentators self-appointing themselves as ethicists. 

  1. Some, but few in my experience, have any deep understanding of moral reasoning and ethical theories or applied ethics. 
  2. As for business or organisational experience few seem to have been in any real positions relevant to this role within working structures. 
  3. Another often catastrophic failing is the sometimes the lack of awareness of what AI/ML technology is, along with the technical and statistical aspects of fairness and bias.
  4. A limited knowledge even of GDPR is often apparent and the various international dimensions to the law and regulations.
  5. As for pedagogy and teaching – mmmm.

Conclusion

To be fair much of this is new but as the paper righty says, we need to stop people simply stating they are ethicists without the necessary qualifications, expertise and experience of the practical side of the role. AI Ethicists are crucial for ensuring the ethical development and use of AI technologies. They need a mix of practical moral expertise, real competences in the technology, a deep knowledge of the laws and regulations, and the ability to educate others to navigate the complex ethical issues in AI. At the moment the cacophony of moralising activists need to give way and let the professionals take the roles. Establishing clear competencies and professional support structures is essential for the growth and recognition of this new profession. 


My favourite AI quote....

This is my favourite AI quote, by E.O Wilson:

The real problem of humanity is the following: We have Paleolithic emotions, medieval institutions and godlike technology. And it is terrifically dangerous, and it is now approaching a point of crisis overall.” 

There’s a lot to unpick in this pithy and brilliant set of related statements.

By framing the problem in terms our evolutionary, Paleolithic legacy, as evolved, emotional beings, he recognises that we are limited in our capabilities, cognitively capped. Far from being exceptional, almost all that we do is being done, or will be done, by technology. This we refuse to accept, even after Copernicus and Darwin, as we are attached to emotional thought, beliefs rather than knowledge, emotional concepts such a soul, Romanticism around creativity and so on. We are incapable of looking beyond our vanity, have some humility and get over ourselves.

To moderate our individualism, we created Medieval institutions to dampen our human folly. It was thought that the wisdom, not of the crowd, but of middle managers and technocrats, would dampen our forays into emotional extremes. Yet these institutions have become fossilised, full of bottlenecks and groupthink that are often centuries old, incapable of navigating us forward into the future. Often they are more about protecting those within the institutions themselves than serving their members, citizens, students or customers. We lack trust in global institutions, political institutions, educational institutions, businesses and so on, as we see how self-serving they often become.

When Godlike (great word) technology comes along, and threatens either ourselves or our institutions, we often react with a defensive, siege mentality. Generative AI in Higher education is seen as an assault on academic integrity, using generative tools an attack on writing, getting help leading to us becoming more stupid. All sense of proportion is lost through exaggeration and one-sided moralising. No high-horse is too high for saddling up and riding into the discussion.

Wilson’s final point is that this produces an overall crisis. With Copernicus it led to printing, the Reformation, Enlightenment and Scientific Revolution. With Darwin, Church authority evaporated. With the Godlike technology AI, we have created our own small Gods Having created small Gods, they help us see what and who we are. It is seen as an existential crisis by some, a crisis of meaning by others. At the economic level a crisis of tech greed, unemployment and inequality. 

But it’s at the personal level where the Paleolithic emotions are more active. As Hume rightly saw, our moral judgements are primarily emotional. That is why many, especially those who work in institutions express their discontent so loudly. Technology has already immiserated blue collar workers, with the help of white collar institutions such as business schools. It is now feeling their collar. AI is coming for the white collar folks who work in these institutions. It is actually ‘collar blind’ but will hit them the hardest. 


Wednesday, July 10, 2024

Lascaux: archaeology of the mind - a LIM (Large Image Model) and a place of teaching and learning


Having written about this in several books, thrilling to finally get to Lascaux and experience the beauty of this early spectacular form of expression by our species. The Neanderthals had gone and this was the early flowering of the early Homo sapiens.

This is archaeology of the mind, as these images unlock our cognitive development to show that over tens of thousands of years we represented our world, not through writing but visual imagery, with startlingly accurate and relevant images of the world we lived in – in this case of large animals – both predator and prey – of life and death.

As hunters and gatherers we had no settled life. We had to survive in a cold Ice Age climate when there were no farms, crops and storage, only places small groups would return to after long nomadic journeys on the hunt. It was here they would converge in larger groups to affirm their humanity and, above all, share, teach and learn.

Cave as curriculum

After a period of disbelief, where it was thought such images were fraudulent and could never have been created by hunter gatherers tens of thousands of years ago, we had lots of perspectives; from Victorian romanticism of cave 'art' through to shamanic drug induced experiences, finally moving towards more practical, didactic interpretations.

The didactic explanations seem right to me and Lascaux is the perfect example. It was much used, purposeful and structured. A large antechamber and learning journeys down small branched passageways show narrative progression. like early churches, it is packed with imagery. Movement is often suggested in the position of the legs and heads, perspective is sometimes astounding and the 3D rock surface is used to create a sculptural effect. You feel a sense of awe, amazement and sheer admiration.

Narratives are everywhere in these exquisite paintings. Working from memory they created flawless paintings of animals standing, running, butting and behaving as they did in the wild. They dance across the white calcite surface but one thing above all astounded me - they made no mistakes. They used reindeer fat and Juniper which does not produce sooty smoke to light the cave, also scaffolding and a palette of black (manganese) and a range of ochres from yellow to orange and red. Flints scored the shapes, fingers palms, ochre pencils, straws, spitting techniques and stencils were used to shape, outline and give life to these magnificent beasts.

Learning journey

Entering the large rotunda with a swirl of huge bulls, horses and stags, you get a sense of the intensity of the experience, the proximity to these animals, their size and movement. But you are attracted by the scary dark hole existence of two other exits

The first has a foreboding warning – the image of a bear with his claws visible just next to the entrance. One can imagine the warning given by the teacher. Then into the hole of darkness of what is now called the Sistine Chapel of cave painting, a more constricted passage, just enough in places for one person to pass, with images much closer to you. At the end, a masterful falling horse in a pillar of rock, which you have to squeeze around, then into an even more constricted long passage with predatory lions. The narrative moves from observing animals in the wild to their death and finally to the possibility of your death from predators.

Choose the other side passage and you get a low crouching passage, at one point there is a round room, full of images, and at the back after a climb, a steep drop into a hidden space where a dead man (only figure in entire cave) lies prone, the charging bison’s head low and angry, its intestines hanging out. Beside the man lies a spear thrower and the spear is shown across the bison’s body. Beside him a curious bird on a stick.

What is curious are the dozens of intentional signs, clearly meaningful, often interpreted as the seasons, numbers of animals in a group and so on. It isprproto-writing and they have a teaching and learning purpose.

The cave is a structured curriculum, an ordered series of events to be experienced, gradually revealed and explained to the observer in a dark, flickering, dangerous world.

Setting the scene

Let's go back to the cave opening again. As you enter there is a strange, hybrid creature, that seems to say What is this? What animal could it be? The point may have been that we see animals as first glimpses, often at a distance and must learn to identify what they are – predator or prey? It has circular markings on its body, long straight horns and what looks like a pregnant belly. This seems like the spot an experienced hunter would explain that variability in markings, horns, body shape and knowing colour and breeding seasons matters to the hunter.

Expertise was rare, as people died young. The known had to be passed down generations not just by speech and action but permanently as images that told stories. This was a way of preserving thatvrare commodity - cultural capital.

Basic skills

As you enter the huge ante-chamber, which could have held the entire hunter and gatherer group, you literally walk in and find yourself beneath a huge herd of animals. It would have been a surprise, not possible in the real world, a simulation. This is an introduction to many different species.

It has been carefully composed. You are in a representation of the real world, a simulation that focuses on what matters, what you must avoid as predators, and kill as prey. It needed a huge communal effort, as scaffolding had to be manufactured and built, materials gathered and skilled artists themselves trained and selected. This is an organised group, creating an organised venue for organised learning.

The effect of large animals coming at you out of the dark, within the confines of a cold cave would have been terrifying, like being a horror movie. The flickering lamps revealing a horned head here, a tail there. It is as if they understood the idea of attention, simplicity of image and their impact on learning and memory.

Hunting

As hunters late palaeolithic people tended to hunt a specific species at any one time of the year. This matches the imagery, where one can stop at an image of one species (they had to enter difficult passages with small lamps) and move from one species to another sequentially. There are narrative structures within the images; breeding pairs, animals in motion, different seasonal coats. At the end you encounter a masterpiece – the falling horse, with a bloated stomach, dead,

Break-outs

In another long side cave, like a long break-out room, the images are entirely different, a bewildering set of outlines and scores that suggest a more direct telling of what it is to hunt. Like a huge blackboard, they have been drawn and overdrawn by what seems like more improvisational hands. Here, I think, they were explaining the details of hunting. This was the chalkboard lecture hall. It is low and requires one to crouch, more likely to sit and be taught. 

New teachers clearly overwrote on those that came before, as there were no board cleaners! There a huge range of animals in these drawings - horses, bison, aurochs (bulls), ibexes, deer, a wolf and lion. They are often partial images, especially heads, which suggests some specific points were being made about what you need to look for as a hunter. It is a series of drawings over-writing the earlier work, over a long period by different people. 

In this area there is a shaft, climb down and there is a black scene of a human figure lying prone beneath a wounded bison, its intestines hanging out, its head low as it charges. This is the only image of a person in the whole cave. Flint knapping debris and ochre covered flints were found here, indicating the teaching of tools for butchering. One can imagine this being a specific, final lesson – kill or be killed.

Sound and speech

What is missing are the sounds of the teachers and learners. But even here we have some clues. One image, called the Roaring Stag is prominent. I have heard this in the Highlands of Scotland while camping in Winter. The noise is incredible like wolves all around you. It is likely that these sounds would have been simulated in the cave, an intense and frightening amplifier. You can imagine people in the dark suddenly frightened by the sound of rutting stags.

Communal knowledge

I wrote about this in my books on AI and 3D mixed reality, as they tell us something quite profound, that learning, for millions of years, was visual. We were shown things. This is our primary sense and as learning was about the world we knew and skills we had to master, images were our content. But we also have meaningful symbolic communications - not yet rwiting as we know it but an account of sorts and a sense of number.

Additionally, this was the first example of a communally shared learning experience. What we learnt and knew was not owned by others. It was a shared dataset, brought together by the whole group, to be shared for mutual benefit. It took a huge communal effort to create the first LIM (Large Image Model). There were no arguments about who drew or owned what, no ethical concerns about the dangers of sharing our data, just the need to share to survive and thrive.

Conclusion

Altamira was my first illustrated cave, many years ago. I can still remember the shock of that experience, a visceral surprise. Lascaux is even more wondrous. These places reveal the awakening our species - Homo sapiens, the ‘knowing man’. When we began to teach and learn, preserving and passing on our cultural knowledge. We became smarter and more cunning. The Neanderthals, who dabbled in such cave representations, were already long gone. We had separated the known from the knower, so that it could be passed on to many others. We were on the path to externalising ideas, refining them, reflecting upon them and using this knowledge to create new things, moving from tools to technologies; farming, writing, printing, the internet and AI. We became Homo technus.


Sunday, July 07, 2024

Labour, growth, productivity and AI

Labour Party Manifesto was led by a single goal – GROWTH. 

Problem – easy to promise, hard promise to keep.

Yet there are two letters that got zero mention by any party in this election – AI. The evidence is clear, that significant productivity gains can be achieved using this tech. You don’t need AGI to reap the benefits, just focus on straight utility, making things faster, better and cheaper.

Despite the global shift to a digitised economy, productivity growth has been declining since the 2000s. Artificial Intelligence will drive a productivity surge that supports long-term economic growth. Excitement is growing over "generative" Artificial Intelligence, which utilizes advanced computer models to create high-quality text, images, and other content based on their training data. Many are speculating whether this technology could revolutionise labour productivity, similar to the impact of the steam engine, electricity, and the personal computer. One is Tony Blair who has said this should be the first thing a Labour government looks at.

The positive effects on productivity have always lagged behind the invention of new technologies. However, AI seems to have had immediate effects due to its rapid adoption, which is down to its:

Fremium model

Ease of use

Cross sector application

Clear efficacy

2024 is the year of broad adoption, with real productivity gains in organisation-specific tasks. Whatever the process, AI seems to significantly shorten and also increase quality of output.

Initial studies show significant product gains, as well as increases in quality, even in creative tasks. In fact, it is the sheer variety of tasks that matters.

A further advantage is as a solution to recruitment needs. In increasing productivity, less staff are needed for the same amount of work, hence growth.

We are not in the EU therefore not subject to the EU AI Act. In being more aligned with the US and having English as our language we have an advantage within Europe.  

Policies

In schools we need to encourage the use of this technology to increase:

1. Teacher productivity through automated production, teaching & marking

2. Learner productivity through the adoption of these tools in the learning process

3. Admin productivity

4. AI in all teacher training

5. Reduction in curriculum content

In tertiary education, where we have Jaqui Smith as the skills, further and higher Minister, we need:

More short transition courses

Lecturer productivity

Leaner productivity

Focussed, robust AI research

In business we need to incentivise and support the creation and growth of startups. But for startups to truly thrive, several conditions must be met:

Efficient and high-speed dissemination of technical information, including:

Open-access technical publication

Open-source software

Data available for research

Smooth movement of personnel between companies, industry and academia

Smaller equity slices by Universities

Easier access to venture capital

Application of AI in SMEs

In large organisations we need:

Good measurement and studies in productivity

Cases studies of successful implementations

Implementation across the public sector

Tax breaks for AI in companies exporting

Implementation

Above all we need to stop the doom mongering. It is not that AI practitioners are exaggerating its capabilities, that mostly comes from straw manning by the media, a few well known doomsters and speculative ethicists. These people will deny most ordinary people growth and prosperity while they sit on good institutional salaries and pensions.

We have the additional problem of politicians not being tech savvy, few having technical or scientific degrees, and many grazing on media reports and old-school advisors of the same ilk.

What we need from National Government are policies that accelerate this competitive advantage. NOT create more institutes. Enough already with the quangos like Digital Catapult. They do not help. Ignore the trade quangos who write anodyne reports written by low level experts. Avoid the consultancy trough frameworks, reports and documents.

Conclusion

There is one man who could make a big difference here - Sir Patrick Vallance. He has a strong research background and understands the impact it will have economically. We have an opportunity here to not get entangled with the negativity of the EU and forge ahead with an economic growth model based on increased productivity and new jobs. The other Ministers have shown little in the way of innovation in policy. He has the chops to get at least some of this done.

 

Tuesday, July 02, 2024

Mary Meaker's tech report 'AI & Universities' is right but delusional

Mary Meaker’s tech reports were always received with a degree of reverence. Her technology  PPTs were data rich, sharp, clear and were seen as a springboard for action. This latest report is the opposite. Data-rich but delusional. 

Curiously, her data is right but her focus on Higher Education as the driver of growth and social change is ridiculously utopian. The institutional nature of Higher Education is so deeply rooted in the campus model, high costs and an almost instinctive hatred of technology, that it cannot deliver what she advises. She is therefore both right and wrong. Right in her recommendations, wrong in her prediction. Higher Education’s reaction to the AI storm was to focus largely on plagiarism, with a pile of negativity expressed thinly veiled as ‘ethics’.

The research base has shifted out of Universities into companies that have market capitalisations in their trillions, huge cash reserves, oodles of talent and the ability to deliver. Higher education at the elite end has huge endowments but remains inefficient, expensive, has a crisis of relevance and is wedded to a model that cannot deliver on her recommendations.

Where she is right in is pointing towards the US advantage, set by Vannervmar Bush in the 1940s, where industry, Higher Education and Government were seen as a combined engine for growth and chance. 

Vannervar Bush's Vision

Vannervar Bush (1890 - 1974) was the Dean of the School of Engineering at MIT, a founder of Raytheon and the top administrator for the US during World War II. He widened research to include partnerships between government, the private sector and universities, a model that survives to this day in the US. He claimed that his leadership qualities came from his family who were sea captains and whalers. He was also a practical man with inventions and dozens of patents to his name. In addition to his Differential Analyzer, he was an administrator and visionary who not only created the environment for much of US technological development during and after World War II leading to the internet but also gave us a powerful and influential vision for what became the World Wide Web.

When World War II came along he headed up Roosevelt’s National Defense Research Committee and oversaw The Manhattan Project among many others. Basic science, especially physics, he saw as the bedrock of innovation. It was technological innovation, he thought, that led to better work conditions and more “study, for learning how to live without the deadening drudgery which has been the burden for the common man for past ages”. His post war report saw the founding of the National Science Foundation, and Bush’s triad model of government, private sector and Universities became the powerhouse for America’s post war technological success. Research centres such as Bell labs, RAND Corporation, SRI and Xerox PARC were bountiful in their innovation, and all contributed to that one huge invention - the internet.

Bush was fascinated with the concept of augmented memory and in his wonderful 1945 article As We May Think, described the idea of a ‘Memex’. It was a vision he came back to time and time again; the storage of books, records and communications, an immense augmentation of human memory that could be accessed quickly and flexibly - basically the internet and world wide web.

Fundamental to his vision was the associative trail, to create new trails of content by linking them together in chained sequences of events, with personal contributions as side trails. Here we have the concept of hyperlinking and personal communications. This he saw as mimicking the associate nature of the human brain. He saw users calling up this indexed, motherlode of augmenting knowledge with just a few keystrokes. A process that would accelerate progress in research and science.

More than this he realised that users would be able to personally create and add knowledge and resources to the system, such as text, comments and photos, linked to main trails or in personal side trails - thus predicting concepts such as social media. He was quite precise about creating, say a personal article, sharing it and linking it to other articles, anticipating blogging. The idea of creating, connecting, annotating and sharing knowledge, on an encyclopedic scale anticipated Wikipedia and other knowledge bases. Lawyers, Doctors, Historians and other professionals would have access to the knowledge they needed to do their jobs more effectively. 

In a book published 22 years later, Science Is Not Enough (1967), he relished the idea that recent technological advances in electronics, such as photocells, transistors, magnetic tape, solid-state circuits and cathode ray tubes have brought his vision closer to reality. He saw in erasable, magnetic tape the possibility of erasure and correction, namely editing, as an important feature of his system of augmentation. Even more remarkable was his prophetic ideas around voice control and user-generated content, anticipating the personal assistants so familiar to us today. He even anticipated the automatic creation of trails, anticipating that AI and machine learning may also play a part in our interaction with such knowledge-bases.

What is astonishing is the range and depth of his vision, coupled with a realistic vision on how technology could be combined with knowledge to accelerate progress, all in the service of the creative brain. It was an astounding thought experiment.

AI and growth

AI is now fulfilling Bush’s vision in moving our relationship with knowledge beyond search into dialogue, multimodal capabilities and accelerated teaching and learning, along with the very real implementation of the extended mind.

But the power has shifted out of the University system into commerce. Higher Education has retreated from that role and the research entities such as Bell Labs and Xerox Parc are no longer relevant. She is right in seeing the US power ahead of everyone on AI and productivity. The danger is that this produces an even lazier Higher Education sector that doesn’t adapt but become even more of an expensive rite of passage for the rich. 

 

Thursday, June 27, 2024

Is the Dispatches C4 programme a completely faked experiment?

Dispatches on Channel 4. 12 households, supposedly undecided voters, were put to the test and force-fed AI deepfakes. Split into two groups, At the end they go through a mock election.

The programme lays its stall out from the start – they’re out to conform what they already believe. This is not research, it’s TV’s odd version of confirmation bias. The three experts are the usual suspects and, guess who, the famously fragile interviewer Cathy Newman. They are actually zealots for deepfake danger – they are willing this to happen. A dead-giveaway are the past examples of deepfakes they pull up – all of left-wing folk – Hillary Clinton and Sadiq Khan. One of the three experts is, of course, a Labour Party Comms expert!

Here’s the rub. Fed a limited diet of information that massively increases the ratio of fake to real news renders the whole experiment useless. The presence Channel 4, of camera crews, lighting as they are set up to watch the fakes adds to the confirmation that this is authoritative content. They are deliberately set up to receive this ‘leaked’ news. It completely destroys any notion of ecological neutrality in the experiment. In fact, you’re almost forcing htem into believing what you’re telling them. The programme actually becomes a case study in bad research and investigation – it actually becomes fake new in itself, a ridiculously biased experience masquerading as supposedly authoritative journalism.

Actual research

Hugo Mercier’s book Not Born Yesterday debunks the foundations of this moral panic about deepfakes, with research that shows it is marginal at the edges, they are quickly debunked and few actually believe them. He argues that humans are not as gullible as often portrayed. Instead, they are selective about the information they believe and share and explores how social dynamics and evolutionary history have shaped human reasoning and belief systems, making us more resistant to deception than commonly assumed. Most of us weren’t born yesterday. Language didn’t evolve to be immediately believed as true.

Brendan Nyhan a world class researcher, who has extensively studied the impact and implications of deepfakes for many years, is clear. His research focuses on the potential threats posed by deepfakes to democratic processes and public trust. Nyhan argues that while deepfakes represent a significant technological advancement, their real-world impact on public perception and misinformation might be more limited than often suggested. He emphasises that the most concerning scenarios, where deepfakes could substantially alter public opinion or significantly disrupt political processes, are less common and that the actual use and effectiveness of deepfakes in altering mass perceptions have been relatively limited so far.

Deepfakes touch a nerve

They are easy to latch on to as an issue of ethical concern. Yet despite the technology being around for many years, there has been no deepfake apocalypse. The surprising thing about deepfakes is that there are so few of them. That is not to say it cannot happen. But it is an issue that demands some cool thinking.

Deepfakes have been around for a long time. Roman emperors sometimes had their predecessors' portraits altered to resemble themselves, thereby rewriting history to suit their narrative or to claim a lineage. Fakes in print and photography have been around as long as those media have existed.

In my own field, learning, a huge number have for decades, Dale’s Cone is entirely made up, based on a fake citation, fake numbers put on a fake pyramid. Yet I have seen a Vice Principal of a University and no end of Keynotes at conferences and educationalist use it in their presentations. I have written about such fakery for years and a lesson I learnt a long time ago was that we tend to ignore deepfakes when they suit our own agendas. No one complained when a flood of naked Trump images flooded the web, but if it’s from the Trump camp, people go apeshit. In other words, the debate often tends to be partisan.

When did recent AI deepfake anxiety start?

Deepfakes, as they're understood today, refer specifically to media that's been altered or created using deep learning, a subset of artificial intelligence (AI) technology.

The more recent worries about AI creating deepfakes have been around since 2017 when ‘deepfake’ (portmanteau of deep learning & fake) was used to create images and videos. It was on Reddit that a user called ‘Deepfake’ starting positing videos in 2017 of videos with celebrities superimposed on other bodies.

Since then, the technology has advanced rapidly, leading to more realistic deepfakes that are increasingly difficult to detect. This has raised significant ethical, legal, and social concerns regarding privacy, consent, misinformation, and the potential for exploitation. Yet there is little evidence that they are having any effect of either beliefs or elections.

Deliberate deepfakes

The first widely known instance of a political AI deepfake surfaced in April 2018. This was a video of former U.S. President Barack Obama, made by Jordan Peele in collaboration with BuzzFeed and the director’s production company, Monkeypaw Productions. In the video, Obama appears to say a series of controversial statements. However, it was actually Jordan Peele's voice, an impressionist and comedian, using AI technology to manipulate Obama's lip movements to match his speech. We also readily forget that it was Obama who pioneered the harvesting of social media data to target voters with political messaging.

The Obama video was actually created as a public service announcement to raise awareness about the potential misuse of deepfake technology in spreading misinformation and the importance of media literacy. It wasn't intended to deceive but rather to educate the public about the capabilities and potential dangers of deepfake technology, especially concerning its use in politics and media.

In 2019, artists created deepfake videos of UK politicians including Boris Johnson and Jeremy Corbyn, in which they appeared to endorse each other for Prime Minister. These videos were made to raise awareness about the threat of deepfakes in elections and politics

In 2020, the most notable deepfake video of Belgian Prime Minister Sophie Wilm├Ęs showed her give a speech where she linked COVID-19 to environmental damage and the need to take action on climate change. This video was actually created by an environmental organization to raise awareness about climate change.

In other words, many of the most notable deepfakes have been for awareness, satire, or educational purposes.

Debunked deepfakes

Most deepfakes are quickly debunked. In 2022, during the Russia-Ukraine conflict, a deepfake video of Ukrainian President Volodymyr Zelensky was circulated. In the video, he appeared to be making a statement asking Ukrainian soldiers to lay down their arms. Deepfakes, like this, are usually quickly identified and debunked, but it shows how harmful misinformation during sensitive times like a military conflict, can be dangerous.

More recent images of Donald Trump were explicitly stated to be deepfakes by their author. They had missing fingers, odd teeth, a long upside down nail on his hand and weird words on hats and clothes, so quickly identified. At the moment they are easy to detect and debunk. That won’t always be the case, which brings us to detection.

Deepfake detection

As AI develops, deepfake production becomes more possible but so do advances in AI and digital forensics for detection. You can train models to tell the difference by analysing facial expressions, eye movement, lip sync and overall facial consistency. There are subtleties in facial movements and expressions, blood vessel giveaways, as well as eye blinking, breathing, blood pulses and other movements that are difficult to replicate in deepfakes. Another is checks for consistency, in lighting, reflections, shadows and backgrounds. Frame by frame checking can also reveal flickers and other signs of fakery. Then there’s audio detection, with a whole rack of its own techniques. On top of all this are forensic checks on the origins, metadata and compression artefacts that can reveal the creation, tampering or its unreliable source. Let’s also remember that humans can also be used to check, as our brains are fine-tuned to find these tell-tale signs, so human moderation still has a role. 

As deepfake technology becomes more sophisticated, the challenge of detecting them increase but these techniques are constantly evolving, and companies often use a combination of methods to improve accuracy and reliability. There is also a lot of sharing of knowledge across companies to keep ahead of the game.

So it is easier to detect deepfakes that many think. There are plenty of tell-tale signs that AI can use to detect, police and prevent them from being shown. These techniques have been honed for years and that is the reason why so few ever actually surface on social media platforms. Facebook, Google, X and others have been working on this for years. 

It is also the case, as Yann Lecun keeps telling us, that deepfakes are largely caught, quickly spotted and eliminated. AI does a good job in policing AI deepfakes. That is why they have not been caught flat-footed on the issue.


This blind trial paper raises some serious questions on assessment

In a rather astonishing blind trial study (markers were unaware) by Scarfe (2024),  they inserted GenAI written submissions into an existing examination system. They covered five undergraduate modules across all years of BSc Psychology at Reading University.

The results were, to my mind, not surprising, nevertheless, quite shocking.

94% AI submissions undetected

AI submission grades on average half grade higher than students

What lessons can we learn from this paper?

First, faculty can’t distinguish AI from student work in exams (94% undetected) and second which is predictable, also that AI outoerformed students, with a half grade higher, again unsurprising, as a much larger study, Ibrahim (2023) Perception, performance, and detectability of conversational artificial intelligence across 32 university courses showed that ChatGPT’s performance was comparable, if not superior, to that of students across 32 University courses. They added that AI-detectors cannot reliably detect ChatGPT’s, as they too often claim that human work is AI generated and the text can be edited to evade detection.

Framing everything as plagiarism?

More importantly, there is an emerging consensus among students to use the tool, while faculty tend to see it only through the lens of and among educators to treat its use as plagiarism. 

The paper positions itself as a ‘Turing test’ case study. In other words, the hypothesis was that GPT-4 exam outputs are largely indistinguishable from human. In fact, on average the AI submission scored higher. They saw this a a study about plagiarism but there are much bigger issues at stake. As long as we frame everything as a plagiarism problem, we will miss the more important, and hard, questions. 

This is becoming untenable.

Even primitive prompts suffice?

As Dominik Lukes at the University of Oxford, an expert in AI, noted about this new study; “The shocking thing is not that AI-generated essays got high grades and were not detected. Because of course, that would be the outcome. The (slight) surprise to me is that it took a minimal prompt to do it.”

We used a standardised prompt to GPT-4 to produce answers for each type of exam. For SAQ exams the prompt was:

Including references to academic literature but not a separate reference section, answer the following question in 160 words: XXX

For essay-based answers the prompt was:

Including references to academic literature but not a separate reference section, write a 2000 word essay answering the following question: XXX

In other word, wit the absolute minimal effort undetectable AI submissions are possible and produce better than student results.

Are the current methods of assessment fit for purpose?

Simply asking for short or extended text answers seems to miss many of the skills that an actual psychologist requires. Text assessment is often the ONLY form of assessment in Higher Education, yet psychology is a subject that deals with a much wider range of skills and modalities.

Can marking be automated?

I also suspect that the marking could be completely automated. The simple fact that a simply prompt to score higher than the average student suggests that the content is easily assessable. Rather than have expensive faculty assess, provide machine assed feedback to faculty for more useful student feedback.

Will this problem only get bigger?

It is clear that a simple prompt with the question suffices to exceed student performance. I would assume that with GPT-5 this will greatly exceed student performance. This leads into a general discussion about whether white collar jobs, in psychology or, more commonly the jobs psychology graduates actually get, require this time, expense (to both state and student). Wouldn’t we bet better focusing on training for more specific roles in healthcare and other field, such as HR and L&D, with modules in psychology?

Should so many doing a Psychology degree? 

Over 140,425 enrolments in psychology in 2022/23. It seems to conform to the gener stereotypical idea that females tend to choose subjects with a more human angle, as opposed to other sciences 77.2% are female and the numbers have grown massively over the years. Relatively few will get jobs in fields directly related to psychology, even indirectly, it is easy to claim that just because you studied psychology you have some actual ability to deal with people better.

The Office for Students (OfS) suggests that a minimum of 60% of graduates enter professional employment or further study. Graduate employability statistics for universities and the government are determined by the Graduate Outcomes survey. The survey measures a graduate's position, whether that's in employment or not, 15 months after finishing university. Yet the proportion of Psychology graduates undertaking further education or finding a professional job is relatively small compared with vocational degrees 15 months after graduation (Woolcock & Ellis, 2021).

Many graduates' career goals are still generic, centring on popular careers such as academia, clinical and forensic Psychology that limit their view of alternative and more easily accessible, but less well paid, jobs (Roscoe & McMahan, 2014). It actually takes years (3-5) for the very persistent, to get a job as a Psychology professional, due to postgraduate training and work experience requirements (Morrison Coulthard, 2017).

AI and white-collar jobs?

If GPT-4 performs at this level in the exams across the undergraduate degree, then it is reasonable, if not likely, top expect the technology to do a passable job now in jobs that require a psychology degree. Do we really need to put hundreds of thousands of young people through a full degree in this subject at such expense and with subsequent student debts when many of the tasks are likely to be seriously affected by AI? Many end up in jobs that do not actually require a Degree, certainly a Psychology degree. It would seem that we are putting more and more people through this subject but for fewer and fewer available jobs, and even then, their target market is likely to shrink. It seems like a mismatch between supply and demand. This is a complex issue but worthy of reflection.

Conclusion

Rather than framing everything as a plagiarism problem, seeing things as a cat and mouse game (where the mice are winning), there needs to be a shift towards better forms of assessment but more than this some serious reflection on why we are teaching and assessing, at great expense, in subjects, that are vocational in nature but have few jobs available..