With AI we are in the most interesting decade in the history of our species. I can think of no better field in which to think, write and work.
Ideas are easy, implementation hard
My first AI-like project was in the early 90s when I designed an intelligent tutoring system to teach interviewing skills. It had sentence construction as input and adaptivity in the sense of harvesting data as the learner used the system. Written in Pascal, it was clever but not yet smart, as the limitations of the hardware, in terms of processing power and memory, were extreme by today’s standards. Much of the effort went into making things work within these brutal constraints. Even then, we had controlled access to video clips (36 mins), thousands of stills and two audio tracks 112 mins) on Laserdiscs, which we used to good effect, simulating full interviews. You could feel the power of potential intelligence in software.
Jump to 2014 and those hardware limitations had gone. You could build an adaptive, personalised system, which we did at CogBooks. I invested personally in this system (twice) and brought investment in. We did oodles of research at Arizona State University and it was sold to University of Cambridge in 2021. It worked. For many years we had also been playing with AI within Learning Pool having bought an AI company. But my real project journey with modern AI started in 2014, when we build Wildfire, using 'entity analysis', open input and the semantic interpretation of open text answers. The whole thing was starting to taker shape.
Jump to November 2022 and things went a little crazy. I have barely been off the road speaking about GenAI in learning, written books on the subject, blogged like crazy, and recorded dozens of podcasts. Far more important, has been the real projects and product we have built for a number of clients and companies. This is the hard part, the really hard part. Ideas are easy, implementation is hard.
Optimal AI project
What makes a successful AI project? What are the factors that make them a success? The good news is that we have just completed a fascinating project in healthcare that had all the hallmarks of the optimal project. This was our experience.
1. Top-down support
The project started with top-down support, a goal and budget. Without top-down support, projects are always at risk of running out of support. That’s why I’m suspicious of Higher Education projects, grant-aided projects, hackathon stuff and so on. I prefer CEO, Senior Management or Entrepreneur driven initiatives, with real budgets. They tend to have push behind them, clear goals and, above all, they tend to be STRATEGIC. Far too many projects are mosquito project that fail because they end when the budget runs out and have no real impact or compelling use. Choose your use case(s) carefully and strategically. We have been through this with large Global companies - a rational approach to use cases and their prioritisation. Interestingly, AI can help.
2. Bottom-up understanding
This project also had a great client, grounded in a real workplace (a large teaching hospital), a clear budget and solid team. We made sure that everyone was on the same page, understanding what this technology was and could do. The two non-technical team members knew their process inside out but here’s where they really scored – they made the effort to understand the technology and did their homework. This meant we could get on with the work and not get bogged down in explaining basic concepts such as how an LLM works, context window and the need for clean data and data management.
Many AI projects flounder when the team has non-technical members that don’t know the technology, namely AI. It is not that they need competence in building AI, just that they need to understand what it is, the fact that it evolves quickly and that its capabilities grow rapidly.
3. Optimal team
The team also had a top-notch AI developer who has been though years of learning projects. This combination was useful, He had already built products in the learning field, understood the language of learning and its goals. The team was just three people. This really matters. Use Occam’s Razor to determine team size – the minimum number of team members to reach your stated goal. Too many AI projects include people with little or no knowledge of the basic technology. They often come with misconceptions about what they think it is and does, along with several myths.
4. Mindset matters
More than knowledge, is mindset. What cripples projects are people within the organisation who act as bottlenecks – sceptics, legal departments who do not understand data issues, old-school learning people who actually don’t like tech and anyone who is straight up sceptical of the power of AI to increase efficacy. Believe me there are plenty of those folk around.
The mindset that leads to success is one that accepts and understands that the technology is probabilistic, data-driven, that functionality will increase during the project and things change very fast. I’d sum this up by saying you need team members who are both willing to learn fast and keep their minds open to rapid change. It also means accepting that most processes are too manual, that bottlenecks are hard to identify and that processes CAN be automated.
5. Agency shift
You also have to let go and see that this technology has ‘agency’ and that you will have to hand agency over to AI. The technology itself will reveal the bottlenecks and insights. Don’t assume you know at the start of the project, they will be revealed if you use the technology well. This is no time for an obsession with fixed Gantt charts and designs that are fossilised then simply executed. It is like ‘agile on steroids’.
6. Manage expectations
7. Push beyond prototype to product
I can’t go into a huge amount of detail about the client but the topic was Surgical Support - a life and death topic, with little room for error. It involved training and taking source material and turning it into usable support (not a course) for hospital staff. Processes were automated, SME (Subject Matter Expert) time reduced, delivery time to launch massively reduced so the team had more time to focus on quality, as opposed to just process and easier to maintain, as just updating the documents means the system is always current. The savings were enormous and increases in quality clear.
This success meant we could call upon the top-down support to push the project beyond prototypes into product with a broader set of goals and more focus on data management. It has given the organisation, management and team the confidence to forge ahead. With massive amounts of time saved and increased efficacy, we saw that success begets success.
Conclusion
If you don't have both TOP-DOWN and BOTTOM-UP support along with a tight team with the right mindset, you will struggle, even fail. This is a radically new and different species of technology, with immense power. It needs careful handling. The small team remained fixed on the strategic goal but was flexible enough to choose the optimal technology. Without all of the above the project would have floundered in no-man's land, with scope creep, longer timescales and the usual drop in momentum, even disappointment.
The project exceeded expectations. How often can you say that about a learning technology project? This was a young team, astounded at what they had done, and this week, when they presented it at a learning conference, their authentic joy when expressing how it went, was truly heartening. “It was crazy!” said one when she describing the first results, then further inroads into automating in minutes, jobs that had traditionally taken them days, weeks even months. Everyone in the room felt the thrill of having achieved something. In 2024 AI has suddenly got very real.
PS
So many commentators and speakers on AI have never actually delivered a project or product. We need far more focus on practitioners who share what they think works and does not work.
No comments:
Post a Comment