Why learning analytics? Dashboard trap, decision making, how to start, false idols...

Why learning analytics?

The problem with Learning Analytics, is that it can be as much of a trap than saviour. While it tops the poll of future trends in online learning it suffers from ‘dashboard dogma’. Many presentations around learning analytic are full of dashboards with no end of pie charts and donuts, but this is the confectionary of data analytics. If your end point is a ‘dashboard(s)’ you’re merely looking through the window of the bakery. I write in detail about this in my new book 'AI for Learning' including types of learning data, LXPs, LRSs and using data to create a learning organisation but here's an introduction to my thoughts on learning analytics.

Trends

Learning Analytics leapt in at Number 1 on Donald Taylor’s International Survey. What’s interesting is the next four positions, as they are all related:

 1 Learning Analytics

 2 Personalised/adaptive learning

 3 Collaborative/social learning

 4 LXP (Learning Experience Platforms)

 5 Artificial Intelligence

I’ve been working solely on this cluster of things for the last six years and want to tease out a simple point – they have to be seen as a whole as they are all intimately related:

 1 fuels 2/3/4/5

 4 is the delivery platform for 1/2/3/5

 5 really matters

Dashboard trap?

Beyond numbers of people taking courses, literally ‘bums on seats’, which is to measure the wrong end of the learner, learning has never been particularly analytic. Few collect detailed data to describe learner behaviour with even basic analysis. Fewer still delve deeply into that data for insights to inform, predict or prescribe future decision making and action. It is not clear that dashboards improve the situation much. It is still a descriptive ‘bums on seats’ approach to data. Visualisation, in itself, means little. One visualises data for a purpose - in order to make a decision. It often masquerades as doing something useful, when all it is actually doing is acting as a cul-de sac.

The dashboard on your car is there primarily so that you can regularly monitor your speed. A secondary and less regular use is to monitor fuel. Even rarer are signals to indicate when temperature rises, oil or tyre pressure needs attention. In fact, most of the data use in a car works invisibly, on your engine, brakes and other systems. In other words, it automates the use of data. As cars have become more sophisticated they take tasks away from you and use data to automate processes. This will reach its zenith with self-driving cars. In learning, no one dies, so we can move towards automation much quicker.

The learning world attracts ‘people’ people, with an interest in the development of others, rather than many from a scientific or analytic background, with an interest in systems and data. This, in a sense, pushes learning professionals away from learning analytics. We must overcome this reliance on qualitative perceptions and judgements, including the old and laboured Kirkpatrick schema, which is statistically naive. 

It is time to move towards a more serious, data-led approach. But relax, this does not mean becoming an expert in data or statistics, as the technology does all of the computation and heavy lifting on the maths and stats. Learning professionals will not be analysts but consumers of data and data-driven automation. The analysis will largely be done for them.

Decision making

In the end this is all about decision making. What decisions are you going to make on the back of insights from your data? Storing data off for future use may not be the best use of data. The least efficient use of data is storing it in pots with dials on the front. Perhaps the best use of data is dynamically, to create courses, provide feedback, automatically deliver adaptive learning. 

The world is becoming more data-driven, organisations more data-driven and even at the level of the individual, personalised service is an expectation. Almost everything you do online is data-driven (search, all social media, Amazon, Netflix etc). Yet learning remains stubbornly resistant. But it is time the learning world responded by being sensitive to this need for data as fuel, and algorithms as the rocket, that will allow us to boldly go to places we have never been before.

Data is everywhere. You are a mass of data points, your face is data for face recognition, your body a mass of data points for healthcare, your behaviours area data points for online organisations, you network online with other people and information which are all data points, your car is a data point for GPS. You are and live in a sea of data. This is also true of learning. What you know, when you learnt things, how well you know things, your performance. Like it or not, you are all masses of data points. This doesn’t diminish your humanity, it informs decision making and can make life easier and more productive.

If you have a LMS (Learning Management System) you will, most likely, have been gathering data under the SCORM specification. Unfortunately, this has an old and severely limited capability, as it focuses on who did what, when and did they complete courses. If you have been using Kirkpatrick, you will most likely have been gathering the wrong data with little analysis. It is time for a rethink.

Moving beyond this specification, xAPI has been defined by the same people who gave us SCORM. This is a new specification more suited to the current landscape of multiple sources for learning and a more dynamic view of how people learn, along with a need for many more types of data than in the past. Similarly with then shift from the LMS to LXP. That is why LXPs and learning ecosystems have appeared. The world has moved on, organisations have moved on, the technology has moved on and so learning professionals should move on.

This means leveraging data to be more focused, efficient and aligned with your organisation’s strategy. It should lead to better decision making, more action, more automation and provable impact on the business – not just dashboards.

How to start?

Don’t think just dashboards. They trap you in a world of reading what IS the case, rather than deciding what SHOULD be the case. We need to derive an OUGHT from an IS and push them beyond dashboards to decisions, actions and the automation of process.

What we need is strategic view of data use. Here’s the good news, a schema has existed for a long time in data science, classifying data use into five areas:

5 Levels of data use in learning: 

 Level 1:  Describe

 Level 2:  Analyse

 Level 3:  Predict

 Level 4:  Prescribe

 Level 5:  Innovate

This allows you to think ambitiously about data, moving beyond mere description (dashboards) towards using to help improve and shape teaching and learning. 

Level 1: Describe

What does learning data tell us about what is happening?

Data that describes what is the case, describing learners, their behaviour and the technology is descriptive. That’s what dashboards do. This is the simple world of tracking and visualisation. Don’t get stuck with dashboards only – they are merely descriptive. 

Level2: Analyse

What does learning data tell us about why it’s happening?

Analysis gives you deeper insights into data, evaluation, business performance, ROI and may, even at a simple level, provide useful insights in terms of informing decisions and action. Don’t worry, the software should do the analysis for you – you don’t have to become a data scientist!

Level 3: Predict

What does learning data tell us that is likely to happen?

Data that predicts what your organisation or group or individual learner performance are likely to be can predict performance, predict dropout and recommend action. Recommendation engines drive most of what you do online (Google, Social Media, Amazon, Netflix). This allows you to deliver personalised learning.

Level 4: Prescribe

What does learning data tell us should happen?

This is where data makes things happen. Nudges and other push techniques can be executed, spaced practice applied, personalised and adaptive learning applied. The software literally uses data to enact something for real. This is how data is actually used in the real world, to automate processes.

Level 5: Innovate

How can learning help us innovate?

Beyond our basic four levels, lies the use of data for more innovative uses in learning such as sentiment analysis, content creation, curation and chatbots. There is a wide array of data-driven techniques that can be used to bring learning into the 21^st century.

False idols

One can decide to let the data simply expose weaknesses in the training. This requires a very different mindset, where the whole point is to expose weaknesses in design and delivery. Is it too long? Do people actually remember what they need to know? Does it transfer? Again, much training will be found wanting. To be honest, I am somewhat doubtful about this. Most training is delivered without much in the way of critical analysis, so it is doubtful that this is going to happen any time soon.

One could also look for learning insights into ‘how’ people learn. I’m even less convinced on this one. Recording what people just ‘do’ is not that revealing if they are clickthrough courses, without much cognitive effort. Just showing them video, animation, text and graphics, no matter how dazzling is almost irrelevant if they have learnt little. This is a classic GIGO problem (Garbage In, Garbage Out). 

Some imagine that insights are buried in there and that they will magically reveal themselves  - think again. If you want insights into how people actually learn, set some time aside and look at the existing research in cognitive science. You’d be far better looking at what the research actually says and redesigning your online learning around that science. Remember that these scientific findings have already gone through a process of controlled studies, with a methodology that statistically attempts to get clean data on specific variables. This is what science does – it’s more than a match for your own harvested data set. 

Business relevance
Learning departments need to align with the business and business outcomes. Looking for correlations between, say increases in sales and completed training, gives us a powerful rational for future strategies in learning. It need not be just sales. Whatever outcomes the organisation has in its strategy needs to be supported by learning and development. This may lift us out of the constraints of Kirkpatrick, cutting to the quick, which is business or organisational impact. We could at last free learning from the shackles of course delivery and deliver what the business really wants.

Another model is to harvest data from training in a diagnostic fashion. To give a real example, they put the employees of a global bank through simulation training on loan risk analysis and found that the problems were not what they had imagined - handing out risky loans. In fact, in certain countries, they were rejecting ‘safe’ loans - being too risk averse. This deep insight into business process and skills weaknesses is invaluable. But you need to run sophisticated training, not clickthrough online learning. It has to expose weaknesses in actual performance.

Business 
You may decide to just get good data and make it available to whoever wants to use it, a sort of open data approach to learning. But be careful. Almost all learning data is messy. It contains a ton of stuff that is just ‘messing about’ – window shopping, In addition to the paucity of data from most learning experiences, much of it is odd data structures, odd formats, encrypted, in different databases, old, even useless. Even if you do manage to get a useful clean data set, You have to go through the process of separating ‘Personal’ from ‘observed’ (what you observe people actually doing), ‘derived’ making deductions from that data, ‘analysed’ (applying analysis to the data). You may have to keep it ‘anonymised’ and the privacy issues may be difficult to manage. Remember, you’ll need real expertise to pull this off and that is in very short supply. A LRS (Learning Record Store), such as Learning Locker, is a good start.

Conclusion

There’s a ton of learning technologists saying their new strategy is data collection in 'learning record stores' and 'learning analytics'. On the whole, this is admirable but the danger is in spending this time and effort without asking ‘Why?’ Everyone’s talking about analytics but few are talking about the actual analysis to show how this will actually help increase the efficacy of the organisation. Some are switched on and know exactly what they want to explore and implement, others are like those that never throw anything out and just fill up their home with stuff – but not sure why. 

Learning analytics is too often seen as 

 Level 1:  Describe - Dashboards

 Level 2:  Analyse - Insights

 Level 3:  Predict - Foresight

 Level 4:  Prescribe - Action

 Level 5:  Innovate – Innovative actions

Of course, all of the above is fine in theory. In practice, organisations have different capabilities. As usual with new paradigms, there is a maturity curve, although that involves a wider set of criteria, including:

 People/culture

 Systems/processes

 Technology/resources

Having spent the last few years doing all of the above, I think we are about to enter a new era, where smarter software (AI/data-driven) will deliver smarter solutions. I now see real clients use data, not just to produce dashboards, but to drive engagement, learner support, content creation, curation, assessment, sentiment analysis, chatbots and so on. My book ‘AI for learning’ is now available on Amazon. Happy to help with any of this stuff… DM me on Twitter or contact me on the form here…

Conferences are going through an extinction-level threat. Online conference need buzz, don't just shove your old conference on to Zoom...

The live conference industry ground to an absolute halt. But like many who experience an extinction-level event, it should lead to some reflection. Unfortunately, it hasn't led to much online innovation. Many online conferences simply present what they did in a building, on a screen. Having presented at hundreds of face-to-face conferences and many online conferences, I’ve been surprised at the lack of fresh thinking.

I’m hearing academics and business leaders reconsider their annual spend. They are shifting that spend to other forms of researching and marketing. Flying large numbers of people all over the world now looks increasingly odd, if not immoral, not only in terms of the virus but also in terms of climate change and efficacy. The sight of a swarm of private jets at Davos disgusted many but that is nothing compared to the day-in, day-out, climate denying activity of the conference business. Let’s not kid ourselves that these are practical venues, the perennial attraction of Vegas, Orlando and Hawaii have always been dubious.

The common denominator to all this is that cramming huge numbers of people into rooms in conference centres has several major problems. First they have to get there, increasingly from all corners of the globe, exacerbating climate change. Secondly, the risk of accelerating contagion in a pandemic. Third, it is not clear that the current model is that useful. There are many other reasons for questioning these old habits, and habits they are. Two examples of bad habits, that always surprise me; first, seeing 'posters' at academic conferences, it always seems so school-like, so adolescent. second, the literal reading of a paper from a lectern. Do we really have to travel thousands of miles to see this stuff?

Perhaps, far too little use is made of online conferences.

So what makes a good online conference? 

We know a lot about what works here... online has, in fact a several advantages - cost, time, more audience participation, links, able to leave presentation easily, post-conference learning and follow up... and so on... but they have to be run differently. Having been involved with a few, here’s my initial thoughts…

1. Needs a compere

A physical conference has the building to hold it together. There is a sense of place and you choose from a schedule, which rooms to go to. Online, I value a MC or compere with the presence, communication skills, organisational skills, often with a touch of charm and humour, to motivate people and provide guidance and help. They need to be comfortable in front of a camera and be concise and clear communicators. Throughout the conference, they can feed back themes that have arisen, marshal views from attendees and stimulate discussion and online participation. A compere  not only hold things together, they keep the show on the road, give a sense of occasion and buzz, signpost forward, keep everyone involved. They also need to be able to cope with things that go wrong…

2. Scheduling

Conferences needed to cram everything into a day or two, as you had to get there, pay for accommodation and so on. When that necessity goes, ry having your even over a week or month. Schedule just one event or theme a day. People are busy and can choose. what they attend. The 1-3 day schedule is an artefact of physical conferences.

4. Use green screen, drop-in graphics and music

This has become a genre in itself but odd backgrounds ring the changes and can be fun. Don't be scared to have a little fun with this. If the speaker is in Brighton, show the beach! PR expert Alex Shapiro uses this all the time. If you are calling. in people from all over the country or world, make that a feature. Get the speakers to say where they are and why they love their home town or living in the country, whatever. Create great images and GIFs to keep the event flowing, just as a TV station does between programmes.  Music can be used to good effect, before the proceedings begin and between speakers. For a big conference how about a house band, like those on TV chat shows.

5. Lightness of online being

Given that communication is at a distance, I like it when the compere introduces some lightness to the proceedings. You can set up little competitions – spot the X, even ask for pics of attendees rooms (seen that work well), ask for jokes... give out prizes. Has the speaker written. book, offer a few free copies as prizes. It gives some social cohesion to the affair. I rather like the idea of making it more like a live TV show… I liked it when a speaker showed her dog on screen. 

6. Interviews

Rather than endless talking heads i.e. close ups of faces on Zoom staring at you out of he screen, try shaking up the format. Interviews work well and the interviewer can drop in questions from the online audience. Watch The Joe Rogan snow to see how this can be done well. To be honest this can be pre-recorded and edited.

7. Participation

Rather than the wooden 'Q and A' at the end of sessions, that often get ditched as the speakers overrun, you can engage before, during and after presentations, with varying levels of participation: formal 'Q and A', chat, moderated questions and so on. This can be a much higher level of participation than a real conference. Speakers can respond with links to relevant material. Moderated questions, I think, work best, even stopping in the middle to take a few. Set a challenge so that people are paying more attention to the speaker.

8. Social events

Conferences are sold on the social networking side but witness the people who sit next to their colleagues in sessions and talk to the people they know and work with during the coffee breaks. During lunch, coffee breaks or with special breakout groups, it is possible to set up discussion groups or let social groups coalesce. These groups, coffee groups, pubs etc can carry on afterwards, as people share social media details. They can be topic based and compered. Alternatively, you can encourage social media activity to get your messages and content out to a huge global audience. Let these groups form. Wenger talks a lot about these 'communities of practice' – they can be encouraged. In my experience anyone who wants social interaction with the speakers and other attendees will be able to do so to a far higher degree than in a live conference.

9. Record everything

Talks and participation are easily recorded for future access. Indeed, the recording becomes trivial and in a format that is not the speaker like a matchstick person at distance on a stage but an intimate close-up with cuts to their slides. Miss a session and it will be available as a recorded event immediately afterwards.

10. Post-conference

I’ve given talks at hundreds of conferences around the world and am often shocked to see that most attendees don’t take notes. They WILL forget, not only what they think they will remember but even what sessions they attended. That’s how the brain works – it’s a forgetting machine. Learning Pool recently ran an online conference where they used their LXP software, integrated with Zoom to hold the conference within a learning environment. This allows follow up and learning from the event to a much higher degree than is possible with physical attendance. I like this idea of turning conferences into richer learning experiences with more follow up. Provide transcripts (alternative to notes), extra links and. resources - curate these from the speakers.

Accept that shit happens - behind scenes producer

In physical conferences, speakers screw up all the time. Presenters that can’t find/operate their PowerPoints, overrun, go back or too far forward on their slides. Online you can have complete centralised control. You can also troubleshoot internally and externally. For example, speakers can be muted, unmuted at a distance, slides ready. Even attendees can get help, the usual problems being audio. A good behind the scenes producer, with technical skills really does help make things flow.

Lower costs on both sides

On costs, both sides save a pile of money. For attendees, no travel, accommodation, subsistence, less opportunity loss. For organisers, no venue, food and less labour costs. It’s a win-win. Conference fees can be minimal or waived, as sponsorship money can pay for the much reduced costs. An interesting model has been tried during this pandemic, an online conference where 50% of the revenue is shared among the speakers.

Conclusion

Sure, some things will be lost, the drunken conversations and late nights in the Hotel bar, the chance to visit some foreign capital. But behaviours will change after this pandemic. People will not rush back to restaurants, cinemas, travel and cruises. On conferences, organisations and individuals will think twice before going back to things that were clearly bad for the planet. The online economy will grow – online learning, online shopping, online payments, online streamed entertainment and online conferences.

Thiel – Critique of Higher Education… it offers the same thing year after year, at a higher and higher price...

Peter Thiel was the co-founder of PayPal, first outside investor in Facebook and has since invested in many companies including LinkedIn and Yammer. He describes himself as a conservative-libertarian and espouses original views on business, social structures and education, that many find, if unpalatable, certainly interesting.

Zero to One

In his book on entrepreneurship, Thiel is critical of those who imagine that entrepreneurship can be taught. This, he thinks, is flawed, as “The paradox of entrepreneurship is that such a formula necessarily cannot exist: because every innovation is new and unique.” He doesn’t think that successful, network businesses can be built by MBA types, who are drilled in seeing what is the case, rather than the all-important absences or gaps. He is also critical of educational systems that drive competition, an obsession with grades, which in turn lead to conformity. The best minds in the world now focus on driving people towards careers in online advertising or the unproductive, and risk averse world ,of law and finance. This he thinks limits, rather than encourages personal ambition and progress.

Critique of Higher Education 

He has likened Higher Education to the Catholic Church on the eve of the Reformation, “you have this priestly class of professors that doesn’t do very much work, people are buying indulgences in the form of amassing enormous debt for the sort of the secular salvation that a diploma represents." He asks young people to think again about the burden of student debt, and sees it as a form of ‘indentured servitude’. Like the Catholic Church, Higher Education turned into a global, institutionalised phenomenon that demanded increasingly large sums of money from people, for an experience that was much the same as it had been for decades. The cost of indulgences, as well as the transfer of productive wealth into the non-productive church, was a major catalyst for the Reformation. People were literally becoming indebted to the level of indenture to the church. This was impoverishing the populace while enriching the institutions. Thiel thinks this has happened in Higher Education.

Salvation

The insidious side of the Catholic Church was the threat, that if you did not pay up, you were damned. This is mirrored by the modern threat that if you do not get a Degree, you’re damned as a failure, sent to some sort of economic hell, never being admitted to the heaven that is gainful employment and wealth. Criticising Higher Education is like “saying there’s no Santa Claus” claims Thiel. This is a feature of all bubbles, believes Thiel, where ‘groupthink’ takes over and false assumptions become absolute beliefs, and even debate of the negative consequences is seen as “party-pooping”.

Conformity

Higher education, he thinks, is a bubble fed by a vague abstraction - the word ‘education’. Is it an investment decision for a good job? Is it mere consumption, a four-year party? Or, as he thinks, an ‘insurance policy’ that is not worth as much as you think it is worth. He charges the system with conformity, a position also taken by Noam Chomsky. Diagnostically, Higher Education suffers, he thinks, from a massive failure of the imagination, a failure to consider alternative futures. The net result is that everyone conforms and marches in lock-step to college to do similar degrees which results in homogenisation and lot less freedom of action, as people believe that everything has been exhausted, and the likes of law and finance are the only possible ways forward. He adds that the lack of focus on teaching has turned the system into an “incredible racket”. 

Higher Education as bubble

Technology does more with less, education does the opposite, it offers the same thing year after year, at a higher and higher price. No one could really claim that the huge hikes in pricing reflect corresponding hikes in the value of University tuition. So what’s happening? Universities are complicit in this. They raise prices because they can, without attention to lowering costs through online learning, fourth semesters etc. In fact the quality of tuition may have fallen, with more students and less qualified lecturers, matched by salary inflation at the top, higher numbers of administrators and wasteful capital expenditure in largely empty buildings. Like the housing market, where people rushed to take out loans (mortgages) based on the belief that the value of their asset will always rise (or at least stay the same), many suffered a shock when the value dropped. Huge hikes in prices for the buyer, now seem unrelated to the real price of the degree. This is exactly what happened in everything from tulips to internet stocks and housing. 

There is, for Thiel, no compelling evidence that the future worth of degrees will be guaranteed. That is the mistake made in all bubbles. In a bubble, real demand is brutal, and in a buyers’ market may lead to degrees being simple indicators of ‘class’ rather than intrinsic value. Universities may be creating their own bubble, dislocating cost from real value. Institutional brand ranking may lead employers to dismiss degrees from institutions perceived as second-rate. In short, your degree may become a liability while your debt remains all too real. In short, HE has all the hallmarks of a bubble. Resistant to influence from the outside it is heading towards a crisis, especially for middle-class students who are amassing enormous amounts of debt. As financial pressure mounts, the Reformation needs to come from the outside.

He sees Covid as having been disruptive but feels the gem is up as they're now an echo chamber. Tracking your entire life towards college seems more contrived than ever. It's burning out kids. 

The Diversity Myth

Long a critic of DEI training, he wrote a book with David Sachs, criticising the myth in the 1990s, he sees the crazed culture wars as endlessly pessimistic and his position as prophetic. For Thiel, diversity of people doesn't necessarily lead to diversity of thought and the words used in DEI are ill-defined. It is often a distraction. Identity can make you unique or different. That's the paradoxical contradiction at the heart of identity politics. 

Thiel fellowship

To action his beliefs, about the inefficiency and often irrelevance of college, Thiel’s Fellowship programme funded $100k to each of 20 people under 20, to create their own companies. The programme challenges the idea that college is the only path for young people.

Criticism

Thiel has backtracked on some of his more extreme positions, such as his attacks on multiculturism and diversity, expressed in the book The Diversity Myth and his Fellowship programme has not been the success he predicted. It has been criticised for replacing education with ‘get rich quick’ programmes. His extreme views on the role of women and their political liberation have been roundly attacked as antediluvian.

Influence

Thiel is a contrarian, and although many of his ideas seem outlandish, his critique of Higher Education articulates a growing dissatisfaction with the status quo. Many see his position as exaggerated but it is cogent and based on his not inconsiderable experience in investments and predicting the future. Whether he is right will be proven by future events.

Bibliography

Thiel, P.A. and Masters, B., 2014. Zero to one: Notes on startups, or how to build the future. Broadway Business.

Sacks, D.O. and Thiel, P.A., 1995. The Diversity Myth." Multiculturalism" and the Politics of Intolerance at Stanford. The Independent Institute, 134 Ninety-Eighth Avenue, Oakland, CA 94603.

Nielsen - When online goes bad... eye-tracking data, bad internal search

Jakob Nielsen, a Dane, has long campaigned for better usability on the internet. A ferocious critic of excessive and self-absorbed web design, especially Flash, and highly critical of designers who see the medium as a mere form of expression, rather than performing real acts of communication and learning, he offers sage advice on best practice is based on actual user responses (thinking aloud and eye-tracking).

Best practice

A key concept for Nielsen is consistency. Users, he claims, crave for consistency. They expect to learn how to use a website or piece of online learning, but don’t expect to worry about the rules changing. The unexpected breaks the user’s confidence in the system making them feel insecure. This is especially destructive in online learning, where the cognitive dissonance disrupts the learning experience. In general, what’s important for Nielsen in screen interfaces is:

Easy to learn

Efficiency

Memorability

Low error rate

Satisfaction

This is why it is important to test, through voiced user trials. Users matter as users are either your customers or learners. Annoy them or switch them off any you switch off revenue or learning.

Eyetracking

As readers scan screen text, far more than they scan written text, Nielsen advises corrective techniques:

·      subheads

·      bulleted lists

·      highlighted keywords

·      short paragraphs 

·      a simple writing style 

·       de-fluffed language devoid of marketese

His later three year, eye-tracking trials confirmed how little text people actually read on websites. Heat maps and gaze plots were used to recommend best practice on page layout, menus, site elements, images and advertising. This was a more objective form of user-watching, and ‘thinking aloud’ which has remained his primary method of testing for usability.

Flash 99% BAD

His famous ‘Flash: 99% Bad’ article characterised Flash as a usability disease. He does not criticise the tool itself, only its tendency to work against usability. Flash makes things unusable for three main reasons.

First, it encourages design abuse through gratuitous animation. Since we can make things move, why not make things move? It’s not that animation has no role to play, only that, on the whole, it’s a distraction. Interestingly, this was backed up in detailed research by Mayer. Animation takes up useful cognitive attention and distracts from learning unless it is relevant and purposeful.

Second, it reduces the granularity of user control, reverting to presentation type sequences. Flash sequences at the start of websites are among the most indulgent and annoying feature of the web. This also annoys users and learners and contributes to users switching off attention.

Third, non-standard interfaces are introduced and not easy to use by users and learners who are used to more common conventions. True and disturbing.

These usability problems are not inherent in Flash and use of this tool has improved over the years. Indeed, he developed usability guidelines for Flash (that were mostly ignored). His position remains as follows, “The problem with most Flash is that it’s irrelevant and gets in the way of users”. The download time is only one of the many problems, and even with instantaneous download, users prefer to visit sites that contain more straightforward content.

Accessibility

Nielsen’s study on Disabled Accessibility: The Pragmatic Approach, showed that accessibility problems should come as no surprise, ‘After all, countless usability studies of websites and intranets have documented severe usability problems, low success rates, and sub-optimal user performance, even when testing users with no disabilities.’ In general, improving accessibility improves usability, which in turn improves performance, leading to cost benefits and savings.

The value of Jakob Nielsen’s prioritised approach is that he undertook real accessibility trials of websites with users with several different types of disabilities on a range of assistive technologies, including a control group. His conclusions could be said to run against the grain, in that he recommends a pragmatic, gradual approach to making existing websites (and online learning) accessible. His advice has largely been ignored by an over-prescriptive approach to accessibility, whereas most have quietly adopted his pragmatic approach.

Criticism

It can be argued that users also want aesthetic and other effects which enhance their experience when using screen-based interfaces. His ‘ideal’ websites and home pages do leave one underwhelmed. So they have a point, especially in learning, where motivation and sustained attention are important. There are many tribes in web and online learning design – usability experts. Like Krug, Norman and Nielsen, learning experts, graphic artists, who treasure their aesthetic and design judgements, coders and the customer, who often wants to impress their bosses (and users) with something that looks, well ‘flash’. Most websites and online learning are therefore compromises.

Yet, his work remains relevant, especially in pointing to the excesses of elaborate design. He’s not arguing for ugly content, only usable content. He has no problem with using readable fonts, especially for longer pieces of text. Few notice that Arial is the default font in Wikipedia but it is, and with good reason. On the whole, readers tend to prefer non-serif fonts like Arial, Verdana or Tahoma for screen text. Nielsen’s point is that, in the end, it’s users that matter and successful businesses, like Google and Amazon, keep things simple.

Bad internal search

He claims that the biggest fault in contemporary web design is bad internal search. Poor headlines and page summaries are another bugbear. He feels that too little of the budget is spent on this feature. I have to agree. What users enter into your search box is perhaps the most important data you can gather. It shows what users, and not designers, really want.

Influence

Nielsen is not afraid to challenge those who see the internet as a medium for designers as opposed to users. His user-centred research confirms, time and time again, that real people want simpler, more consistent and less elaborate models and content. His advice, informed as it is by research, is invaluable for online learning and web designers alike. But we should be cautious about seeing everything solely through the Puritan eyes of the usability expert as there are other qualities that matter in some contexts.

Bibliography

Nielsen J. (1990) Hypertext and Hypermedia (1990)

Nielsen J. (1993) Usability Engineering (1993)

Nielsen J. (1999) Designing Web Usability: The Practice of Simplicity 

Nielsen J. (2001) Homepage Usability: 50 Websites Deconstructed

Nielsen J. (2006) Prioritizing Web Usability

Nielsen J. (2008) Eyetracking Web Usability

Nielsen J. (2012) Mobile Usability

Norman: tech should be 'invisible' in learning…

The space between humans and technology is messy. Donald Norman’s touchstone for successful technology is that it should be invisible, intuitive and so easy to use that we can focus on the real task. Technology must conform to human needs, not the other way around. We must therefore use ‘user-centred design’ to humanise technology. This is true ergonomically but also true of interfaces which should render technology invisible.

Usability and learning

Usability, or user-centric design, is critical in online learning, as the crippling restraints of working memory mean that any cognitive overload or unnecessary cognitive effort on navigation will, by definition, squeeze out or delay learning. I see this time and time again with confusing menus, icons, whizzy graphics and unnecessary clutter. Distractions destroy attention, so confusion in navigation and usability undercuts learning.

Technology first, invention second, needs last

Although Norman is an academic, he believes that technology drives progress in user interfaces and design, providing lots of incremental changes in functionality and usability. He is no believer in ethnographic or academic studies that attempt to find out what people do and want. In this sense he is a follower of Brian Arthur and believes that ‘science, engineering and tinkerers’ produce the real progress. Research should focus on user-centred research on actual devices to make improvements, not try to define the future.

The Psychology of Everyday Things

Norman made his name with The Psychology of Everyday Things, where he takes a wry look at product design in everyday objects such as computers, telephones, car windows, dashboards, doors etc. to show good and bad practice. It’s full of examples explaining why people push when they should pull, click the wrong buttons and generally fail to complete the simplest of everyday tasks with physical and online technology.

Don’t keep user in dark

His advice is straightforward and has plenty of relevance in online learning and web design. His first rule is ‘Design for usability’. Usability, or ease of use, is paramount. Don’t make navigation difficult. Make things visible – don’t keep the user in the dark. A good example of how this goes wrong is the poor use of icons in navigation. Programmes sometimes have graphics that look like icons but are not active, merely illustrative. You click on them and nothing happens. Even worse, you may click on an image or icon and something unexpected happens.

Mapping

Mapping is another of his principles in design. To steer a car you turn the wheel to the right to go right and left to go left. This is mapping. Apply this to navigation on the screen. To go forward the arrow should face to the right and left to go back. Pull fingers a[part on touch screens to enlarge, pull together to reduce in size. In general, in navigation, feedback (another Norman design principle) is also important. You need to know when you’ve arrived at a destination.

Use conventions & coherence

In his later works, such as The Invisible Computer he tackles, not objects, but computer interfaces. How do new users understand what to do? First, follow conventional usage, both in the choice of images and the allowable interactions. Convention can constrain creativity, but on the whole, unless we follow the major conventions, we usually fail. Those who violate conventions, even when they are convinced that their new method is superior, are doomed to fail. You cannot successfully introduce a non-qwerty keyboard today, or reverse the window scroll bar convention. For better or for worse, human culture changes slowly, if at all.

Use words to describe the desired action (e.g. ‘click here’ or use labels in front of perceived objects). Words alone cannot solve the problem, for there still must be some way of knowing what action and where it is to be done. This requires a convention of highlighting, or outlining, or depiction of an actionable object. It is also well known that single word labels fail for most people. Thus, road signs often use graphics - an international standard on road sign graphics exists.

Follow a coherent conceptual model so that once part of the interface is learned, the same principles apply to other parts. Coherent conceptual models are valuable and necessary, but there still remains the bootstrapping problem; how does one learn the model in the first place? Use conventions, words, and metaphors to increase invisibility.

3 forms of emotional design

Do screen projectors, alarm-clock radios, lights in hotel rooms annoy the hell out of you? Have you given up trying to programme your household heating system? Norman sees our emotional responses to design in terms of:

Visceral (appearance)

Behavioural (performance)

Reflective (memories and experience)

Interestingly he thinks Americans value 2 more than 1&3, whereas Europeans, at least the cultural classes, value 1&3. This is fascinating. He claims that different people buy things with different fuel mixtures of the three emotions. Different companies design to different types of emotions. Greta companies deliver all three.

As he explains in Living with Complexity, it is not that technology delivers too much complexity. The fact is, we live in a world of complexity, with complex technologies that do complex things. Live with it – that’s reality. The enemy is not complexity, it is dreadful design. You should not be expected to shake out some salt or pepper on to your hand to determine which cellar contains what. Complexity needs to be tamed, masked or made invisible with good design.

Conclusion

Norman’s books can be a bit trying to read as they jump between different styles and approaches. Nevertheless, they constantly illuminate the design process. A consistent critic of inconsistent and gimmicky web design as well as common mistakes in the design of hardware and interfaces, he was a pioneer in seeing user-centred design as a game changing force, not only in real-world objects, but also on the screen. We are only now starting to see the importance of his advice in online learning and web design with interfaces which are truly invisible in the sense that that they allow learners to learn, avoiding the cognitive effort taken to use a cumbersome interface.

Bibliography

Norman, D. (1986) User Centered System Design 

Norman, D. (1988) The Psychology of Everyday Things

Norman, D. (1992) Turn Signals are the Facial Expressions of Automobiles 

Norman, D. (1993) Things That Make Us Smart

Norman, D. (1994) Defending human attributes in the age of the machine

Norman, D. (1998) The Invisible Computer

Norman, D. (2004) Emotional Design: Why We Love (or Hate) Everyday Things

Norman, D. (2004) Living with Complexity

Norman, D. (2007) The Design of Future Things

Gatto – I quit, I think…

John Taylor Gatto was an award-winning teacher who became a fierce critic of contemporary schooling. As New York City ‘Teacher of the Year’ for three consecutive years, he had been a teacher for 30 years, when he suddenly resigned, in 1991, disillusioned with his profession and the education system in the US.

I Quit, I Think

His resignation letter to the Wall Street Journal attacked the curriculum of class and dependency, where children were locked into a pyramidal sorting structure. Confinement, tests, bells, fixed periods, standardization; all forced teachers to play the role of master to the children as disciples. Parents and families, he thought, were deliberately excluded and treated as adversaries, with children robbed of their time and childhood. To be clear, he was a conservative, libertarian who supports home schooling and open source learning, doesn't believe in state-trained teachers and thinks that the market should be allowed to flourish as the agent of change.

History of US Education

The full title ‘The Underground History of American Education: A Schoolteacher’s Intimate Investigation Into the Problem of Modern Schooling’ is an unflinching history and critique of US schooling. In a somewhat fragmented style, he takes us though the many influences on US education. In particular, however, the Prussian provenance of US schooling is exposed, where the aim was to regiment people into being loyal citizens of the state. Aided by Horace Mann's "Seventh Annual Report" to the Massachusetts State Board of Education in 1843, the Prussian idea of order, a fixed curriculum and certification was firmly established. He fingers James Bryant Conant and Alexander Inglis for imposing the strict subject divisions, age-grading, ranking by constant testing and other subtler strictures. In the late nineteenth and twentieth century the demands for labour encouraged this warehousing into large schools with standardized process and product. The system, he thinks, has become fossilized and unfit for purpose. In his sights are also religion, behaviourism, Taylorism, business, bureaucracies, centralised control, adjunct staff and administrators in schools and many other supposedly false gods. He identifies 22 agencies; Government agencies, special interest groups and the knowledge industry, who control and distort education to their own ends.

Critique of schooling

In 'Dumbing Us Down' he sees both teachers and children as infantilized by the rigid structures of schools and schooling, “virtual factories of childishness” where boredom is the norm. Teachers have to teach everything out of context and age segregation is applied in a way that contradicts the normal mix of ages outside of school. Teachers themselves are victims of the 'staffroom' culture, which he wanted to ban. School, for Gatto, is a sorting mechanism that creates a culture of acceptance, indifference and emotional dependency. You are taught to accept your place in life, your class and fixed destiny; you have no privacy, no place to hide. By being drilled, you become indifferent to many subjects and mentally dependent on teachers. Additionally, the curriculum and rules confuse children, who never really grasp the crazy sequence of subjects and knowledge, which they learn but quickly forget. Above all, it robs them of their free time. The solution to problems is often more pre-school, more homework, more schooling. The consequence of all this is that children become indifferent to the adult world and its achievements and wonders. Curiosity is crushed, the future ignored and compassion suppressed as the weak are preyed upon. No one, he thinks, escapes from school with their humanity intact, students, teachers or parents.

In an unorthodox fashion, he recommends plenty of solitude for children so that they can learn to live with themselves and conduct internal dialogue and not become addicted to company, the crowd and peer groups, which schools promote. This is very far from the current, social constructivist orthodoxy.

Recommendations

Education is not the same as schooling and, as a libertarian, he recommends less not more schooling and homeschooling but not in the sense of simply doing that to pass the standardized tests. He wants young people to have more contact with the adult world outside of school, do something they excel at, be challenged, do community work, learn to be autonomous learners, encouraged to use their imagination, problem solve, deal with set-backs. He makes an interesting distinction between communities and networks, which really masquerade as communities.

Criticism

His writing can often seem polemical, even hyperbolic, yet he marshals an astonishing amount of evidence and testimonies for his positions, as well his own considerable, personal experience. Another common criticism is that he is high on criticism, low on solutions. His libertarian alternatives have been attacked as unworkable or worse, putting large numbers of disadvantaged children at further disadvantage. To be fair, a libertarian, by definition, does not posit full and fixed systematic solutions. Nevertheless, in practice, many aspects of learning and education seem to need the organization and structures that Gatto wants to dismantle.

Influence

Immersed in teaching and schooling, Gatto’s critiques have more power because of his background as an outstanding teacher in disadvantaged schools. His critique of compulsory schooling is at times convincing, at times a diatribe. In line with Illich, and to a degree with Ken Robinson, he forces teachers, parents and educationalists to face up to some disturbing aspects of schools and schooling. What he perhaps fails to do it solve the problems.

Bibliography

Gatto Dumbing Us Down: The Hidden Curriculum of Compulsory Schooling (1992).

The Exhausted School (1993).

A Different Kind of Teacher: Solving the Crisis of American Schooling (2000)

The Underground History of American Education (2001)

'Against School' (2003)

Weapons of Mass Instruction: A Schoolteacher's Journey through the Dark World of Compulsory Schooling (2008)