
At $69 it's fantastic value.
All the evidence points towards Levels three and four being rarely attempted as all of the resource focuses on Levels 1 and 2. It is not necessary to do all four levels. Given the time and resources needed in evaluation better to go straight to Level four.
Level 1 - keep 'em happy
Favourable reactions on happy sheets do not guarantee that the learners have learnt anything, so one has to be careful with these results. This data merely measures opinion. Learners can express satisfaction with a learning experience yet might still have failed to learn. For example, they may have enjoyed the experience just because the trainer told good jokes and kept them amused. Conversely, learning can occur and job performance improve, even though the participants thought the training was a waste of time! Learners often learn under duress or through experiences which although difficult at the time, prove to be useful later. This is especially true of learning through mistakes and failure.
Too often applied after the damage has been done. The data is gathered but by that time the cost has been incurred. More focus on evaluation prior to delivery, during analysis and design, is more likely to eliminate inefficiencies in learning.
I went to lots of brilliant comedy shows in the Edinburgh Festival this year, and was as happy as I've been allyear, but can't remember a single, damn joke.
Level 2 - Testing, testing
Recommends measuring difference between pre- and post-test results but pre-tests are often absent. End-point testing is often crude, often testing the learner’s short-term memory. With no adequate reinforcement and push into long-term memory, most of the knowledge will be forgotten, even if the learner did pass the post-test.
Level 3 - behave yourself
At this level the transfer of learning to actual performance is measured. This is complicated, time consuming and expensive and often requires the buy-in of line managers with no training background, as well as their time and effort.
Many people can speak languages and perform tasks without being able to articulate the rules they follow. Conversely, many people can articulate a set of rules well, but perform poorly at putting them into practice. This suggests that ultimately, Level three data should take precedence over Level two data.
Level 4 - does the business Fewer shortcomings. The ultimate justification for spending money on training should be its impact on the business. Measuring training in relation to business outcomes is exceedingly difficult. However, the difficulty of the task should not discourage efforts in this direction.
What to do? Should you evaluate at all? Of course, it is one thing to critique the Kirkpatrick model, another to come up with a credible alternative. I’d say apply Occam’s Razor - minimise the number of entities you need to reach your goal. Put the over-engineered, four-level, Kirkpatrick model to one side as it is costly, disruptive and statistically weak. Focus on one final quantitative and qualitative analysis.
I liked Stephen Kerr’s view, the CLO at GE, then Goldman Sachs - Kirkpatrick asks all the wrong questions, the task is to create the motivation and context for good learning and knowledge sharing, not to treat learning as an auditable commodity. He would literally like to see Kirkpatrick consigned to the bin.