How to measure training effectiveness

Four levels of measurement

The four levels, widely known as the Kirkpatrick model, build on each other. A program that participants disliked rarely changes behaviour, but a program they enjoyed may still change nothing. Measure at least the first three levels for every significant program, and the fourth for programs that are expensive or aimed at a specific business problem.

• Reaction: did participants find it relevant and useful? A five-question form at the end.
• Learning: did knowledge or skill improve? A test before and after, or a practical demonstration.
• Behaviour: are people doing it on the job 30 to 60 days later? A manager's observation checklist.
• Results: did the business measure move? Errors, complaints, sales, rework or accidents.

Worked example: billing training for store cashiers

A Bengaluru supermarket chain trains 40 cashiers on billing, returns and exchanges. The pre-test average is 52 percent and the post-test average is 81 percent. Participants rate the session 4.3 out of 5. Thirty days later, store managers observe each cashier for one shift using a ten-point checklist, and 34 of the 40 follow the returns process correctly, which is 85 percent. In the quarter before training, billing errors were 1.8 per 1,000 bills; in the quarter after, 0.9 per 1,000. The error rate halved, and because nothing else in the billing process changed that quarter, the training can reasonably take much of the credit.

Choose the result measure before training

Agree with the business owner, before the program is designed, which measure should move and by how much. Take a baseline for at least a quarter. Where possible, compare a trained group with one trained later, such as one region before another, so you can separate the training's effect from seasonal changes. A result measure chosen after training usually ends up being whichever number happened to improve.

Traps when measuring training

Measurement goes wrong in predictable ways, usually because it is added at the end rather than planned at the start. Keep these traps in mind when you design the evaluation, not after the program has run. A one-page evaluation plan, written with the program brief, avoids most of them.

• Treating a good feedback score as proof the training worked
• Tests that check memory of slides rather than skill in the task
• No baseline, so there is nothing to compare against
• Crediting training for changes caused by new staff, tools or seasons
• Measuring behaviour after a week, before new habits settle
• Never sharing results with the managers who released their people

Step by step

1. Agree the business result first. With the business owner, pick the measure that should improve, such as billing errors per 1,000 bills, and the size of improvement expected.
2. Take a baseline. Measure the result for at least one quarter before training, using the same data source you will use afterwards.
3. Run a pre-test. Test knowledge or skill before the session, so you know what participants already knew and can show real learning.
4. Collect reaction feedback. Use a five-question form at the end: relevance, clarity, practice time, trainer and what the participant will do differently.
5. Run a post-test. Use the same or an equivalent test at the end of the program and compare averages and individual scores.
6. Observe behaviour at 30 to 60 days. Give managers a short checklist of the behaviours taught and ask them to observe each participant during normal work.
7. Compare the result measure. After a quarter, compare the business measure with the baseline and note other changes that could explain the difference.
8. Record completion and follow-up goals. In ZeniaHR, mark participants complete in Learning, which records the completion, and add the new skill as a competency goal with a weight in their next review cycle.

Frequently asked questions