Measuring the Effectiveness of Training

How do you know that your training programme is effective? Would assessments at the end of a training programme help you measure the effectiveness of the training?

Maybe to an extent, but not quite.

Assessment Acumen: Do You Have It?, an article by Margery Weinstein in a recent issue of Training Magazine has the following quote from Roger Chevalier.

“The biggest mistake is we follow-up with students based on whether or not they've acquired knowledge, and that's a problem. The outcome we're looking for is actually a change in behaviour,” he says. “I think all trainers need to redefine learning as not just the acquisition of knowledge but as the ability to demonstrate a desired behaviour, and that learning is everything, and anything, that contributes to the change in behaviour we’re looking for.”

As Roger rightly points out the primary goal of training is to impact the learner’s job performance. Assessments that test for new knowledge at the end of training do not accurately convey whether the learner will demonstrate the desired behaviour on the job.

(By the way, I’m not saying that assessments at the end of training are not required. What I’m saying is that such assessments cannot be the primary means of measuring training effectiveness.)

So, to measure the effectiveness of training, we need to focus on evaluating the transfer of the learner’s news skills and knowledge on to the job. This kind of evaluation corresponds to the third level in Donald Kirkpatrick’s model of evaluation. The evaluation at this level attempts to answer the question: are the newly acquired skills, knowledge, or attitude being used in the learner’s everyday environment?

Of course, conducting such evaluations is challenging. First, we need to determine when we can conduct such an evaluation. It’s not fair to expect a learner’s performance behaviour to change immediately after the training.

But when is it fair to do the evaluation? After a month? After two months? I believe that would depend on the kind of training and the length of training. There are no easy answers here.

And how do we plan and conduct the evaluation? Ed Mayberry has some answers in her article in Learning Circuits.

Dr Jeanne Farrington also proposes a six-step method to evaluate learning transfer. Look at her article, Measuring Transfer for Results and Glory, in a recent issue of the DSA newsletter.

The Assessment Acumen article referred to earlier in this post gives some pointers on what some companies are doing to better evaluate if their training programmes are working.