A Baker's Dozen (+1) Design Disasters

For a training professional, it always helps to read over and over again about what can go wrong while designing, developing, and delivering training courses.

Becky Pluth, in a recent article in Bob Pike's Training and Development e-Zine, lists the following as a baker's dozen (+1) training design disasters.

1. Using language that belittles participants or puts on airs
2. Designing the training first and then writing the terminal and enabling objectives
3. Spending too much time on the nice-to-know versus the need-to-know
4. Chunking the content into unmanageable learning portions
5. Stating objectives of the session and then not meeting them
6. Not providing a roadmap of where the session is going
7. Telling stories that don’t match the message but are your “favourite” and everyone “loves” them
8. Telling participants what to do versus showing them and allowing them to DO
9. Not building activities that teach your content into the training
10. Creating job aids that are conceptual instead of behavioural
11. Using a PowerPoint deck as your handout
12. Not reviewing or revisiting content throughout the session
13. Using the same activity multiple times
14. Overusing one form of media (DVDs, gaming, books, etc.)

For elaboration on the above points, read Becky's full article.

What Motivates Learners

For any training programme to be successful, learner motivation is an absolute must. So, what are the training characteristics that motivate learners?

Edmund Saas, a professor of education, has some answers.

In the late 1980s, Edmund conducted a study to determine student perceptions of those college classes likely to result in high and low classroom motivation. Edmund surveyed about 700 of his educational psychology students for the study.

The results of the study are discussed in Motivation in the College Classroom: What Students Tell Us, an article in the Teaching of Psychology journal (subscription required).

The following eight characteristics emerged as the top contributors to learner motivation (in order of importance):

   1. Instructor's enthusiasm
   2. Relevance of the material
   3. Organisation of the course
   4. Appropriate difficulty level of the material
   5. Active involvement of learners
   6. Variety of instructional methods
   7. Rapport between instructor and learners
   8. Use of appropriate examples

The first three characteristics had considerably higher ratings than the rest of the characteristics.

The top characteristics that students perceived as non-motivating were expectedly the opposite of the motivating characteristics:

   1. Lack of variety in instructional methods
   2. Disorganisation of the course
   3. Little or no active learner involvement
   4. Lack of involvement of the instructor
   5. Lack of interpersonal warmth from the instructor

If you want further descriptions of the motivating characteristics, see David Gershaw’s article titled Motivating Students to Learn.

Measuring the Effectiveness of Training

How do you know that your training programme is effective? Would assessments at the end of a training programme help you measure the effectiveness of the training?

Maybe to an extent, but not quite.

Assessment Acumen: Do You Have It?, an article by Margery Weinstein in a recent issue of Training Magazine has the following quote from Roger Chevalier.

“The biggest mistake is we follow-up with students based on whether or not they've acquired knowledge, and that's a problem. The outcome we're looking for is actually a change in behaviour,” he says. “I think all trainers need to redefine learning as not just the acquisition of knowledge but as the ability to demonstrate a desired behaviour, and that learning is everything, and anything, that contributes to the change in behaviour we’re looking for.”

As Roger rightly points out the primary goal of training is to impact the learner’s job performance. Assessments that test for new knowledge at the end of training do not accurately convey whether the learner will demonstrate the desired behaviour on the job.

(By the way, I’m not saying that assessments at the end of training are not required. What I’m saying is that such assessments cannot be the primary means of measuring training effectiveness.)

So, to measure the effectiveness of training, we need to focus on evaluating the transfer of the learner’s news skills and knowledge on to the job. This kind of evaluation corresponds to the third level in Donald Kirkpatrick’s model of evaluation. The evaluation at this level attempts to answer the question: are the newly acquired skills, knowledge, or attitude being used in the learner’s everyday environment?

Of course, conducting such evaluations is challenging. First, we need to determine when we can conduct such an evaluation. It’s not fair to expect a learner’s performance behaviour to change immediately after the training.

But when is it fair to do the evaluation? After a month? After two months? I believe that would depend on the kind of training and the length of training. There are no easy answers here.

And how do we plan and conduct the evaluation? Ed Mayberry has some answers in her article in Learning Circuits.

Dr Jeanne Farrington also proposes a six-step method to evaluate learning transfer. Look at her article, Measuring Transfer for Results and Glory, in a recent issue of the DSA newsletter.

The Assessment Acumen article referred to earlier in this post gives some pointers on what some companies are doing to better evaluate if their training programmes are working.

How Much PowerPoint?

Today, PowerPoint appears to be omnipresent in every instructor-led training programme. But, is it too omnipresent? Omnipresent to the extent that it actually hinders learning?

Perhaps.

But how does one measure the omnipresence of PowerPoint? Based on the number of PowerPoint slides used?

Yes, that is one parameter we could look at. So the question is: what’s the maximum number of slides one should use for an hour of effective training?

There seems to be no clear answer to the question. I believe no significant research has happened in this area.

However, we do have a few pointers that give us some hints. Though the available pointers or guidelines relate to presentations rather than training, I believe we could extrapolate the guidelines to training too.

Guy Kawasaki, the venture capitalist, is often quoted on his 10/20/30 rule for presentations: a PowerPoint presentation should have 10 slides, last no more than 20 minutes, and contain no font smaller than 30 points.

Of course, Guy is mostly referring to presentations about raising capital, making a sale, and so on. And, he’s talking about short and crisp presentations.

Training presentations could be considerably longer than just a few minutes. But, as the presentation time increases, the relative number of slides should decrease. So, I don't think Guy would recommend having 30 slides in a presentation of one hour. Those are far too many slides.

Jeff Taylor, the founder of monster.com, is reported to use 11 slides for an hour of presentation. John Chen, the CEO of Sybase, is said to use 15 slides in an hour. Tony Robbins, the motivational guru, apparently uses just 5 slides in two hours!

FKA (Learning and Performance consultants), in one of their newsletters, says that their research suggests using a maximum of 12 slides for an hour of effective training.

Guila Muir, a train-the-trainer specialist, has a far more extreme rule of thumb:

• For a 30-minute presentation, use 1-2 slides
• For an hour’s presentation, use a maximum of 4 slides
• For an all-day training session, use a maximum of 8 slides

When I posed the question to Saul Carliner (prolific researcher, author, and e-learning expert), he offered the following advice:

• For a presentation of less than 10 minutes, use 1 slide per minute
• For a presentation of 11-30 minutes, use 1 slide per 2 minutes
• For a presentation of 31-90 minutes, use 1 slide per 3 minutes (or 20 slides per hour)
• For a presentation longer than 90 minutes, use 1 slide per 5 minutes (or 12 slides per hour)

Assuming that most training sessions last for more than an hour, the guideline that emerges suggests having about 5-12 slides per hour of training.

Would you agree?

Useful Instructional Methods

Many instructor-led technical training programmes appear to suffer from the “death by PowerPoint” syndrome. In many cases, the instructor is reduced to a reader of PowerPoint slides rather than being a facilitator for learning. Needless to say, the training programmes are not as effective as they should be.

To improve the efficacy of training programmes, curriculum designers must consider a variety of instructional methods or tactics for delivering specific chunks of content. (The specific instructional method you choose depends on a few factors: the kind of content, the number of students in the class, the time at your disposal, and so on. But, that's a topic for another time.)

The results from a study conducted by Darryl Sink (Darryl L. Sink & Associates, Inc) suggest some useful instructional methods. In the study, Darryl and a group of training professionals evaluated a list of instructional methods in relation to learning domains, number of students, and desired outcomes. The following is their list of the 16 most useful instructional methods rated on a 5-point scale.

Instructional Method	Score
Game	4.01
Role Play	3.91
Problem Solving/Lab	3.83
Simulation	3.82
Project	3.78
Guided Lab	3.73
Field Trip	3.70
Team Project	3.69
Lab	3.67
Discovery, Group	3.65
Cooperative Group Learning	3.58
Think Tank/Brainstorm	3.57
Apprenticeship	3.51
Seminar	3.46
Demonstration	3.46
Case Study	3.45

A higher score means that the instructional method is more useful under more conditions than the instructional method with a lower score. Click Descriptions of Key Instructional Methods for descriptions of the above methods and others.

Of course, you and I may rank these instructional methods a little differently based on our own experiences. But that doesn't take anything away from Darryl's study. It will definitely help to have the results of the study by your side at your design desk.

P A F

Presentation, Application, and Feedback (PAF). These are the three elements of FKA’s learning model in the systematic learning process. This is a simple yet effective model that can help instructional designers develop training that works.

In the Presentation phase, new knowledge and skills are presented to the learner. But before the information is passed on, the motive for the information transfer is established. The motive answers the learner’s question: what’s in it for me? (WIIFM). The presentation phase also includes a test for understanding to ensure that learning has indeed happened.

In the Application phase, the learner is given an opportunity to practice or use the skills and knowledge just presented. This phase is important because people learn by doing.

In the Feedback phase, the learner is given constructive feedback to reinforce the skills and knowledge just used. The learner is also offered suggestions for improvement.

Not surprisingly, the PAF model roughly aligns with the nine instructional events from Robert Gagne’s Conditions of Learning. The nine events are gain attention, inform learners of objectives, stimulate recall of prior learning, present the content, provide learning guidance, elicit performance, provide feedback, assess performance, and enhance retention and transfer.

FKA also recommends that the presentation phase should take
30-40% of the learner’s time, whereas the application and feedback phases should take up the rest of the time (60-70%). Adhering to this time allocation guideline is extremely important in developing effective training programmes.

Many training programs do not allot sufficient time for the application and feedback phases of the systematic learning process. This could be a major reason for the overabundance of inadequate and ineffective training courses that we see around us.