quote:

Part of the problem with the conversation about learning and evaluation is the tendency to forget the essential nature of what must take place in-between.....coaching, practice, reinforcement, feedback, accountability and organizational support for the learning.

Interesting observation. It just struck me that this is the difference between the learning that takes place in colleges and universities and the learning that takes place in organizations. In the former, the professor not only delivers the "learning event" (usually lecture), s/he also does follow-up coaching, assures that students practice what they are learning, reinforces what was learned, and holds students accountable. Evaluations are usually spread out over the course of the learning period, providing frequent feedback. In corporations, training is a one-shot encounter with a trainer/facilitator. There's no way a manager could step in after a one-shot event and do all that a professor would do to assure that learning takes place.

quote:

In corporations, training is a one-shot encounter with a trainer/facilitator. There's no way a manager could step in after a one-shot event and do all that a professor would do to assure that learning takes place.

And therein lies the rub. In order for training to stick it must be reinforced on the job with coaching, practice,feedback, accountability and organizational support. Precisely the role and responsibility of managers. Not the trainers/facilitators...but the managers.

It takes those activities to rewire the brain and make learning permanent. It took Tiger Woods a full year to get back to his championship form after changing his swing technique. A full year...and that is with a million dollar coach and daily practice.

One thing I suggest for my T&D colleagues is that training be followed up with a post training support survey that asks the learner for feedback on what the organization is doing to help make the training stick. A simple series of questions that ask about encouragement, coaching, feedback, recognition, rewards...et al.

quote:

Not the trainers/facilitators...but the managers.

Bingo... not to mention the employees themselves. It's unfortunate how many people seem to forget that it is the individual who is in control of and responsible for him/herself.

I am not an M&E expert. I have a question on measurement of ROI, say, or other post-Kirkpatrick metrics.

Consider three common WLP interventions, all of which are "training":

• A half-day event describing company insurance programs, leave programs and the like. This might be part of an "onboarding" scheme.
  
• A four-day hands-on course teaching Java programmers how to program in C# for Windows Vista. The programmers will be creating all new software in C# instead of Java because of a client mandate. (Do not consider loss-of-client in the ROI analysis.)
  
• A one hour e-learning segment (JIT) on copying MS Word formatting from one document to another.
  

IMHO, it would be very difficult to do a cost-effective, real-world, comprehensive ROI analysis on these. If I am wrong, please let me know how you would do it.

Travelerjjim,

I agree with you. Have clients ever asked you to show ROI for this type thing? I will be surprised if they have. I have spent the past 15 years setting up on-the-job training programs, from initial analysis through evaluation, for all types of organizations and industries - medium to very large companies mostly. Not one client has shown the slightest interest in evaluating the program! Not one! During the first implementation of the OJT program at an airline, I even enlisted a NASA aviation research scientist to help develop the evaluation piece of the program and work with the company on the evaluation. (NASA was willing to provide that service to this particular organization for FREE.) Were they interested in taking advantage of that opportunity? NO - not even a little bit. I have subsequently spent a lot of time further developing the evaluation part (mainly because of the book), but clients do not use the evaluation piece. They can see the results first hand and I suppose that's the reason. I never hear them say that management wants to see any evaluation, much less ROI - it's obvious from the program itself. You can SEE the results.

I think that must be the key to this evaluation thing. Managers who insist on ROI have obviously NOT been seeing any results in their training programs. If they saw results, they would not bother. That's the only thing that makes sense.

I would imagine the examples you listed would fall in the same category - i.e., they can see the results.