If a training session has a positive result on the reduction of costs, quality, and reduction of accidents, for instance, then one could conclude that a level two and three have also been achieved. However, if it has not made a significant impact on lowering costs, quality, and reducing accidents, then one could conclude that levels one, two, and three, have not been achieved. It would seem that some type of pre-measuring occurs before training to aid the final outcome assessment. Would you agree or disagree?

At the risk of telling you something you already know.....

Level 3 evals are where the rubber meets the road...so focus there. Pre-assessment is not only appropriate, it is invaluable when it comes to monitoring progress.

My blog (www.360fyi.com) has more conversations about assessments.

Jeff

Thanks, Jeff. I appreciate your feedback. Your blog has also been very helpful as well.


Court

I'm not a big fan of Kirkpatrick's Typology for a bunch of reasons. But here's my take:
1. You can get a positive result for levels 2&3 and still not achieve much with level 4. How could that be so? It may be that the human skills and knowledge aren't that relevant to the goal. For instance, I could (to use a simplistic example), train an inefficient workforce to manage time better and yet sales could still plummet (because our products are inferior on the market). Or we've overestimated the importance of skill and knowledge to this problem (if we could wave a wand and instantly improve the workforce skills and knowledge of all workers in the USA, that still wouldn't change the problems that financial institutions are in--failing to lend, over-reacting by refusing to take any risks at all). Or if we sell the products at a loss, if I improve productivity (let's say at a level 3--people produce more widgets) than we'll end up losing MORE money (a negative level 4 and level 5-ROI).

2. There is this tendency to assume that levels 1, 2, 3, & 4 are connected (if we hit #4 than obviously we got 1,2 & 3). I don't think so:
--people could hate the class and still benefit from it. Conversely, they could love it and it doesn't help the organization. I've seen both examples many a time.
--We could achieve level 4 because the training class sends a message to people "we're serious about this" so they pay more attention to detail. They don't enjoy the class, they don't learn anything significant from it, the behavior we seek to measure at work doesn't change....yet we get better results.
--the only real connection I see is that for a trainer who is reactive, positive level 1, 2 and 3 data suggests an impact on level 4 (thus allowing the trainer to claim some value-added from the course).

I do agree with Jeff about pre-assessments. Ultimately, I'd encourage people to focus less on the 4 levels. Ask yourself "why are we evaluating?" and the answer to that question should tell you what you ought to be measuring and tracking. If we want to see if the pilot is well designed, do a lot of level 1 stuff. If the course is locked in (it's required by law--maybe a certification program or state licensing) than why would you do level 1 evaluation?