quote:

Originally posted by Laura L:
Here's an interesting article...

http://www.swcp.com/access/eval.htm

my favorite excerpt from the summary:
"Kirkpatrick's fourth level of evaluation, results, is still difficult to measure. The difficulty here, as Kirkpatrick himself points out, is the ability to separate training from the multitude of other variables that can impact long-term performance."

Curious - does this mean you agree with Kirkpatrick, or not?

IMHO I think Kirkpatrick is wrong - it is comparatively easy to separate these factors out, even with the so-called difficult to measure soft skills stuff. A formal, rigorous root cause analysis is the tool of choice here.

Svenson and Rinderer has Training and Development Strategic Plan Workbook that is quite good. It is much broader than just evaluation, but provides the context and then evaluation. ASTD's book Evaluation Basics has a lot of tools that offer structure for all four levels. Actually, levels 2, 3. and 4 have two parts. There is also an evaluation plan that can help plus sample instruments.

Here would be my best evaluation resources:

All of Bob Brinkerhoff's stuff is very good and I believe that his model should be the default/baseline approach for evaluating in the HR world.

I would argue that Toni Hodges' book (Linking Learning and Performance) is the single best, most practical book out there on evaluation and measurement in the HR field. If I was looking for only ONE source that could help me practically improve evaluation skills and provide a range immediately useful tools, it would be Toni's book.

Judy Hale has very smart, common-sense things to say about evaluation (both in her presentations at various conferences as well as in parts of her book the Performance Consultant Fieldbook).

On ISPI's website, there is a wide list of white papers and PI articles available for download. The one that most closely matches my approach/philosophy towards evaluation is Alan Ramias' "What's all the fuss about measurement?" He does a superb job just cutting through all of the nonsense to illustrate how simple evaluation really is.

By the way, I totally agree with Martin's comment above earlier about Kirkpatrick. There are so many trainers running around muttering silly comments like "it's so difficult to prove level 4/level 5 (business impact and ROI)" or "you can't measure intangibles" or "training is intuitively valuable--we shouldn't have to show impact."

Level 4 (business impact) is one of the EASIEST measures (using Kirkpatrick's typology). The reason many classes can't show level 4 impact is b/c they don't have any real value to the business. If you do a good front-end analysis down to a root cause--and establish the problem is a lack of skills/knowledge, then showing level 4 or 5 is a snap. Showing level 4 impact is difficult for situations where it is open enrollment training or where there was no FEA done previously.

quote:

Originally posted by Martin Schmalenbach:
Curious - does this mean you agree with Kirkpatrick, or not?

IMHO I think Kirkpatrick is wrong - it is comparatively easy to separate these factors out, even with the so-called difficult to measure soft skills stuff. A formal, rigorous root cause analysis is the tool of choice here.

I'm not sure I'm understanding your comment... There are so many factors that affect a person's performance -- how can you guarantee that training (or not) is what impacted his/her success/failure? I never take credit for the success or failure of a trainee once they leave the controlled environment. Out in the real world, no less than 15-20 other factors can and do affect an individual's job performance, not the least of which is that individual's internal motivation (which you have nothing to do with).