I cannot understand why we do not design our soft skills training to achieve level four results for soft skills training. If level four is what we want, why do we keep producing programs that only give us the lower levels? What do you think? Do we not seem to defeat our own purpose with designs that yield only levesl 1-3?

I think it's time to stop thinking of it in terms of "levels" (I know - everyone's stuck in Kirkpatrick mode... and I wish people would remember that it's not a hierarchy, but I digress), and to start beginning with impact and ROI in mind. If there is no tie-in to the business goals and performance expectations in the first place, why is the course even being offered?

In other words -- if your soft skills classes were designed up front tied directly to specific learning needs that tie directly to specific performance gaps/opportunities that tie directly to business goals, the proof will be in the pudding. You have your ROI when you ensure that the offerings are needed in the first place for acheiving business goals. (see Brinkerhoff - High Impact Learning for details)

In other, other words - you might already have your ROI, assuming you implemented things that were necessary in the first place.

Usual disclaimers apply

Laura,

I agree that we shouldn't think about levels.
In fact if we design to get what Kirkpatrick calls level four, won't that inlcude levels 1-3?

psyche -- not necessarily... that's the reason I pointed out that these are not hierarchical levels. They are distinct types of assessing the learning situation.

1 - Did they like it?
2 - did they get it?
3 - are they using it?
4 - was it all worth it?

...and while if someone likes something, he may be more likely to attend to what he's doing and more likely to at least try to apply it to the workplace...there are too many other factors that impact performance to really concretely tie everything together in a neat little package and attribute it to training.

Consider the participant who just wants to get the heck out and barely takes any time to respond to your "level 1" eval. Does this really tell you anything? Maybe he just liked the snacks you served and liked not having to be at his desk for 4 hours.

Likewise, just because you can show that business goals are being reached, there's little chance you can boil that all back down to the fact that people were "trained" (I hate that word) to follow a new call guide and really liked the class.

There are lots of examples...but I think you get my point.

Basically, it's my contention that attempting to measure ROI after the fact is, well, generally a waste of energy. The ROI needs to be established up front. That helps ensure buy-in from everyone - most importantly your participants. Also, it's been suggested that "ROI" isn't the right way to look at the services we trainers/designers offer. Think in terms of organizational impact, and present it that way.

Laura,

You have good insight and have obviously thought a lot about this.

So training should really be tied directly to strategic plans. If so, why does so much training seem to be in an objectiveless vacuum?
It is all about topics and not about achieving specific objectives and how to do it. Do you agree?