Kaliko-

That is exactly where the disconnect is!! It absolutely has to do with impact. That's what training needs to be about, impact!

If they can now do C then something should change in their day to day work. And that "something" is what training should be getting credit for. And if what they can do at the end of training is no longer happening a day or two later then you need to ask yourself, why not? Why is training not impacting work?

There is no such thing as inertia in training, everything must be constantly measured and evaluated. Covey - "Begin with the end in sight". What are you trying to impact? How will you impact? How will you know when its has happened? How will you measure the result.


I am well aware of Kirkpatrick I still can't understand where people struggle with measuring the impact that training has.
My rules:
Sales people that go through sales training should be making more sales then others - check!
Customer service reps who go through CS training should be rated higher by customers - check!
Managers who got through management training should have higher scores and greater retention then managers who did not - check!

quote:

If they can now do C then something should change in their day to day work.

Not necessarily. All you can say about C is whether or not the learner could do something immediately after training. It does not mean that it resulted in real learning, that behavior changed, or that that the behavior change resulted in an impact on anything - Again, read Kirkpatrick. I think you totally misunderstand what the a three-part objective is and its limitations. You cannot use that to show impact on performance.

[quote]That's what training needs to be about, impact![quote]
Right, but not the way you are trying to claim.

[quote]And that "something" is what training should be getting credit for.[quote]
No. You cannot give training any credit for this for the reasons LL and Robert have pointed out.

[quote]And if what they can do at the end of training is no longer happening a day or two later then you need to ask yourself, why not? Why is training not impacting work?[quote]
Even IF they can do what they could do at the end of training a year later - it STILL would NOT say anything about impact on the work performance itself. Again, read Kirkpatrick or Phillips or any other evaluation book.

[quote]My rules:
Sales people that go through sales training should be making more sales then others - check!
Customer service reps who go through CS training should be rated higher by customers - check!
Managers who got through management training should have higher scores and greater retention then managers who did not - check![quote]

That is the ideal - but just because somebody can do C - does not mean anything - you just can't input X and expect Z. There is a huge difference between what C implies and that is meant in the learning profession by impact.

I suggest some more reading. When people in the field misuse the tools of the trade and claim erroneous findings, that hurts the whole profession's credibility.

quote:

Why, in heavens name, shouldn't training be quantifiable??!! Are you being asked to increase retention? Increase productivity? Increase sales? Increase knowledge?

You can't show cause and effect without a proper research design, and some way of controlling for influences upon performance not a result from training. In effect you need a control group, but even then it's a problematic interpretation.

Even in a best case scenario in "the field", with a control group, cause effect results are very difficult to interpret.

Group A gets training on task Z. Group B gets training on the same task.

One month later Group A performs better on task Z.

Training impact? Maybe. Or is it because Group A simply got more attention from their managers. Or that they actually learned nothing in training, but simply sitting in a class for one day oriented them differently, but didn't cause the skill increase.

....

Can you establish this stuff? Yes, actually you can with more sophisticated research designs which nobody will pay for or allocate time for in "the field".

Anyone want to test their logic skills and design as foolproof as possible a test of causality to show that training impacts on something important at work?

(and let's not forget LL's contribution, too which is spot on). How do you control for all of it?

Thing is, you come to me in my role of CEO and try to demonstrate cause effect with poor data and I fire you, because either a) you should know your logic is flawed, but you are too dim to know and/or b)you know it's flawed but you try and con me.

Uh, byebye, don't let the door hit you on the way out.

The REAL question we should be discussing is how bad is bad re: evidence of training effectiveness, and is some evidence, albeit questionable, better than not bothering?

The Training World Discussion LIst
http://groups.google.com/group/the-training-world

Kaliko, Robert and LL-

All of your responses are the very reason that training still struggles to have a seat at the executive table. Your unwillingness to stick your neck out and commit to having to create an impact is what is holding the training/OD profession back.
Of course there are other performance factors, dozens, possibly hundreds that prevent a "pure" scientific study, but that's not our job. We need to act the same way that sales, customer service, finance and all of the other departments act. I can't imagine ever sitting down with a CEO and saying "Well, we are going to offer the training. You may or may not see an impact from it. If things improve, it may or may no be our doing and if things get worse, it still may or may not be our doing". That's ridiculous!!

Our job is to directly impact the performance of employees. You don't need to conduct a scientific analysis in order to measure impact, because you will not be asked for one. You will be asked to impact performance. If you design training and performance increases, that's fine, you did what you were asked to do, you are part of the solution. If you design training and performance decreases, you can find an excuse for why it's not your fault, but that's what it comes across as, an excuse. You need to account for all potential variables and still be able to show that your work has a positive impact on what it is supposed to impact.

This comes up a lot when I am interviewing candidates. They typically have a long list of things they did, but rarely have anything about the impact they had on the organization. I only hire people who can show me that they clearly understand the connection between their role and the success of the organization. I have little use for people who just do their tasks, because they are asked to do it. They lack the innovation, drive and business savvy that is the cornerstone of the training profession today.

quote:

Our job is to directly impact the performance of employees. You don't need to conduct a scientific analysis in order to measure impact, because you will not be asked for one. You will be asked to impact performance. If you design training and performance increases, that's fine, you did what you were asked to do, you are part of the solution.

I think we are ultimately saying the same thing although our methods might be different. I believe that it is trainers who are responsible for their poor positioning in an organization or organizations. Trainers typically lack the political savvy to influence the perceptions of decisionmakers. I've already written about that.

Where we differ is in the value of measurement. We both believe that if it was possible to create unequivacable cause-effect evidence we should do so, vigorously. Where we differ is that I believe it is not possible, and faking it doesn't convince people, while you are ok with less than perfect measurement.

Perhaps you are correct. Perhaps poor measurement for cause-effect relationships is better than nothing. I believe I broached that question earlier in this thread.

Interesting story. Several times in my career I offered clients the opportunity to do a proper assessment of the effectiveness of my training by measuring, etc, and setting my fee so that if we achieve the agreed upon work performance outcomes i would earn a $10,000 fee (that's an example), but if the result did not occur I would only receive expense money ($500).

No client ever was willing to do so. I don't know what that means.

The Training World
http://thetrainingworld.com