Happy New Year everyone...

Deeg - 6+ weeks after your initial posting about proving the ROI of training, to what extent are your objectives met? Was the question that you first asked the one you really wanted to ask, or did that root question evolve for you over the course of the discussion?

My curiosity here is that initially you asked about proving the ROI of a programme, yet the only posting I could see that specifically pointed you to a clear process was the link to the 20/20insight video. The process outlined in this video is, I believe the same process advocated by Jack Phillips.

Here's the 'rub': it doesn't PROVE ROI!!

Why? Because it recommends ESTIMATING the impact (because of the belief that scientifically proving the impact costs too much to do - it's not worth the hassle so to speak).

In my book an ESTIMATE of impact is a long way from PROVING impact. To be fair to Phillips, 20/20 insight and the reality of most situations, as the video said, many executives can quickly reach some agreement as to the relative contributions each factor, such as training, market conditions, etc, have on bottom line results. So why get all 'scientific'?

Here's why: in my experience, and the experiences for example of many performance improvement projects that have taken a more scientific approach: managers and employees often have a distorted, even downright wrong impression of what the drivers and root causes are of the current performance levels (i.e. current performance problems!). And when the fight for budgets, even survival, is on in an organisation, just how accurate will this so-called consensus be now?

When executives are happy with estimates etc then give them estimates. When proof is needed, estimates aren't going to be as much help as you may think, or need them to be.

My own personal experience and view is that PROVING the ROI is neither difficult nor expensive, but it does take more effort than simply estimating the impact...

Using control groups to determine the impact only of the training is a valid way of doing things IF DONE RIGHT - the way it is usually described in the training world is flawed only in that the assumption is that THE ONLY factor differentiating one group from the other is that one group has received training. In all other respects the 2 groups are identical. This is almost always NOT the case...

Trend analysis with just 3 data points is risky - it assumes that there are no other effects of significance that are at work before the first data point you use, or after the third data point you use - again, where's the proof?

This doesn't mean these 2 methods should be avoided - they're just not up to the job of PROVING the impact...

An interesting discussion thread - thank you all!

Best wishes for 2008

Martin

Hi Martin,
Thanks for your input!

quote:

Deeg - 6+ weeks after your initial posting about proving the ROI of training, to what extent are your objectives met? Was the question that you first asked the one you really wanted to ask, or did that root question evolve for you over the course of the discussion?

I am about waist deep in the project and I guess you could say that my question has evolved. 6 + weeks ago I was looking to identify a dollar amount as ROI. After all of these posts I decided that it will be extremely difficult and time consuming to put a dollar amount on the ROI. Even if it is possible it would just be an estimate. Senior Mgmt would prefer I spent my time on other projects rather than torturing myself showing them how valuable my work is.
After reaching this peak of frustration I decided that I will measure the increase in knowledge from the beginning to the end of the project.
It is frustrating because I won't be able to point to any business impact because of it but I don't think this is the right project to do this on. Somebody mentioned that I put the horse before the cart in this project and I think that sums it up perfectly.