Hi,

Have been trying to contribute to your article, finally got access to the discussion board...

Think your point is valid. The important thing here is that as "Training Professionals" how do you know that the increase in performance of an individual is because of the training intervention or due to other "X" variables (Supervisor's instructions, fear of failure, seasonality etc.). There are other softer methods of doing the same, however, they do not provide any concrete data. The critical solution here is to be able to identify these "X" factors and remove them from the actual equation. Ofcourse, since the performance of an individual can be tracked over a period of time, what has really worked for us is using data modeling techniques to get some of these results. We have been able to decouple some of these variables and plot the actual proficiency of the person and this can give you a clearer picture to determine if the Intervention really made a difference or not and if it did, be able to quantify the same. Once you have been able to get the actual difference in proficiency, you can calculate the ROI (and this time it's pure data - no speculation!!)

Jiten Kapoor
Keane Worldzen
+91-124-5125000 Extn -5020

This message has been edited. Last edited by: Jiten Kapoor,

What about the a factor such as -- training wasn't the issue in the first place (i.e., the poor performance was caused by another factor or a plethora of other factors, not lack of skill or knowledge of the job that needs to be done)

Laura,

You are absolutely correct. This is the biggest part of the challenge -- like I mentioned earlier--and is not easy! What you are talking about are competencies required to do a particular job or task to a predefined standard – how do you determine if the poor performance scores are because of lack of knowledge, skill or will - correct? Also, knowing for sure that they are not the contributing factors for non-performance is half the job done. You just delineated those variables...

There are two specific paradigms that we are looking at - Training for new hires and for employees who are already in production - the strategy for both should be different. For the new hires, they have a common starting point (may or may not have prior experience) you are trying to get them from 0 to x level, and for the ones on the production floor – you may want to bring them from x to x1. Now your obvious question would be for the ones who are already in production and are at a level lower than x or x1 – what do you do about them? Read on…

Pre-planning – Some of the things you should do/ have –

1. Do a competency mapping for the function/ process.
2. Do a benchmarking of the KPIs (key performance indicators) for that function/ process (comparing with the monthly targets for team/ portfolio, and looking at where your high performers are on those KPIs). Delineate all known variables from the performance figures. This will give you your performance blueprint. Word of caution – do benchmarking by selecting just one function (what you are trying to do is trying to keep other variables constant as far as possible).
3. Select the function this individual belongs to and compare his performance with the blueprint, for each KPI.
4. Also, you should complete your Readiness assessment for that group/ individual.

This would essentially give you the specific training need to focus on for maximum difference. To the question we posed in para 2, similar exercise can be carried out for anyone already in production irrespective of the level. Post training, you need to track the individual/ group’s performance and see if the needle moved, if yes, you were successful. For new hires, it is easier to show ROI as you can demonstrate if you have been able to bring down the time to get to level X and then calculate the actual $$ impact due to the intervention.

I am not too much in agreement with the methods suggested in the main article. I found them very subjective in nature – e.g. conducting post training interviews, asking experts to estimate the impact of training etc., they would instead produce many other variables, for some of which it would be impossible to delineate the effect. Well to be honest, I feel that Kirkpatrick’s model for evaluation is not even a model because it tells you only if the training went well, it does not tell you anything if the training didn’t go well. If it did, we would not be having this discussion and you would not be looking for this answer..….

Jiten Kapoor
Keane Worldzen
+91-124 - 5125000 Extn - 5020

This message has been edited. Last edited by: Jiten Kapoor,

Hi All,

Thanks Jiten for sharing your experiences.

Just to clarify for readers of the original article - I am not advocating these subjective techniques as Jiten refers to them - quite the opposite and I say so towards the end of the article. I have a very clear view on this, but I wanted to get some discussion around this.

I am not a huge fan of the techniques advocated by many that rely on estimates and poor use of control groups etc. Sure, they may placate some 'C' level officers especially when times are easy, but in my experience, when times are heard, the pressure is on to show a return in some form, or stop spending on the training. You don't have to like this reality, but you do have to live with it - or take yourself to a different organisation (reality). In my experience (not that my experience is any special measure in itself, but it's all I've got!) the overheads involved in using these subjective, less robust methods can actually be more than the rigorous, scientific methods I am in favour of.

I share Laura's question along the lines of "why is it necessary to isolate the effects of training?" There are many cases where to do so is meaningless. These will be cases where the training is essential to improved performance - the training is integral, it cannot stand alone with any meaning, and so to isolate it for the purposes of ROI calculations is also meaningless - you have to look at the ROI of the entire performance improvement project.

Have a great weekend.

Kind regards

Martin

Well, actually no -- I'm referring to other factors such as bad managers, lack of resources, no team work, poorly outlined organizational goals, weird corporate culture, poorly defined processes, etc... things that, most of the time, the employee has absolutely no control over.

Even if part of the problem was a need to learn or improve a skill, it wont mean diddly if the employee has a really bad manager who never follows-up. (for example)