Hi @ all,

im trying to find a method to find, describe and hopefully control the external bias (variabel disturbance) in a field experiment (82 courses of a sales training).

Does anyone have experience or literature dealing specially with this topic?
Best Regards
Jürgen

Hi Jurgen

Just so I'm clear, are you asking about quantifying the effects of variables other than (including?) training that have an influence on performance?

I'm guessing that your field experiments are to look at the effects that training interventions have on actual performance, acknowledging the fact that other factors/variables are almost certainly affecting the performance too.

Kind regards

Martin

Hi Martin,

that´s true. The main problem in my "experiment" are the factors that are affecting performance beside the training.

Kind regards
Jürgen

We've addressed this issue by first looking at time to proficiency. Then after training you can look at how much you reduced time to proficiency. For example if you have new salespeople making their first sale in month 3 rather than month 9. That change is easily measure and tracked to ROI. If only include the training event, you will have a more difficult time in measuring. On the other hand if you include all of the practice, experience and coaching in the training, you will have eliminated most of the unknown variables.

The method suggested by Learning Path Consultants would certainly give an indication about one output variable, which is time to proficiency, but does it account for all other significant input variables? I can't see how from the response, though it is very true to say that a correlation calculation would certainly go some way to providing clarity. And the thing to remember with correlation is that it isn't causation!! Also, if you can wait until 3 or 9 months after the training to see if it has worked, then no problems with this approach, but sometimes it is necessary to know in advance of the training going ahead.

A root cause analysis, where all possible variables listed (by brainstorm of experts?) and then data is gathered, experiments made, to quantify the contribution of each variable to the overall output variable, e.g. time to proficiency, sales volume, error rates - what ever - and then using a Pareto analysis to enable you to identify the top 6-10 variables, so that say 95% of the output is accounted for.

By this stage you will have a pretty clear understanding of what variables feed in to the 'process black box' that produces the output in question (sales, error rates etc). It makes it so much easier then to see where training can help improve performance, and consequently so much easier to get an ROI on the training. If you tackle for example absence through very targetted training, and absence contributes 34% of the problem or variation in output (e.g. late delivery) and the problem as a whole is valued at $300K in cash (e.g. fines), then you reduce the impact of absence by say 40%, then the value of this is training is $300K x 34% x 40% or $40,800.

If this training is going to cost $50,000 it seems to be a very bad investment, BUT, if tackling absence is an absolutely essential part of the overall project to reduce absence (by tackling the top 6-10 variables driving the problem), the overall cost of which is say $100K and the outcome is a reduction in fines paid to just $50K, then the training is worth doing, but it's isolated, individual ROI is in essence meaningless.

This is why I advocate tackling projects and looking at the ROI of the project, ensuring that where possible the training is an essential part of the project.

My recent experience is that CFOs and Finance Directors are interested in projects as a whole having good ROI, not individual components of it, and tend to be more interested in how well the training will support the project and less about the ROI of the training per se.

I have a pile of articles on this area that are freely downloadable at www.5boxes.com/resources

Good luck!

Martin