Hello all,

I am wondering if anybody has done a level three evaluation of training for new hires. A level three evaluation shows a change in behavior, however, as new hires, they have no history against which to measure.

I could measure the number of deviations to standard operating procedures (SOPs) by previous new hires (thus showing they haven’t learned what was taught) in order to generate a baseline, but the data mining process is too complex and time consuming. Also, how long is someone considered a “new hire”?

Do any of you know of an easier way to measure this?

The training I'm trying to measure is largely on-the-job training (OJT) for laboratory employees.

Thanks,
Matt

Don't bother with trying to force a match between current and previous new hire population. If it has any utility at all it would be to gauge how HR is doing in the hiring process.

You still need to establish a baseline for each person. Periodic assessments can help monitor progress and identify gaps in the training activities.

Personally, I would wait several months before attempting any baseline work at all...in order to afford the person the space to get acclimated to the work and the culture.

Jeff

Matt,
Always start out before any evaluation effort by asking "why?". Or more specifically, "to what purpose am I evaluating?"

Now, the primary purpose of any on-boarding or orientation program is to be sure that new employees are effective employees. A lot of this stuff involves simply data/knowledge (which a level II exam would test effectively). But to the extent there skills they need to acquire, you can test that. Or, if the purpose is simply to see if they're a semi-productive employee, you can test that.

Now you're absolutely correct that you don't have a baseline, a "pre-test" if you will. So what? If the purpose is to determine "post-orientation, is the new employee doing/behaving/performing as he/she is supposed to?" than your level III can measure that (assuming you're measuring the right stuff--in my experience most measurement of behavior is a waste of time and resources--but that's for another topic).

Now let me anticipate a question/challenge: with no baseline, how do you know that the training made the employee productive? But the assumption the vast majority of organization make is that new employees need the orientation so they sheep-dip everyone, it's usually not worth the effort to test prior to the training.

I don't think there is a lot of value in you being able to argue that the orientation/OJT intro stuff has a certain increase in performance due to it. Rather, you just want to be able to say "we hired X number of people, they all went through the SOP training and 2 months after testing, all were competent." And if anyone questions you "well, how do you know that was due to the training?" you can point out that first, it's a reasonable assumption that a brand new employee with no experience is not ready from day-1 to instantly perform and second, you could easily test that by putting people on the job without orientation, without OJT intros for a month, testing performance, then training and then measuring the difference. But the cost to this measure would be exceedingly high. And the potential cost to the business (imagine an untraining, unoriented worker on the job) even greater. Then you smile sweetly as you look at the person who asked the question and ask back "so are you saying you'd like us to try that? To determine which new hires don't need training? Or do we continue assuming that ALL benefit from it?"