I work in the construction industry, more specifically site contracting and quarries. We are struggling to get a real idea of the impact and effectiveness of our training programs.

We have been offering training for two years now and have yet to determine a way to effectively evaluate the training. We thought things were good until we sent a survey to help develop level 2 training courses and the resulting questions were all about topics covered in level 1 courses.

We currently use smile sheets for training and pre test-post tests. Any other suggestions? The evaluation we are looking for is not monetary but whether there is retention of information. Please let me know if you have any suggestions.

Thanks,

Kris

We had a similar problem with Boeing inspectors several years ago. In reality, they could not even pass level 1 qualification tests. Yet, many were certified level 2 and were carrying out inspections - many of them incorrectly. The problem was that training was under the arm of HR. They hired semi-retired inspectors to do classroom training. It was worthless of course since they were not trained to train anyone and their skills were obsolete. And they posted the answers to the written tests on the board for all to use during the tests. Besides the classroom training, they had the old degenerating buddy system of OJT.

The solution was to totally eliminate classroom training. Engineering (quietly) took over the inspector training and involved the inspectors in creating their own structured OJT system - from scratch. Inspectors literally had to learn everything all over because they had so many bad habits as well as knowledge and skill gaps. All of the inspectors were divided into groups of 6-8 each. They spent one to two hours per week with a trained facilitator and expert engineers to learn how to do their jobs as they participated in determining how to do the tasks. While learning, they created new procedures for every single task and tested them on-site as each one took turns playing the roles of trainer and trainee. It took one year to finish the compete set of tasks. By then all the inspectors were proficient in level 1 and many were proficient in some level 2 tasks. By the end of the following year, most of the inspectors were certified level 2. (Their qualification tests were significantly different of course.) Inspectors loved the team sessions and even volunteered to come in on their own time to work on the new procedures. Those sessions were a tremendous learning opportunity - more so than if they had just participated in structured OJT by itself once it was developed. Engineers at the company are still in charge of the training.

The very fortunate thing was that the factory superintendents as well as the engineering management supported the program 100%. The reason? - they were desperate. Sometimes the only thing to do is to start all over with a blank slate.

This may be simplistic but have you asked the workers which form of training was most effective for them? Perhaps having "OTJ" training is best? Perhaps they learn better in a lecture setting? Who knows. Obviously there is a problem with knowledge transfer so the training is not successful...some other questions....are you getting feedback from everyone or just a few people? You need to determine what is stopping the transfer. Could it be the managers? The hardest thing about training is making sure it is delivered at the right time. I can take a course and ace it, but unless I have an opportunity to apply my knowledge what I have learned is useless. Also, when the courses were designed, were Instructional Design concepts used? It is important most of all to understand how your workers learn best and to contruct training around that and at the right time. Not an easy job!

BTW, I just started designing our training program at my compandy. I'll let you know if I'm successful. I might be the one posting like you in 2 months.

Good luck. Please keep us posted. I am sure we all want to know what works!

I believe that evaluation of training should begin before training is even designed. Definition of what training is supposed to accomplish and how that accomplishment can be detected or measured should be "Step 1." If that is done, the actual evaluation process is a lot simpler than if it is not. Unfortunately, Step 1 is most often bypassed.

However, even if Step 1 is bypassed, all is not lost (misplaced maybe but not completely lost).

Document two items: What is the training supposed to accomplish? (Document the answers, not necessarily the question.)

The second item is how would one determine if that the training had accomplished its purpose?

These two questions by themselves won't give you the definitive answer but could lead to other questions that might yield more light on the subject.

Thank you all for your responses. I look forward to put to use the suggestions. And Jackie, I will definitely keep you all posted.