To all:

This is my first time posting to this board. Please bear with me as I try to properly pose my question and my request to any respondents.

I work at the Submarine Learning Center in CT, and have been asked to look into possible alternate methods of learning/training assessment. The number one thing that comes to people's minds when one speaks of measurement is, of course, Donald Kirkpatrick's theories of measurement, which are used extensively at Level One and Two.

The basic items that are measured in most military setting are satisfaction with the training received, the training environment, a graduate’s opinion of the instructor’s abilities, the attendees’ level of knowledge gained and their ability to perform tasks as they were taught. I understand they are looking to expand the sphere of what they are looking at, such as time to achieve competency, cycle times, and how they can apply the resultant data to improve courses of instruction.

Therefore, my question to this board and the training community in general: are there other methods or systems that your organization is using to measure learning and performance? Is it proprietary in nature, or is it available to the public domain somehow?

I appreciate your time and thoughts in this matter. Thanks to all in advance.

FLB

Pick up and read High Impact Learning (Brinkerhoff). I think it will provide you with extremely useful information and guideance to respond to this query.

Hi FLB

I agree with Fanatical Facilitator about Brinkerhoff's book.

Back in the mid-late 1990s I was responsible for running the ground-based training of aircrew for the British Air Force's Chinook helicopter force, so have some empathy with your situation and environment. For safety and security issues presented by the military operations environment when your graduates go to sea they have to be ready, able and willing to put their training in to practice in very difficult and urgent circumstances and to do so essentially 100% correctly - flying is similar in this respect, and so the emphasis on what/where to assess 7 evaluate, (and associated tools is) in my experience, a little different than a typical commercial situation/environment.

I'd be happy to converse off-line initially as it may prove quicker and easier to get into specifics which, whilst not presenting issues of national security, would be very specific to the military environment (and therefore perhaps a little inaccessible or even boring(!) for many readers here). I'd be happy to summarise for the wider community afterwards.

Best wishes,

Martin

I think you will agree with me that there is no one instrument or approach to use that will solve your challenge without serious massaging. at best.The real good news is that you can and should structure your own. Some suggestions towards that end have a serious talk with the individuals doing or supervising what performance is to be measured(they set the performance goals not you). What performance level now and in the future is required in your changing environment.What is performance in this case How does this desired performance fit into the overall picture and the overall goals of the situation?
If all this sounds a bit vague or you thought of it already, ok. Understand the people, goals and how this fits into the overall scope of things and the evaluation will become clearly evident.
You can do it yourself. In fact it would be a better.

Nero

Hi Sub - it seems to me that you have a good handle on the data you are looking to gather:

"I understand they are looking to expand the sphere of what they are looking at, such as time to achieve competency, cycle times, and how they can apply the resultant data to improve courses of instruction."

but I would echo Neros input - at this early stage in your proposed changes, I feel it is very important to set up semi-structured interviews with an audience sample. I would look to draw out peoples anecdotal evidence of performance improvement (or lack of). Look to draw out their perceptions of same. From this series of appropriate interviews should come a better understanding of where to do some quantitative research.

I have used this approach in the past and used the following (somewhat warm and fuzzy) questions:

- How long did it take you to FEEL ON TOP of your job/new product?

- As you looked around you, was there anyone really flying ahead in their mastery? What do you think made it so?

- As you looked around you, was there anyone really flying falling behind in their mastery? What do you think made it so?

- If you were in my shoes, what three things would you change in the current training programme to get better and faster results?

These are just some sample "qualitative" questions.

You can find more case studies on approaches to evaluation etc. at the TRAINING EVALUATION part of my website:

Training Evaluation

Good Luck, Mike Collins.