ASTD Homepage    ASTD Discussion Boards  Hop To Forum Categories  Evaluation & ROI    Level 1 Benchmark Scores
Go
New
Find
Notify
Tools
Reply
  
-star Rating Rate It!  Login/Join 
Posted
Hi, I'm trying to find some benchmarking on Level 1 evaluation scores. We are currently achieving a score of 6.3 out of 7 (90%) and need to report to our leadership how that compares to other companies. Any benchmarking information that you could point me to would be much appreciated. Thanks.
 
Posts: 5 | Location: El Dorado Hills, CA | Registered: September 08, 2008Reply With QuoteEdit or Delete MessageReport This Post
Posted Hide Post
No. Asking for such a thing is an indication of a lack of understanding regarding the purpose of a level 1 evaluation. Educate your stakeholders about the goal and purpose of getting learner feedback. Inform them that there is nothing meaningful in comparing the feelings of your participants to the feelings of participants in other organizations. Inform them that the feedback after a course simply allows participants to share their feelings about the learning experience and that internally, you are trying to ensure the best possible learning experiences, but that it makes no sense to attempt to benchmark that against some external company that has different business goals, etc. from you.
 
Posts: 90 | Registered: July 30, 2009Reply With QuoteEdit or Delete MessageReport This Post
Posted Hide Post
SSFN is absolutely, completely spot-on here. Do NOT compare your level one scores with other companies. Example: company A does courses on job hunting and resume writing to a group of employees who are about to be laid off (they've been notified of this). Their evaluation scores will be mediocre (b/c of the nature of their situation). Company B does courses to recently promoted employees preparing them for their new jobs which involve significant pay increases as soon as they complete the training. Those evaluations are significantly better. Training is done by same trainer (let's say, an outside consultant) with same standards of material. But level one evals will be different by company because of the circumstances of the "learners."

Furthermore, I would argue that using level one scores as an assessment of a training groups performance is very weak. Last year you averaged a 5.8 and this year you averaged a 6.2--does that mean you got better? Maybe you taught "funner" topics this year? Maybe you laid off a bunch of older, high paid workers and replaced them with recent college grads who've had no training, less to compare against and appreciate everything they get (so scores get elevated)? Comparing the scores of one type of course (say...leadership) to another (consulting skills or technical writing) is especially abusive since those involve different content, different groups, different motivation and in many cases, required vs. voluntary attendance.

Comparing level one scores is like comparing overall training attendance ("last year 52% of the employees attended training and this year 56% of the employees attended training so therefore we're doing a better job."). No, only unless there is a specific requirement about everyone attending training (like every driver needs to be certified or everyone has to attend the sexual harassment course in order to reduce your insurance liability fees).

Ultimately, I would argue that the best ways to demonstrate to management your worth would be ROI or proven performance in terms of results (what Kirkpatrick would call level IV). But in the short-term, here are some other ideas (especially in terms of benchmarking) that are far more worthy than comparing level I scores across courses and companies:
--demonstrate that you have a wider, deeper and more relevant range of training content than your competitors. For instance, competitor A just has a technical writing course (one size fits all) while you have content that differentiates on the basis of the writing and the role of the participant.
--speed of response: your ability to respond to client demonstrated performance needs. Particularily conversion of training content to job aids is impressive (faster delivery to the field, less cost in most cases).
--client satisfaction of impact (if you can't show ROI or demonstrated level IV than do an ROE analysis--see Toni Hodges' book for more details on this).
--generic vs. industry specific content. Competitors use generic, open-enrollment content and you use industry specific content.
--alignment with business priorities. Identify "Job One" in the company and then see what percentage of training content is specific to that business priority.
 
Posts: 250 | Location: Northern Virginia | Registered: February 24, 2004Reply With QuoteEdit or Delete MessageReport This Post
Posted Hide Post
jhopp01,

Your Company: 6.3/7 (90%)

My Company: 6.8/7 (97%)

Your company delivers training focused on building core competencies, is performance based, requires pre-work and practice sessions and is followed up with mentoring/coaching.

My company's programs actually stink. Off-the-shelf, targeted to nothing in particular, but are delivered in a brand new off-site conference center (can you say "jacuzzi").

My company's numbers are better than yours.


_____________________________________
www.commonwealthmetrics.com
 
Posts: 171 | Location: US | Registered: February 04, 2008Reply With QuoteEdit or Delete MessageReport This Post
  Powered by Eve Community  
 

ASTD Homepage    ASTD Discussion Boards  Hop To Forum Categories  Evaluation & ROI    Level 1 Benchmark Scores

© American Society for Training & Development (ASTD)
Linking People, Learning and Performance
Terms and Conditions
1640 King Street, Box 1443 . Alexandria, Virginia, 22313-2043, USA
Phone: 703.683.8100 . Fax: 703.683.8103