ASTD Homepage    ASTD Discussion Boards  Hop To Forum Categories  Evaluation & ROI    Basic Likert Scale
Page 1 2 
Go
New
Find
Notify
Tools
Reply
  
-star Rating Rate It!  Login/Join 
Posted Hide Post
Although research results are inconsistent as to which actually discriminates better, scales without a neutral point tend to have an upward bias. In other words, if you don't give people a neutral option, they lean toward the "nice" option. This isn't really a problem as long as you are aware of the issue and you aren't mixing scales.
 
Posts: 40 | Registered: April 05, 2007Reply With QuoteEdit or Delete MessageReport This Post
Posted Hide Post
Seems to me that's a problem with scales in general. Too easy, too general... no matter how carefully you word what the numbers stand for or how specific you get with the items to which the learner responds.

For example...let's take a typical smiley sheet item --

We'll use a scale from 1 - 5 (1 being strongly disagree, 5 being strongly agree and 3 being neutral)

The instructor was helpful and patient.

Let's say one grumpy person in class gives you a 1. I GUARANTEE that you already were aware of the grumpy person who didn't want to be there, so does it really help you to know on a smiley sheet that they didn't think you were helpful or patient? If people give you extremely high or low scores without explaining why, how will you improve what was bad or continue what was good? There is nothing constructive about numbers on a scale.

What about another typical smiley sheet item:

The materials were helpful and informative.

What of the typical scenario where instructors have little if any control over the classroom materials? And what of it if someone marks this low without explaining specifically what was not helpful or not informative? How can anybody improve materials based on responses to such a general question without any construcive detail?

Again, I must ask -- how many of you can honestly say that you always get specific, constructive feedback when you use scale-driven smiley sheets?

The only informative thing about these scales is if there is a very distinct trend for any ONE class. If EVERYONE or MOST of the people in the class give you a 1 for helpful and patient, then you know you probably need to examine yourself. Still, what is it you're supposed to examine? What did you do or didn't you do that gave people that impression? Without their specific information regarding WHY they gave you a 1, it makes it difficult to discern what, precisely, they responded to.

I once had a learner complain that my bangs were distracting. Sometimes, there are things that really don't deserve a response or "fix".
 
Posts: 890 | Registered: August 16, 2006Reply With QuoteEdit or Delete MessageReport This Post
Posted Hide Post
For Likert scales, in many academic circles, the preferred approach is to use a 7 point scale (or rating). Only the ends of the scale should be labeled with something like Strongly Disagree or Strongly Agree. Having the intermediate labels provides too much guidance for the raters.

Please feel free to email me if you would like additional detail or supporting research.

A word of warning on Level 1's, just because someone likes training (or is satisfied), doesn't mean it is effective. Level 1 surveys can be very deceiving. When left to their own devices, people will often pick the least effective training (based on a study of US Postal workers).
 
Posts: 6 | Location: Ohio | Registered: May 15, 2007Reply With QuoteEdit or Delete MessageReport This Post
  Powered by Eve Community Page 1 2  
 

ASTD Homepage    ASTD Discussion Boards  Hop To Forum Categories  Evaluation & ROI    Basic Likert Scale

© American Society for Training & Development (ASTD)
Linking People, Learning and Performance
Terms and Conditions
1640 King Street, Box 1443 . Alexandria, Virginia, 22313-2043, USA
Phone: 703.683.8100 . Fax: 703.683.8103