I agree, Robert. BTW since eve has bold and italics, all caps is kind of annoying...

quote:

To quote a favorite (albeit old) book (ahem...) --

"Ratings can be done very quickly. The faster they are done, the more useless they are."

You show such very good taste in quoting such a wise and knowledgable author (ok, for those that don't know it's from one of my books on performance management, don't know which one to be honest).

...but actually, the quote isn't completely accurate -- it's a useful fiction that encapsulates a more complex proposition into a less complex proposition so it can be assimilated.


errr.....well ok, a soundbyte.

The reality is that ratings themselves are neither good or bad, useful or not. What determines their value is how they are used or misused in light of their limitations, and the purposes for using them.

So, I lied. It's the same for ratings on evaluations sheets, although I don't recall us talking only about ratings here. If I use a scaled instrument, I always invite short written comments to explain, and in any event we debrief at the end verbally, so when I take a look at all the information, I get a pretty good idea of what I need to change.

Have been following this thread for a while now, and feel inclined to jump in with a practical example that perhaps others can use.

When I'm developing a level 1 survey, I use the same approach that Robert describes above - a handful of rating questions and then three standard questions that require a bit of thought:

What was the best part of the course?

What was the least useful part of the course?

What would you suggest be done differently?

I always pay more attention to the responses from the last three questions than to the numerical ratings on the other questions. And I'll admit I tend to make changes mostly based on the last question.