When writing quiz questions for e-learning courses (or any course), I've usually tried to stay away from using "All of the above" as an option in multiple choice questions. I've heard that in general there is a much higher chance of "All of the above" being the right answer when it's one of the choices presented. It seems that people rarely create quiz questions where they include "All of the above" as a possible answer, but the right answer is one of the other choices. Also, isn't there a psychological reason why the quiz taker might lean toward "All of the above" over other choices when they're not sure of the answer?

I don't have any statistics or data to support that up. Does anyone know of any studies, references, etc. that might back me up?

All of the above, none of the above or something like a and b, result from taking the easy road to writing multiple choice questions. You have three answers and just can't think of another. It's also true of quesitons that start with, which of the following is not..

Try and write four possible answers and you'll see it's not easy. However, I think you end up with a better test.

One of the things I don't like about multiple choice tests is that very creative or highly intelligent people will read your question in ways you hadn't intended. So you really need to test your questions to see which ones will be misunderstood.
About Learning Paths

Actually, there is an inherent scoring bias with "All of the above", "Choose all that apply", "A and B" and similar types of answers.

If a user chooses one of the answers, but not all of the above, aren't they at least partially correct?

I agree with LPC- unless it is truly a situation where a user must know "all of the above" (and this applies to "select all that apply" scored questions)- steer clear of these formats whenever possible.

Not easy, but writing good questions never is.

Colleagues,

Are quizes limited to multi-choice only ?

Nero

Hi IDKes:

Anthony, J. Nitko in his book Educational Tests and Measurements, An Introduction offers the following regarding using the All of the above option:

This option, if used at all, should be limited to “correct answer” varieties. It cannot be used with the “best answer varieties because all of the options cannot be simultaneously “best”. Two further difficulties arise: (1) pupils who know that the first option is correct may simply choose it and inadvertently go on the next item; (2) pupils who know that 2 out of 4 options are correct can choose all of the above without having knowledge of the correctness of the third option. This option can also be confusing to elementary and junior high students. Generally, the recommendation is to avoid using it. An item with multiple answers may be rewritten as two or more items.


Hope this helps,

Cj