Hi,

I am currently in a tiff. I am finishing writing a monster training program which is five sessions long. I wrote the first session in December/January of last year. At that time I created a pre test. Well, it's post test time...I have reviewed the original test and have determined several questions (around 7 out of 25) are either no longer correct (e.g., the FLSA changed) or the infromation was skipped in class. So, in order to keep my post test valid, what do I need to do? shold I change the questions or take them off anf re- lly the pretest results to reflect the removed items? Any sugestions woudl be appreciated.

THANKS!

...and people ask me why I continue to preach against the focus on writing content and the overwhelming focus on recall-level questions...

You cannot post-test for a course using questions that cover information or skills that they did not learn via your course. On the other hand, if they were also required to read the updated information, then there is no problem. Don't worry about comparing pretests and post-test information because someone's ability to take tests before or after a learning experience is impacted by a LOT of factors beyond your course design.

stacyn;

Let's do a little root cause analysis first.

1. How is it that material covered by several questions was missed in class and not corrected when discovered?

2. Does the instructor teach from formal (approved) materials, such as an instructor guide or is he/she winging it from memory?

3. Do you have a process whereby the instructors are observed and evaluated on a periodic basis whereby their ability to deliver the required materials is determined and corrections made when deficiencies are seen?

These are just the first three questions I would ask when attempting to figure out the problem and more importantly preventing it from happening again.

As for the test at hand, Laura's right. You certainly cannot test on materials not provided to the students. My suggestion is to either eliminate the items from the test, or teach the materials now. Without knowing the complexity of your materials, have you considered teaching the missed information? Also, Laura's idea of assigning the missed info as reading is a good one as well.

The real question you need to ask yourself is: Do the students have to know/understand the materials covered in those 7 questions? If so, you are obligated to teach and test on that material before sending them back to the work environment.

I don't share Laura's dislike for recall level testing. In many business and corporate persuits, you must verify understanding of facts, figures, procedures, requirements, safety, precautions, etc. And that is required to be engrained in the memories of the employees and must be recallable. In the long run, you can agree with Laura or me regarding testing...that is, of course, your call.

Regards,

Del Laughery
President, Instructional Dynamics

thank you for your input. ALL is apreciated.

to attempt to answer questions and clarify, i will try to better explain the situation. we are a consulting company that works with government employees. the five session course is an optional program. the pre-test was written based on the information we "planned" on teaching. due to time constraints, the covered information was limited; ALL the information was included in the guide, but not all was taught in class. the particpants were not required to do any extra reading but were told that the content was there for their reference. (we could not require any additonal work since they are not our employees).

there is a formal instructor guide, but again time contraint limited the info taught - and teaching the info that was not taught is also not an option due to time constraints. ( we have learned a lot this year and will add time to the program next year).

the purpose of the post test is level 2 evaluation and not necessarily for the particpants benefit (i.e., it does not affect employment). the content, by the way, was on HR (recruitment and selection, comp and benefits, legal issues, ER, etc.). Many of the folks have HR responsibilities at their jobs, but none are HR managers. Therefore, much of the information does not directly relate to their current positions. it was for continuing education.

So, the bottom line is, the post test is to show; learning. 18 of the questions are good questions, 7 are not. the way i see it is i have three options:
1. leave the questions on - and probably throw them our of teh pre and post test when calculating learning.
2. take them off the post ( and thuis, only have 18 questions) and recaculate the pre test scores.
3. create 7 new questions - and still not count them and recacluate the pre without the 7 questions.

taking out your own opinions of whether testing is good or bad - what do you think is my best alternative?

quote:

verify understanding of facts

Dell -- verifying understanding is COMPREHENSION level, not RECALL level. There is a difference. Regurgitation of information (list this, name that, define this) is RECALL. Comprehension requires a bit more brain power and is much better. Not much better, but better than regurgitation. A student might be able to recite the preamble to the Constitution...doesn't mean for one moment that he/she understands it. Recitation - recall. Knowing what it means -- comprehension.