Planning a Test

Two Vital Parts:

1. The content, skills or processes that will be covered on the test
2. The cognitive skills to be measured-Bloom’s Taxonomy

• Knowledge
• Comprehension
• Application
• Analysis
• Synthesis
• Evaluation

Test Format

Recognition Type Items

• Multiple choice
• True and false
• Matching
• Fill in the blanks

Essay Items

Short Essay—Focus on one aspect of an idea or topic

Long Essay—Focus on several aspects of an idea(s) or topic(s) with more depth

The decision as to which format to use should be made based on the outcomes you are seeking to measure.

Example: If you want to know if the student can develop an argument or devise a research plan than the essay is appropriate

To measure a student’s ability to recall factual knowledge, comprehension, or the ability to analyze or apply principles, a recognition test like multiple choice is appropriate.

Other Factors that Influence what Kind of Test to Give

1. Size of your class
2. Time availability
3. Scoring or checking of the test
4. What has been taught/stressed
5. The cognitive level of response expected

How Long Should the Test Be?
Test length is mostly influenced by the time available

Rule of Thumb

• True and false ------- 30 seconds each
• Multiple choice ------ one minute each
• Completion items ---- one minute each
• Short answer items ---- two minutes each
• Multiple choice requiring higher level thinking -- 90 seconds
• Matching items -------------- 30 seconds each
• Short essays ----------------- 10-15 minutes                                                                
• Extended essay ------------- 30 minutes (2-3 pages)

Other Time Considerations

1. A reasonable number of items for a fifty-minute multiple choice test is 50 items, for a true or false test it would be 80-90 items.
2. The fastest student will typically finish a test in about half the time as the slowest    student.
3. Keep in mind the greater the number of items the better the reliability of the test.
4. However, the test should allow almost every student to attempt every question.

What about using an optional items or student choice test?

Permitting students to choose which of several test questions to answer is not considered sound measurement practice. The test actually becomes several different tests and therefore you are not evaluating each student on the same basis. Also, this type of test may cause students to not study all of the material.

Test Item Difficulty

If the test is a norm reference test, ideally the test should produce a reasonably wide spread of scores as you seek to discriminate between students abilities.

If the test is criterion referenced then the difficulty needs to reflect what was taught at the level of understanding that was expected.

The goal is to have test items that are of average difficulty—students that have prepared should be able to do well—students that have not should do poorly.

General Guidelines for Test Item Development

1. Test for important ideas and skills—not trivial details—continually ask the question, "What knowledge, ability or skills are most worthwhile for students to know?"
2. Write items as simply as possible, making certain that the students know exactly what information is being requested—It is easy to be confusing because teachers know the information so well that they often do not recognize what they take for granted or assume students know
3. Make items appropriate for the age and ability levels of the students. The cognitive developmental level of students needs to be taken into account as well as vocabulary, cultural and other background information. Students that are dualistic thinkers will have great difficulty with questions that require them to deal with situations which may have many possible answers or solutions.
4. Make certain that objective questions have only one correct or one best answer. Ask students for the best answer—as some very effective distracters can have elements of correctness to them.
5. Avoid using interrelated items—knowing the answer to one item should not be necessary to answer subsequent items otherwise you are giving undue weight to the first item and make answering any of the next items seem impossible.
6. Avoid irrelevant clues and "give away" items—subject verb agreement, grammatical clues, similar words in the stem of the question as in the answers, etc.
7. Avoid using direct questions from the text—taken out of context they can be misleading---students may use rote memory rather than develop a full understanding of the material for the test.
8. Have someone else review the test before giving it to the students—they can often see the lack of clarity, or unintended clues.

Trick Questions as Perceived by Students
(Roberts 1993)

1. Intent to Trick: The wording of the item may reflect the intent to confuse or mislead on the part of the teacher
2. Trivial content: The question focuses on a point that the student perceives to be insignificant
3. Requires discrimination that is too fine: Requires the student to make distinctions that are much finer than those that were made in class or given as examples
4. Item stems that include irrelevant information—the central question is obscured by information that is completely irrelevant
5. Several correct answers: The differences between answers are so subtle that knowledgeable students interpret them as having more than one correct answer
6. Opposite principle: The question is stated in a way that requires the student to look for the answer in a way that is opposite of how it was presented in class
7. Highly ambiguous items: So poorly worded that it is unclear what the question is really asking

Tests writers should focus on significant ideas that are worth learning and remembering and on meaningful distinctions that are worth making. The items should be written in a manner that is clear and straightforward, so the students who answer incorrectly do so because they did not know the answer and not because they were confused or mislead.

Developing Test Items

Recommendation is to use items that 50-70% of the students can answer correctly.

The items should be difficult enough that students that did not study will get it wrong but students that did study will get it right.

A few items may be used to challenge the upper end of your students, but the general rule is that the items should be of average difficulty.

What makes an item difficult?

• The content it asks about
• The cognitive skill being measured
• The students’ learning experience
• The construction of the item itself

How Often Should I Test?

• Determined by the difficulty of the material being covered in the class
• The amount of material being covered in the class
• The level of the class/100,200 etc.
• Too few tests deprive students and instructors of the opportunity for feedback
• Too few tests lead to a lack of motivation on the part of the students

Can You Give Too Many Tests?

• Yes—Too many tests impinge on the amount of class time available for teaching
• Takes up too much of the teacher’s time checking and preparing test

Testing Recommendations

A gain in final exam scores has been shown to occur in classes where there is a frequency of testing 3-5 tests per semester

Also, giving 3-5 tests provides a more reliable basis for assigning grades

The students’ attitudes are better in courses where there is more frequent testing

It is very important to test early in the class. This gives the students an indication of what the teacher sees as important. It also gives them feedback on how well they are grasping the material at an early enough time when remedial assistance can be beneficial.

Faculty wanting further information about any of these topics are encouraged to contact Terry Doyle at doylet@ferris.edu

      Under Construction -- Watch for Updates