• Multiple choice test can be use to test a great variety of instructional objectives.
• It takes a great deal of time to write quality multiple choice questions.
• The danger is that most of the questions fall into the knowledge/recall category.

Writing the Questions

You need a quality STEM—a quality stem is one in which the students are able to read the stem and formulate a tentative answer even before reading the answer options. The stem should identify only one point that is being tested

A stem may be an incomplete sentence

Example: Of the following coefficients of correlations, the one with the least predictive value is…

OR

Written as a direct question it would read:  Which of the following coefficients of correlations has the least predictive value?

Writing the Stem

• Phrase the question in the form of a completion or short answer question.
• Think of incorrect responses students would likely make to the question.
• The stem should present the problem in the simplest form consistent with precision and clarity.
• Always present a verb in the stem.
• Do not pad the stem with superfluous material this only adds to students reading time.

Examples of Poorly Written Stems

The mean

1. Is the most frequently occurring score in a distribution

2. Corresponds to the 50^th percentile in the distribution

3. Is the arithmetic average of the scores

This question requires too much reading time for the student.

Rewording the stem to present a problem and save the student reading time

The mean of a distribution of tests scores is the

1. Most frequently occurring score
2. 50^th percentile
3. Arithmetic average

Well Written Stem

Suppose you thoroughly and adequately examined a particular type of cell, using a transmission electron microscope, and discovered that it completely lacked ribosomes. You could then conclude that this cell type also lacked

1. A nucleus

2. DNA

3. Cellulose

4. Protein synthesis

 Additional Steps in Writing Stems

After writing the stem it is best to write the correct answer first and then the distractors. This will insure that adequate attention is given to writing one correct or clearly best answer.

State if you want the students to find the correct answer or the best answer. If it is to be the correct answer it must be correct beyond any question.

Instructors usually ask for correct answers when testing in math, grammar, statistics or spelling.

If a student is to find the correct answer they can oftentimes argue that another answer is also correct but when asked to find the best answer there is only one best answer even if all the answers are correct.

Writing Multiple Choice Question Distractors

• Make certain there is clearly one correct or best answer
• That the Distractors are plausible enough to attract students that do not know the material very well.
• If you can’t develop a sufficient number of plausible answers then do not use the question
• Using humor in the answers usually is just a give away to the students. It is a cue to ignore that answer.

Example

The founder of Ferris State University was

1. Ferris Buller

2. Ferris Wheel

3. Woodbridge Ferris

Make the Distractors fairly homogeneous. This will increase the need for the students to be discriminating in their choices.

Avoid giving irrelevant clues to the students. You want to measure their content and cognitive skill abilities not their test taking skills.

Examples of irrelevant clues

• Length clues—the longest answer is often the correct answer.
• Verbal association—using a word in the stem that also appears in the answers
• Grammatical clues

Example Grammar Clue

The coefficient of correlation…social studies test is called a

1. Validity coefficient
2. Index of reliability
3. Equivalence coefficient

Specific stems—these are modifying words or phrases that limit the meaning of sentences

Examples: all, never, always (associated with the incorrect answer) usually, typically maybe, sometimes (associated with the correct answer)

Positives and Negatives

Use positive statements if possible.

Negative statements can be confusing for students to interpret. Research suggests that because students are usually in the habit of searching for true statements, the use of negatives can introduce unwanted bias.

If you use negative wording call attention to it by underlining it.

All of the Above/None of Above

Use options like all of the above or none of the above rarely.

These Distractors are generally too easy.

If even one of the answer choices is recognized as being incorrect then the student also knows that all of the above is incorrect

All of the above maybe a proper use if the instructor is trying to determine if the students have learned all of the relevant characteristics or attributes of a phenomenon

Using none of the above is especially difficult if you are asking students to find the correct answer, as it may be easy to argue that at least one of the answers was correct in some way.

It should be noted that research by Frary 1991 indicates using none of the above as an answer option can increase the difficulty of a question if use properly.

An item in which the distractor is not chosen by any student should be eliminated and replaced is chosen more often than any other option, including the answer especially if most students chose the same wrong answer.

Also eliminate items which all students get right as they do not help to discriminate between students and should be replaced with a more difficult item.

Additional Considerations when Writing Questions

• Item independence. Getting the correct answer to one item should not be contingent upon getting the correct response to other items.
• Avoid letting one answer provide a clue to another answer.
• Arrange the options (answers) in a logical order (alpha order or if numbers in ascending order).
• The correct response choice (a, b, c, or d) should be equally divided Tell the students that you have randomized the answers—this reduces any advantage test wise students may think they have.
• If the items are controversial site the authority whose opinion is being used…According to my lecture or In Freud’s opinion…
• Avoid lifting stems verbatim from the text…this encourages students to memorize rather than fully understand the material.
• Arrange the answer options in vertical columns. This makes the reading easier and less confusing

Summary Checklist for Writing Multiple Choice Items

• Make sure the item measures significant concepts and principles: do not write items covering trivia.
• The stem should present a problem; thus, a verb is necessary in the statement.
• State the item clearly and concisely and include only relevant material.
• Include as much of the item material in the stem; do not repeat words or phrases in each distracter that could be put in the stem one time.
• Write one correct or clearly best answer and three or four plausible distracters
• Avoid giving clues to the right answer; some common clues are grammatical, some involve length of the options, and some use specific determiners.
• Use positively stated stems if possible; otherwise call the students’ attention to the negative.
• Use all of the above and none of the above only rarely, and use none of the above only when asking the students to find the correct answer.
• Place the correct answer in each of the alternative positions (a, b, c, d) an equal number of times and in random order.

Faculty wanting further information about any of these topics are encouraged to contact Terry Doyle at doylet@ferris.edu

      Under Construction -- Watch for Updates