Distractor similarity and item-stem structure: Effects on item difficulty

This article examined two item-writing guidelines: the format of the item stem and homogeneity of the answer set. Answering the call of Haladyna, Downing, and Rodriguez (2002) for empirical tests of item writing guidelines and extending the work of Smith and Smith (1988) on differential use of item characteristics, a mock multiple-choice driver's license examination was administered to high school students with items having item stems that were either open-ended or in question form and with distractors structured to be either similar or dissimilar to the correct answer. Analyses at the test level indicated that the similarly structured distractors raised the mean difficulty level by .12. No effect was found for item-stem format. Differential item function analyses on each of the test items further supported the effect of distractor similarity on test performance. Implications of this study for item writing and standard setting, as well as implications for future research, are discussed.