Diagnostic Opportunities Using Rasch Measurement in the Context of a Misconceptions-Based Physical Science Assessment

Multiple-choice (MC) items that are constructed such that distractors target known misconceptions for a particular domain provide useful diagnostic information about student misconceptions (Herrmann-Abell & DeBoer, 2011, 2014; Sadler, 1998). Item response theory models can be used to examine misconceptions distractor-driven multiple-choice (MDDMC) items as a method for examining patterns of student answer choices given their achievement level. Furthermore, changes in response patterns over time can yield useful information about changes in student understanding following instruction. Because it meets the requirements for invariant measurement, the Rasch model is a promising tool for examining the diagnostic properties of MDDMC items within a coherent measurement framework; however, this application of the dichotomous Rasch model in conjunction with MDDMC items is relatively unexplored (Herrmann-Abell, 2011, 2014). The purpose of this study is to explore the diagnostic information about student understanding of physical science concepts provided by a Rasch-based analysis of MDDMC items. This study examines the relationship between misconceptions and a measure of student achievement as a method for targeting curriculum development and professional development within the framework of design-based implementation research (Penuel & Fishman, 2012). Implications for research, theory, and practice are discussed.