Development and Validation of a Reading in Science Holistic Assessment (RISHA): a Rasch Measurement Study

Researchers in science education lacks valid and reliable instruments to assess students' disciplinary and epistemic reading of scientific texts. The main purpose of this study was to develop and validate a Reading in Science Holistic Assessment (RISHA) to assess students' holistic reading of scientific texts. RISHA measures students' content, procedural, and epistemic domains of reading two texts, one history-of-science text and another socio-scientific text. The initial 24-item RISHA was administered to 161 Grade 9 students from 3 schools. The multidimensional Rasch partial credit model was used to analyze the reliability and validity of RISHA. All items demonstrated good fit and reliability. According to logit scores generated for each domain in Rasch analysis, students in our study performed better in content domain and less well in the epistemic domain. Students also performed significantly better in the epistemic domain of the socio-scientific text than in the history-of-science text. RISHA provides accurate measures in various domains of reading scientific texts and various contexts of scientific texts. We propose that RISHA could potentially be applied to studying the effect of reading-science intervention or predictors of students' performance in each domain of reading scientific texts.