Statistical Comparison between In-Person and Online General Chemistry Exam Outcomes: A COVID-Induced Case Study

In spring 2020, the chemical education community facedan abrupttransition from in-person to online classes, which also necessitatedonline assessments. Building upon an existing three-semester study(F17, S19, and F19) using Rasch modeling and classical testing theoryto improve in-person multiple choice exams, this study investigatesthe impact of online exams (F20, F21, and F22) on assessment qualityand student performance in an undergraduate General Chemistry II class.The Cronbach's alpha and fraction of very good/good questionswere found to dramatically increase across the first two semesters(F17 and S19) and then largely plateaued for subsequent exams, regardlessof in-person or online test administration. Through the use of linkingquestions (i.e., repeated questions from semester to semester) andequating procedures, the results indicated that (1) there was notan obvious or uniform increase or decrease in the exam quality orstudent performance when switching from in-person to online examsand (2) there was no evidence for an increased prevalence of cheatingin the unproctored online exam relative to the prior in-person exams.While this data set is not sufficient to make any universal claims,this case study's outcomes suggest that concerns about increasedcheating on unproctored online exams are not inherently founded.