Identifying Differential Performance in General Chemistry: Differential Item Functioning Analysis of ACS General Chemistry Trial Tests

The development of high-quality assessments can be an intensive process. At the American Chemical Society Examinations Institute (ACS-EI), this follows a general process of test design and content mapping, item construction, trial testing, item analysis, and final test setting. The item analysis portion of this procedure is an important step in using field-testing results to select the best items. This selection is based on validity analysis by experts, both field-test users and test writers, and by students in field testing. The traditional item analysis of the ACS-EI now includes a differential item functioning analysis with subgroups by gender when a sufficient data set is available. The results of this analysis from six trial tests were further evaluated by both content and format. Trends of specific content areas and by format of items reveal that classes of items favored one subgroup over another in at least four instances.