Consequences of Violated Equating Assumptions Under the Equivalent Groups Design

The equal ability distribution assumption associated with the equivalent groups equating design was investigated in the context of a selection test for admission to higher education. The purpose was to assess the consequences for the test-takers in terms of receiving improperly high or low scores compared to their peers, and to find strong empirical evidence of potential violations of the assumption. Test-takers' scores on anchor items from two subtests were estimated using information about test-taker performance on the regular subtests. The results indicated that the anchor test item performance varied sufficiently, both in terms of means and spreads. Therefore, the equal ability distribution assumption could be questioned. Also, the estimated differences between different cohorts of test-takers are large enough to have an impact on the actual admissions decisions. Consequently, our conclusion is that more caution is needed when applying the equivalent groups design in the equating of tests. Assuming equal ability groups is a convenient assumption to make but it can also lead to systematic bias in the equating of test scores with potentially severe implications for test-takers, and this study provides a demonstration of this point.