Comparison of Mantel-Haenszel and Logistic Regression Techniques in Detecting Differential Item Functioning

The purpose of this study was to compare Type I error rates and power for Mantel-Haenszel (MH) and logistic regression (LR) tests for reference and focal groups with varying ability distributions, sample size and sample size ratios. In this study WinGen3 was used to simulate ability estimates and to generate response data. The two-parameter logistic item response model was used to generate the item parameters for both non-DIF and DIF items. The results showed that with equal group distribution, Type I error rates of MH and LR techniques had similar and near the nominal a level. On the other hand, with unequal group distribution, inflated Type I error rates were observed in MH and LR procedures. Furthermore, the highest type I error rates were observed when difference between ability distributions was high and sample size was large. Results also showed that conditions leading to type I error rate inflation also result in artificially enhanced power rates.