Examining Generalizability Aspect of Validity Using Differential Item Functioning

Generalizability is seen as an important aspect as it examines the extent to which scores and interpretations are consistent and can be generalized to population. This paper examined items in Research Methodology Test using Rasch approach of differential item functioning (DIF) to support generalizability aspect of validity. The analysis was done using Research Methodology Test (RMT) responded by two groups of postgraduate students. Based on results obtained, 6 out of 30 items in RMT have DIF size more than .43 logit but have DIF significant lower than 1.96 logit. Further judgment of subject matter experts is required to ensure that these items are really unfair to one group over another. Simultaneously, more extensive analysis required to examine generalizability aspect of validity.