Meta-analysis of interrater reliability of job performance ratings in validity studies of personnel selection

Rating scales are most frequently used to assess the criterion in studies of validity in personnel selection. However, only a few articles report the interrater reliability for these scales. This paper presents four meta-analyses in which the interrater reliability was estimated for Civil Composite Criterion, Military Composite Criterion, Total Composite Criterion, and Single Civil Criterion. Mean reliabilities mere .64, .53, .62, and .40, respectively. The implications of these findings for single and meta-analytic studies are discussed.