Improving content validation studies using an asymmetric confidence interval for the mean of expert ratings

Content validity is often assessed using the mean rating of endorsement provided by content experts for each item of the test. The lack of normality of the sample mean rating poses a major obstacle to the estimation of how far the sample mean is expected to lie from the population mean that it estimates. To overcome this obstacle, this article applies a new asymmetric score confidence interval for the population mean of a rating scale variable to the process of content validation. The application of the score confidence interval to content validity ratings permits test developers to assess the expected stability of the sample mean, determine how close the sample mean is expected to be to the Population mean, and test specific hypotheses concerning the value of the population mean. The calculation of the score confidence interval is described, and a numeric example is provided. Finally, the score confidence interval is applied to a data set of content validity ratings to illustrate its application and the information it provides.