Bootstrap estimates of standard errors in generalizability theory

Estimating standard errors of estimated variance components has long been a challenging task in generalizability theory. Researchers have speculated about the potential applicability of the bootstrap for obtaining such estimates, but they have identified problems (especially bias) in using the bootstrap. Using Brennan's bias-correcting procedures (which extend the work of Wiley), as well as a proposed set of rules for picking a bootstrap procedure, the authors examined the potential utility of the bootstrap technique with multifacet designs in generalizability theory. Using six simulation conditions (normal, dichotomous, and polytomous data with the p x i x It and p x [i:h] designs), the rules proposed in this article were empirically demonstrated to perform well for estimating standard errors of estimated variance components and relative error variance with random models. No single bootstrap procedure performed well for estimating standard errors for absolute error variance, but a combination of bootstrap procedures was identified and empirically demonstrated to have performed well.