Examining the normality assumption of a design-comparable effect size in single-case designs

What Works Clearinghouse (WWC, 2022) recommends a design-comparable effect size (D-CES; i.e., g(AB)) to gauge an intervention in single-case experimental design (SCED) studies, or to synthesize findings in meta-analysis. So far, no research has examined g(AB)'s performance under non-normal distributions. This study expanded Pustejovsky et al. (2014) to investigate the impact of data distributions, number of cases (m), number of measurements (N), within-case reliability or intra-class correlation (rho), ratio of variance components (lambda), and autocorrelation (phi) on g(AB) in multiple-baseline (MB) design. The performance of g(AB) was assessed by relative bias (RB), relative bias of variance (RBV), MSE, and coverage rate of 95% CIs (CR). Findings revealed that g(AB) was unbiased even under non-normal distributions. g(AB)'s variance was generally overestimated, and its 95% CI was over-covered, especially when distributions were normal or nearly normal combined with small m and N. Large imprecision of g(AB) occurred when m was small and rho was large. According to the ANOVA results, data distributions contributed to approximately 49% of variance in RB and 25% of variance in both RBV and CR. m and rho each contributed to 34% of variance in MSE. We recommend g(AB) for MB studies and meta-analysis with N >= 16 and when either (1) data distributions are normal or nearly normal, m = 6, and rho = 0.6 or 0.8, or (2) data distributions are mildly or moderately non-normal, m >= 4, and rho = 0.2, 0.4, or 0.6. The paper concludes with a discussion of g(AB)'s applicability and design-comparability, and sound reporting practices of ES indices.