Quantifying heterogeneity in individual participant data meta-analysis with binary outcomes

Background: In meta-analyses (MA), effect estimates that are pooled together will often be heterogeneous. Determining how substantial heterogeneity is is an important aspect of MA. Method: We consider how best to quantify heterogeneity in the context of individual participant data meta-analysis (IPD-MA) of binary data. Both two-and one-stage approaches are evaluated via simulation study. We consider conventional I-2 and R-2 statistics estimated via a two-stage approach and R-2 estimated via a one-stage approach. We propose a simulation-based intraclass correlation coefficient (ICC) adapted from Goldstein et al. to estimate the I-2, from the one-stage approach. Results: Results show that when there is no effect modification, the estimated I-2 from the two-stage model is underestimated, while in the one-stage model, it is overestimated. In the presence of effect modification, the estimated I-2 from the one-stage model has better performance than that from the two-stage model when the prevalence of the outcome is high. The I-2 from the two-stage model is less sensitive to the strength of effect modification when the number of studies is large and prevalence is low. Conclusions: The simulation-based I-2 based on a one-stage approach has better performance than the conventional I-2 based on a two-stage approach when there is strong effect modification with high prevalence.