Choice of units of analysis and modeling strategies in multilevel hierarchical models

Hierarchical models are common in complex surveys, psychometric applications, as well as agricultural and biomedical applications, to name but a few. The context of interest here is meta-analysis, with emphasis on the use of such an approach in the evaluation of surrogate endpoints in randomized clinical trials. The methodology rests on the ability to replicate the effect of treatment on both the true endpoint, as well as the candidate surrogate endpoint, across a number of trials. However, while a meta-analysis of clinical trials in the same indication seems the natural hierarchical structure, some authors have considered center or country as the unit, either because no meta-analytic data were available or because, even when available, they might not allow for a sufficient level of replication. This leaves us with two important, related questions. First, how sensible is it to replace one level of replication by another one? Second, what are the consequences when a truly three- or higher-level model (e.g., trial, center, patient) is replaced by a coarser two-level structure (either trial and patient or center and patient). The same or similar questions may occur in a number of different settings, as soon as interest is placed on the validity of a conclusion at a certain level of the hierarchy, such as in sociological or genetic studies. Using the framework of normally distributed endpoints, these questions will be studied, using both analytic calculation as well as Monte Carlo simulation. (C) 2003 Elsevier B.V. All rights reserved.