Meta-analysis in public health: potentials and problems

Meta-analysis has evolved as a primary tool for evidence-based medicine. Initially, meta-analysis was seen as a technique that could improve statistical power in a research world of small, underpowered studies. We increasingly recognize that meta-analysis is a critical tool that can help us measure and understand not only summary effects, but also heterogeneity (diversity) and bias. Here I discuss some key themes and challenges for "meta-epidemiology". These include the contrast between randomized and observational evidence; the evolutionary nature of biomedical evidence; the contrast between small and larger studies; the difficulties in appraising study "quality" and its potential impact on the study effects; and the scandal of missing even minimal, key information on the harms of interventions that are otherwise postulated to be effective. I discuss a general outlook about the validity of the evidence in medicine and public health. I suggest that we should learn to live with uncertainty, since the evidence that is available is often limited, biased, or both. This means that we should be prepared to dismiss big chunks of biomedical dogma, including perhaps whole specialties and sub-specialties of current medicine, as false, erroneous, irrelevant or even potentially dangerous for the health of individuals and populations. An effort should be made to shift the accumulation and synthesis of evidence towards answering critical public health-related questions. This paper is based on a lecture presented at the European Public Health Association 2005 annual conference in Graz, Austria.