Subject eligibility criteria can substantially influence the results of alcohol-treatment outcome research

Objective: Most alcohol-treatment studies exclude some patients from participation based on particular criteria (e.g., comorbid illegal drug abuse, homelessness). The current study evaluated whether such eligibility criteria can change the outcome results a study obtains. Method: Five widely used treatment research eligibility criteria-(1) psychiatric problems, (2) medical problems, (3) social-residential instability, (4) low motivation/noncompliance, and (5) drug problems-were applied to two samples of real-world alcohol patients whose outcomes were known. Comparing outcomes of the samples with and without the application of eligibility criteria produced estimates of bias in outcome results, as well as an assessment of change in statistical power. Results: Medical and psychiatric eligibility criteria produced a moderate bias in outcome estimates (e.g., a 10% or less change in outcome results). In contrast, social-residential instability, low motivation/noncompliance, and drug use produced a large (e.g., up to an 18% change) to a very large (e.g., up to a 51% change) bias in outcome estimates. Sensitivity analyses showed that these biases are even larger if eligibility criteria are operationalized in a broad rather than a narrow fashion. Contrary to expectation, eligibility criteria did not produce their theoretically expected benefit of increased statistical power. Conclusions: Researchers who use eligibility criteria should do so judiciously and interpret outcome results in light of potential bias introduced by the ineligibility of some patients for study enrollment. Efforts to integrate findings across treatment outcome studies should also consider how conclusions might be affected by the eligibility criteria used in different research areas.