Room for improvement: a systematic review of the quality of evaluations of interventions to improve hospital antibiotic prescribing

Introduction: In 1999, the British Society for Antimicrobial Chemotherapy (BSAC) and Hospital Infection Society (HIS) convened a working party on optimization of antibiotic prescribing in hospitals. This study was undertaken in order to evaluate the current evidence base on the effectiveness of interventions to change antibiotic prescribing to hospital inpatients. Methods: We have systematically reviewed the literature from 1980 to identify interventions that alone, or in combination, are effective in improving antibiotic prescribing to hospital inpatients. The protocol was peer reviewed and has been published by the Effective Practice and Organization of Care (EPOC) Group of the Cochrane Collaboration (www.update-software.com/cochrane/). Results: We identified 306 papers, of which 91 (30%) met the minimum inclusion criteria for a Cochrane EPOC review. The reasons for exclusion were uncontrolled before and after design (141/306; 46%) and inadequate interrupted time series (74/306; 24%) with fewer than three observations before and after the intervention. Most of the rejected interrupted time series (ITS) studies had only one or two data points before the intervention with many (up to 15) after it. Only 15 (40%) of the 38 included ITS studies had a statistical analysis and 11 of these used inappropriate statistical tests (e.g. t-test of pre- and post-intervention mean data) rather than analysis of time trends. Regression analysis of the proportion of included studies by year of publication did show a significant positive correlation (R-2=0.7886). Nonetheless, of 47 papers published since 2000, only 19 (40%) met the minimum eligibility criteria. Conclusions: The majority of evaluations used fundamentally flawed methodology. There is limited evidence of improvement over time. These problems could be resolved if researchers and referees of protocols or manuscripts implemented the EPOC methodology.