Real-world Antimicrobial Stewardship Experience in a Large Academic Medical Center: Using Statistical and Machine Learning Approaches to Identify Intervention "Hotspots" in an Antibiotic Audit and Feedback Program

Prospective audit with feedback (PAF) is an impactful, but resource-intensive, antimicrobial stewardship activity. Evaluating >17 000 PAF reviews, we built machine learning and statistical prediction models that substantially reduced review caseloads while maintaining high sensitivities, including with minimal or no automation. Background Prospective audit with feedback (PAF) is an impactful strategy for antimicrobial stewardship program (ASP) activities. However, because PAF requires reviewing large numbers of antimicrobial orders on a case-by-case basis, PAF programs are highly resource intensive. The current study aimed to identify predictors of ASP intervention (ie, feedback) and to build models to identify orders that can be safely bypassed from review, to make PAF programs more efficient. Methods We performed a retrospective cross-sectional study of inpatient antimicrobial orders reviewed by the University of Maryland Medical Center's PAF program between 2017 and 2019. We evaluated the relationship between antimicrobial and patient characteristics with ASP intervention using multivariable logistic regression models. Separately, we built prediction models for ASP intervention using statistical and machine learning approaches and evaluated performance on held-out data. Results Across 17 503 PAF reviews, 4219 (24%) resulted in intervention. In adjusted analyses, a clinical pharmacist on the ordering unit or receipt of an infectious disease consult were associated with 17% and 56% lower intervention odds, respectively (adjusted odds ratios [aORs], 0.83 and 0.44; P <= .001 for both). Fluoroquinolones had the highest adjusted intervention odds (aOR, 3.22 [95% confidence interval, 2.63-3.96]). A machine learning classifier (C-statistic 0.76) reduced reviews by 49% while achieving 78% sensitivity. A "workflow simplified" regression model that restricted to antimicrobial class and clinical indication variables, 2 strong machine learning-identified predictors, reduced reviews by one-third while achieving 81% sensitivity. Conclusions Prediction models substantially reduced PAF review caseloads while maintaining high sensitivities. Our results and approach may offer a blueprint for other ASPs.