Development and Validation of a Machine Learning Model for Early Detection of Untreated Infection

BACKGROUND: Early diagnostic uncertainty for infection causes delays in antibiotic administration in infected patients and unnecessary antibiotic administration in noninfected patients. OBJECTIVE: To develop a machine learning model for the early detection of untreated infection (eDENTIFI), with the presence of infection determined by clinician chart review. DERIVATION COHORT: Three thousand three hundred fifty-seven adult patients hospitalized between 2006 and 2018 at two health systems in Illinois, United States. VALIDATION COHORT: We validated in 1632 patients in a third Illinois health system using area under the receiver operating characteristic curve (AUC). PREDICTION MODEL: Using a longitudinal discrete-time format, we trained a gradient boosted machine model to predict untreated infection in the next 6 hours using routinely available patient demographics, vital signs, and laboratory results. RESULTS: eDENTIFI had an AUC of 0.80 (95% CI, 0.79-0.81) in the validation cohort and outperformed the systemic inflammatory response syndrome criteria with an AUC of 0.64 (95% CI, 0.64-0.65; p < 0.001). The most important features were body mass index, age, temperature, and heart rate. Using a threshold with a 47.6% sensitivity, eDENTIFI detected infection a median 2.0 hours (interquartile range, 0.9-5.2 hr) before antimicrobial administration, with a negative predictive value of 93.6%. Antibiotic administration guided by eDENTIFI could have decreased unnecessary IV antibiotic administration in noninfected patients by 10.8% absolute or 46.4% relative percentage points compared with clinicians. CONCLUSION: eDENTIFI could both decrease the time to antimicrobial administration in infected patients and unnecessary antibiotic administration in noninfected patients. Further prospective validation is needed.