Machine learning for developing a prediction model of hospital admission of emergency department patients: Hype or hope?

Objective: Early identification of emergency department (ED) patients who need hospitalization is essential for quality of care and patient safety. We aimed to compare machine learning (ML) models predicting the hospitalization of ED patients and conventional regression techniques at three points in time after ED registration. Methods: We analyzed consecutive ED patients of three hospitals using the Netherlands Emergency Department Evaluation Database (NEED). We developed prediction models for hospitalization using an increasing number of data available at triage, similar to 30 min (including vital signs) and similar to 2 h (including laboratory tests) after ED registration, using ML (random forest, gradient boosted decision trees, deep neural networks) and multivariable logistic regression analysis (including spline transformations for continuous predictors). Demographics, urgency, presenting complaints, disease severity and proxies for comorbidity, and complexity were used as covariates. We compared the performance using the area under the ROC curve in independent validation sets from each hospital. Results: We included 172,104 ED patients of whom 66,782 (39 %) were hospitalized. The AUC of the multi-variable logistic regression model was 0.82 (0.78-0.86) at triage, 0.84 (0.81-0.86) at similar to 30 min and 0.83 (0.75-0.92) after similar to 2 h. The best performing ML model over time was the gradient boosted decision trees model with an AUC of 0.84 (0.77-0.88) at triage, 0.86 (0.82-0.89) at similar to 30 min and 0.86 (0.74-0.93) after similar to 2 h. Conclusions: Our study showed that machine learning models had an excellent but similar predictive performance as the logistic regression model for predicting hospital admission. In comparison to the 30-min model, the 2-h model did not show a performance improvement. After further validation, these prediction models could support management decisions by real-time feedback to medical personal.