External Validation of Models for Predicting Pneumonia after Cardiac Surgery

Background: Approximately 20% of patients become infected after cardiac surgery. Pneumonia is one of the most serious infections, increasing the chance of death 14-fold. The higher frequency of pneumonia after cardiac surgery may be explained by surgical conditions. Focusing on high-risk groups may make several strategies more effective, and external validation is an essential phase of building prediction models to identify such groups. Purpose: To compare the performance of two previously validated prediction models for pneumonia after cardiac surgery (classification and regression tree [CART] and a logistic regression model [LRM]) on an external validation set. Methods: A series of 527 adult cardiac surgery patients at a small private hospital were analyzed prospectively to identify prognostic factors for pneumonia. Pneumonia occurred in 7.6% of patients in this derivation set. The probability of pneumonia onset was estimated by means of CART and LRM using a cut-off point that maximized both sensitivity and specificity without decreasing accuracy greatly. The results were confirmed with a validation set obtained by enrollment of consecutive 333 adult patients undergoing major cardiac surgery. There were significant differences in the fraction of emergency cases in the derivation and validation sets. Results: The LRM selected emergency surgery (odds ratio [OR] 5.28), chronic obstructive pulmonary disease (COPD)(OR 4.29), ventricular dysfunction (OR 2.68), and age (OR 1.04) as independent predictors of pneumonia. The CART model selected emergency surgery, age, unstable angina, body mass index, COPD, weight, and ventricular dysfunction as predictors. The CART model also selected low body mass index, weight, and unstable angina as predictors. Emergency surgery was the strongest predictor in both models. The LRM performed better than the CART model for the global, discrimination, and calibration measures. Conclusion: The LRM model displayed superior performance. A possible advantage of the CART prediction model is that it may be easier to interpret via its graphical presentation than prediction models based on logistic regression. However, there are a number of disadvantages of the CART approach. The LRM model can be used by infection control practitioners for risk adjustment across different periods or units and for evaluation of the efficacy of new technologies.