Comparative analysis of coronary surgery risk stratification models

Background. Preoperative risk assessment models for coronary bypass surgery (CABG) have been proposed, but comparison of them using independent databases needs to be done. Methods. Models of CABG hospital mortality were tested on a set of 3443 patients who underwent CABG including a subset of 3237 patients who had isolated CABG (no valve procedures), in our database since 1991. Four models previously described were designated as Parsonnet (PS), Cleveland (CL), and Society of Thoracic Surgeons version 1 (ST1) and version 2 (ST2). We developed our own Bayesian (BA) and logistic regression (LR) models and calibrated the PS and CL models on 2842 patients operated on prior to 1991. Models were compared with respect to 1) mean predicted mortality, 2) correlation of predicted to observed mortality, 3) Brier mean probability score, 4) descriptive statistics, 4) the C-Index (area beneath the receiver operating characteristic curve), and 5) predictive efficiency. Since the ST1 and ST2 models were developed for use only with isolated CABG patients, these models were compared with the others using an isolated CABG subset. Results. Observed mortality for all 3443 CABG patients was 4.0%. For this group, the mean mortality predicted by PS, CL, BA, LR, was 9.0 +/- 8.0, 6.0 +/- 6.0, 7.6 +/- 15.6, and 5.1 +/- 7.7 (mean +/- standard deviation) respectively. C-Indexes were .80 +/- .02, .80 +/- .02, .83 +/- .02, and .80 +/- .02 (C-Index +/- standard error) respectively. Observed mortality for 3237 isolated CABG patients was 3.7%. For this subgroup, the mean mortality predicted by PS, CL, BA, LR, ST1, and ST2 was 8.4 +/- 7.4, 5.7 +/- 5.9, 6.5 +/- 13.9, 4.5 +/- 6.5, 9.6 +/- 9.1, and 3.0 +/- 3.3 respectively. C-Indexes were .80 +/- .03, .80 +/- .03, .83 +/- .02, .79 +/- .03, .77 +/- .03, and .81 +/- .02 respectively. Conclusions. Existing CABG models can accurately discriminate outcome about 80 percent of the time. Models developed on a national database and those from non-local databases appear to have validity for our local data set. Predictions can vary widely between models and existing methods for comparing models appear to be inadequate. The methodology presented here is applicable for use with patients undergoing interventions in the cardiac catheterization laboratory.