Early mortality from off-pump and on-pump coronary bypass surgery in Canada: A comparison of the STS and the EuroSCORE risk prediction algorithms

Abstract

OBJECTIVE:

Early mortality from off-pump and on-pump coronary artery bypass graft (CABG) surgery was assessed and compared with two widely used risk algorithms for CABG: The Society of Thoracic Surgeons (STS) and the European System for Cardiac Operative Risk Evaluation (EuroSCORE).

METHOD:

From March 12, 2001, to December 31, 2002, 1657 consecutive patients were treated with off-pump CABG and 1693 consecutive patients were treated with on-pump CABG. The predicted risk of mortality scores for the STS and EuroSCORE models were calculated. The predictive accuracy for early mortality was assessed by comparing the observed and expected mortalities for equal-sized quantiles of risk using the Hosmer-Lemeshow goodness-of-fit test. The discriminatory power of the models was evaluated by calculating the area under the receiver operating characteristic (ROC) curves.

RESULTS:

The observed postoperative mortality was 1.8% (95% CI 1.3% to 2.4%) for off-pump CABG and 1.5% (95% CI 1.1% to 2.1%) for on-pump CABG. For both on-pump and off-pump CABG surgery, the Hosmer-Lemeshow goodness-of-fit test indicated good accuracy. The area under the ROC curve was 0.81 (95% CI 0.73 to 0.90) for the STS and 0.79 (95% CI 0.71 to 0.88) for EuroSCORE in off-pump CABG (P=0.567). The area under the ROC curve was 0.82 (95% CI 0.73 to 0.91) for STS and 0.81 (95% CI 0.71 to 0.90) for EuroSCORE in on-pump CABG (P=0.616). The STS-predicted risk of stroke, prolonged ventilation and renal failure were similar to the observed data, with relatively good discriminatory powers for both off-pump and on-pump CABG.

CONCLUSION:

Both the STS and EuroSCORE risk algorithms are good predictors of early mortality from off-pump or on-pump CABG surgery. However, the generalizability of these results in the Canadian context would require a broader sampling of Canadian centres, including ones that provide both on-pump and off-pump CABG.

The comparison of crude mortality among different surgeons, institutions, regions or countries is misleading, because mortality is affected by patients’ preoperative characteristics and differences in health care patterns. To ensure accurate comparisons, mortality data must be adjusted to patients’ risk profiles. Various models have been developed for use in cardiac surgery, and among these, the most recent ones are The Society of Thoracic Surgeons (STS) (1) and the European System for Cardiac Operative Risk Evaluation (EuroSCORE) (2). The accuracy and discriminatory ability of the STS and EuroSCORE risk algorithms have been compared and validated with other statistical models (3–6). They have been applied to predict early mortality after coronary artery bypass graft (CABG) surgery in the United Kingdom, the United States of America and Sweden (2,6–8). These models have been shown to be a good predictor of early mortality after CABG surgery in these countries. The present study is the first to apply these risk algorithms to the off-pump CABG technique and to the Canadian CABG population. The objective of the present analysis was to determine the ability of these models to predict early mortality after off-pump and on-pump CABG surgery in the Canadian adult population.

METHODS

This was a prospective, multicentre study of isolated CABG surgery. Between March 12, 2001, and December 31, 2002, 1657 consecutive patients undergoing off-pump CABG were entered into the Canadian off-pump CABG registry, and 1693 consecutive patients undergoing on-pump CABG were entered into the Hamilton Health Sciences (HHS) cardiac surgery database.

The Canadian off-pump CABG registry

The Canadian off-pump CABG registry is composed of 13 large cardiac centres and 42 surgeons across Canada. The centres were selected based on their high volume of CABG surgeries performed, as well as on individual surgeon experience with off-pump CABG surgery. The research ethics board at each cardiac centre approved the study. Informed consent was obtained from all patients. Between March 12, 2001, and December 31, 2002, 1657 consecutive patients undergoing isolated off-pump CABG were entered into the registry across Canada.

The HHS cardiac surgery database

The HHS cardiac surgery database has maintained the information of all patients undergoing cardiac surgery, including isolated CABG surgery, since 1997. Between March 12, 2001, and December 31, 2002, 1693 consecutive patients undergoing isolated on-pump CABG were included in the study. Eight well-experienced surgeons performed the on-pump procedures. The research ethics board at the HHS approved the study, and informed consent was obtained from all patients.

Surgical technique

Participating surgeons decided on the type of surgical procedure, as well as on the conduits used and vessels grafted. Management of all patients followed standard practice guidelines from surgery to discharge. This included admission to the intensive care unit from the operating room, with subsequent transfer to the intermediate care unit (step down) and ward, or as determined by routine care in respective hospitals. An off-pump CABG procedure was defined based on a reported time of ‘zero’ for cardiopulmonary bypass (CPB) and ‘no’ use of cardioplegia. Conversion from off-pump to on-pump CABG was defined as occurring when a patient was placed on CPB during surgery.

Data collection

To ensure the consistency and standardization of data collection, data were collected following guidelines and definitions outlined in the operations manual specified for the study. Baseline information was collected prospectively for all patients. Also recorded were the date of surgery and discharge, conversion to CPB, location and number of grafts completed, peri- and postoperative use of an intra-aortic balloon pump, mechanical ventilation support for longer than 24 h, in-hospital mortality, and surgical and postoperative complications. To ensure consistency and standardization in data collection, data forms for the off-pump registry were adapted from the HHS cardiac surgery database after making further adjustments to the HHS database.

Detailed methodology, including the definition of patient populations, participating centres, surgical procedures, sample size calculations, data collection, and early and follow-up findings have been described elsewhere (9,10).

Definitions

Definitions and criteria for outcomes measured were uniform for all centres, and were confirmed by diagnostic procedures, radiological results, laboratory tests (enzyme measures), surgeons’ consultation notes, and hospital flow charts or discharge summaries during hospitalization. Preoperative renal disease was defined as the most recent serum creatinine levels being higher than 200 μmol/L, or as a patient being on dialysis at the preoperative visit. Mortality was defined as ‘a patient dying after surgery from any cause (cardiac- and noncardiac-related)’. Stroke was defined as a patient having a ‘neurological deficit at neurological examinations or discharge’. Acute renal disease was described as a patient ‘having documented renal insufficiency or going on dialysis after surgery’. Discharge from hospital was defined as a patient’s ‘stay in hospital from the time of surgery to the time of discharge from the institution at which surgery was performed’.

Risk scores

To calculate the STS risk mortality scores, variables were imported into the commercially available STS risk stratification software (Windows version NT/2000, Outcomes clinical data management system, Armus, USA). The EuroSCORE risk mortality scores were calculated using the EuroSCORE logistic algorithm (2,11,12). The risk factors, along with their corresponding definitions, have been described by Nashef et al (2). Some of the risk factors, such as a critical preoperative state, needed to be developed by combining one or more of the collected variables.

Statistical methods

Homogeneity analysis was applied to evaluate surgeons’ skill variability within the registry and within the HHS database with respect to the patient’s postoperative survival. Data were analyzed separately for the off-pump CABG registry and HHS on-pump CABG database. We used the intention-to-treat approach for data analysis, so if off-pump patients were converted to CPB, they were included in the off-pump CABG group. Categorical data were reported as percentages and 95% CI using Wilson’s method. Continuous data were reported as mean ± SD. Logistic regression analysis was used to determine the accuracy and discrimination of the two risk algorithms on the observed mortalities (death from the time of surgery to discharge). Predictive accuracy was assessed by comparing the observed and expected mortality for equal-sized quantiles of risk by the Hosmer-Lemeshow goodness-of-fit test (7). The analysis was adjusted for centre differences in in-hospital mortality among off-pump CABG registry patients. The discriminatory power of the logistic regression model was evaluated by calculating the area under receiver operating characteristic (ROC) curves. An area of 1.0 under the ROC curve indicates perfect discrimination, whereas an area of 0.50 indicates complete absence of discrimination. Any intermediate value is a quantitative measure of the ability of the risk predictor model to distinguish between survivors and nonsurvivors (7). The areas under the two resulting ROC curves were compared using a nonparametric approach with Stata, version 8.1 (Stata Corp LP, USA) (13). Data were analyzed using SPSS, version 14.0 (SPSS Inc, USA). All tests were two-sided at P<0.05.

RESULTS

Study patients

There was no significant variation in surgeon skills with respect to patient in-hospital mortality in either the off-pump CABG registry (P=0.962) or the on-pump CABG database (P=0.555). There was slight but not significant centre variation in in-hospital mortality within the off-pump CABG registry (P=0.058). One centre with a large experience in off-pump CABG accounted for 47% of the variation.

Risk scores

The risk factors used to calculate the STS and EuroSCORE risk stratification algorithms are listed in Table 1. The frequencies were 0% for aortic, mitral, tricuspid or pulmonary procedures, active endocarditis, cardiac shock, extracardiac arteriopathy, postinfarction septal rupture, previous valve surgery and procedures other than isolated CABG.

TABLE 1.

Proportion of risk factors in 3350 isolated coronary artery bypass graft surgeries (CABG) and variables included in The Society of Thoracic Surgeons (STS) and the European System for Cardiac Operative Risk Evaluation (EuroSCORE) algorithms

Risk factor	Baseline status	STS	EuroSCORE
Age, years (mean ± SD)	65.0±10.0	✓	✓
Female sex, %	24.7	✓	✓
Renal failure, %	8.2	✓	✓
Left ventricular ejection fraction, %
>50	67.7	✓	–
35–50	23.3	✓	✓
<35	9.1	✓	✓
Diabetes, %	30.2	✓	–
Previous MI, %	55.3	✓	–
Recent MI (<30 days), %	20.1	✓	✓
Smoking history, %	68.1	✓	–
Chronic pulmonary disease, %	9.6	✓	✓
Angina, %
Stable	32.5	✓	–
Unstable	56.5	✓	✓
Neurological dysfunction, %	9.6	✓	✓
Pulmonary hypertension, %	1.9	✓	✓
Hypertension, %	65.6	✓	–
Previous PTCA, %	13.4	✓	–
Previous intervention CABG, %	3.5	✓	✓
Congestive heart failure, %	11.9	✓	✓
Peripheral vascular disease, %	13.1	✓	✓
Hyperlipidemia, %	81.7	✓	–
Operative/postoperative IABP, %	2.1	✓	✓
Surgery status, %
Elective	46.2	✓	–
Urgent	51.3	✓	–
Emergent	2.4	✓	✓

✓ indicates the risk variables included in each risk alogortihm. IABP Intra-aortic balloon pump; MI Myocardial infarction; PTCA Percutaneous transluminal coronary angioplasty

Off-pump CABG

Of 1657 patients, 30 died after surgery, and the observed mortality was 1.8% (95% CI 1.3% to 2.4%). The Hosmer-Lemeshow goodness-of-fit test indicated good accuracy for predicting early mortality in off-pump patients for both models (P=0.467 for STS and P=0.337 for EuroSCORE). The area under the ROC curve (Figure 1) was 0.81 (95% CI 0.73 to 0.90) for STS and 0.79 (95% CI 0.71 to 0.88) for EuroSCORE. The discriminatory power (area under the ROC curve) was similar for STS and EuroSCORE (P=0.567).

Figure 1).

The receiver operating characteristic curve for off-pump coronary artery bypass graft surgery. The sensitivity of the prediction of early mortality is plotted against the 1 – specificity for The Society of Thoracic Surgeons (STS) and the European System for Cardiac Operative Risk Evaluation (EuroSCORE) risk algorithms. The area under the curve for the STS algorithm is similar to the EuroSCORE algorithm (P=0.567)

On-pump CABG

Of 1693 patients, 26 died after surgery, and the observed mortality was 1.5% (95% CI 1.1% to 2.1%). The Hosmer-Lemeshow goodness-of-fit test indicated good accuracy for the prediction of early mortality in on-pump patients for both models (P=0.363 for STS and 0.494 for EuroSCORE). The area under the ROC curve (Figure 2) was 0.82 (95% CI 0.73 to 0.91) for STS and 0.81 (95% CI 0.71 to 0.90) for EuroSCORE. The discriminatory power was similar for STS and EuroSCORE (P=0.616). The discriminatory power for STS (P=0.916) and EuroSCORE (P=0.850) risk models was similar for on-pump CABG and off-pump CABG.

Figure 2).

The receiver operating characteristic curve for on-pump coronary artery bypass graft surgery. The sensitivity of the prediction of early mortality is plotted against the 1 – specificity for The Society of Thoracic Surgeons (STS) and the European System for Cardiac Operative Risk Evaluation (EuroSCORE) risk algorithms. The area under the curve for the STS algorithm is similar to EuroSCORE (P=0.616)

The STS-predicted risks of stroke, prolonged ventilation and renal failure were similar to the observed proportions, with relatively good discriminatory powers for both on-pump CABG and off-pump CABG patients. The results are presented in Table 2.

TABLE 2.

Predicted versus observed postoperative complications based on The Society of Thoracic Surgeons (STS) algorithm

Events	STS prediction, n (%)	Observed, n (%)	Area under ROC curve (95% CI)
Off-pump surgery
Stroke	22 (1.3)	21 (1.3)	0.72 (0.60–0.84)
Prolonged ventilation	56 (3.4)	39 (2.4)	0.72 (0.64–0.80)
Renal failure	44 (2.7)	46 (3.1)	0.77 (0.70–0.83)
On-pump surgery
Stroke	22 (1.3)	26 (1.5)	0.72 (0.62–0.81)
Prolonged ventilation	59 (3.5)	58 (3.4)	0.77 (0.71–0.84)
Renal failure	39 (2.1)	40 (2.5)	0.73 (0.65–0.81)

ROC Receiver operating characteristic

DISCUSSION

The present study was the first to apply the STS and EuroSCORE risk algorithms to off-pump CABG patients and to Canadian CABG patients. Cardiac surgical risk assessment is important to patients and health care providers. It would be valuable to preoperatively predict which patients undergoing CABG surgery are at higher risk and require extra care. Health care providers can use this information to inform patients and their families about the expected postoperative course, and to make informed decisions regarding the appropriateness of surgery. The purpose of the present study was to assess the ability of the two widely used risk algorithms, developed from two large cardiac databases, in predicting early mortality in adult, Canadian isolated (off-pump or on-pump) CABG patients. The present study was the largest multicentre, prospective study of CABG surgery in Canada, and its results represent a routine daily surgical practice setting. The results showed that both risk algorithms have good predictive power (accuracy) of early mortality in off-pump or on-pump CABG surgery. The discriminatory power (area under ROC curve) was similar between off-pump and on-pump CABG procedures for both STS and EuroSCORE risk algorithms.

Both the STS national database and the EuroSCORE project were well-designed, high-quality studies. The quality assurance of data collection and risk stratification for proper assessment and outcome improvement in cardiac surgery holds significant importance to the investigators and originators of both studies (14). Both algorithms were developed to score early mortality in cardiac surgery (30 days after surgery for the STS and during hospital stay for EuroSCORE) (1,2,11,14,15). Most studies have applied and some have validated these risk algorithms in isolated CABG surgery (2,4,6,7). Although their findings support the results of the current study, the conclusion remains controversial. Nilsson et al (7) have applied the STS and EuroSCORE risk algorithms in Swedish CABG patients, and found that the EuroSCORE algorithm had a significantly better discriminatory power to predict early mortality than the STS risk algorithm. Bridgewater et al (6) have applied six different risk prediction algorithms on CABG surgery in the United Kingdom. They concluded that there were differences between the British and American CABG populations, and that the North American algorithms were not useful for predicting mortality in the United Kingdom. Other investigators (4) have recommended the use of the EuroSCORE model for predicting operative mortality after CABG surgery in the United Kingdom. Alternatively, Nashef et al (2) assessed the performance of the EuroSCORE risk algorithm in the STS database. They concluded that despite substantial demographic differences between Europe and North America, EuroSCORE performed very well in the STS database, with a discriminatory power of between 0.75 and 0.78 for overall cardiac surgery and isolated CABG surgery. We have tested these risk algorithms in isolated CABG surgery separately for off-pump and on-pump procedures. Our results revealed that both algorithms had good predictive power for both off-pump and on-pump CABG surgery. On the other hand, Riha et al (16) compared the outcomes of off-pump CABG patients classified as high- and low-risk according to the EuroSCORE risk model, and concluded that the EuroSCORE was a good predictor of off-pump CABG outcomes. Other studies (7,17) have reported that the predictive performance of EuroSCORE at higher risk scores is lower than at lower scores. Different factors, such as the small number of high-risk patients, the demographic differences of patients among different regions, differences in lifestyle and differences in the nature of data collection might have contributed to these findings. The fact that the risk factors for isolated CABG surgery are somewhat different than those for other open heart surgeries, particularly for valve surgery and combined surgery (18), might also have affected the prediction of mortality risk scores.

The set of STS postoperative morbidity risk models performed acceptably well on our data, off-pump as well as on-pump. The area under the ROC curve varied from 0.72 to 0.77. The predictive power of postoperative stroke, prolonged ventilation and renal failure was lower than that of mortality. Shroyer et al (8) assessed the STS mortality and morbidity risk models, and reported that the reliability of the STS risk algorithm on the prediction of morbidities needs further assessment.

The present study has certain limitations that are inherent in observational studies. First, the off-pump CABG registry was a multicentre study with different practice settings across Canada, and the HHS on-pump CABG database was from a single centre. This is the major limitation of the present study, with considerable risk of intercentre differences relating to practice patterns, referral patterns, socioeconomic status and health care patterns, because these characteristics may vary between regions in Canada. Second, the selection of surgical procedure (off-pump or on-pump) was based on surgeon preference, and this introduces selection bias. Third, biases such as a learning curve for the off-pump procedure, the technical ability of the surgeon performing off-pump CABG, and variation in surgeon experience within and between centres that influence the operative mortality could not be measured. The STS and EuroSCORE algorithms are also multicentre and self-registered databases; therefore, they are subject to all these biases. Due to these limitations and the fact that approximately only 10% of CABG surgeries are performed off-pump (19), we performed the analyses on off-pump and on-pump CABG cases separately. The comparison of the single-centre on-pump CABG data (as a sensitivity analysis) with the Canadian off-pump CABG registry may explain the effect of centre differences.

CONCLUSIONS

Both the STS and the EuroSCORE risk algorithms are good predictors of early mortality from off-pump or on-pump CABG surgery. These results further add evidence to the value and usefulness of developing large databases such as the STS and EuroSCORE. However, the generalizability of these results in the Canadian context requires a broader sampling of Canadian centres, including both on-pump and off-pump CABG surgeries.