Mortality prediction in Indian cardiac surgery patients: Validation of European System for Cardiac Operative Risk Evaluation II

Abstract

Background and Aims:

Risk Stratification has an important place in cardiac surgery to identify high-risk cases and optimally allocate resources. Hence various risk scoring systems have been tried to predict mortality. The aim of the present study was to validate the European System for Cardiac Operative Risk Evaluation II (EuroSCORE II) in Indian cardiac surgical patients.

Methods:

After obtaining ethics committee clearance, data on EuroSCORE II variables were collected for all patients >18 years undergoing on-pump coronary artery bypass graft (CABG), valve surgery and mixed (CABG + valve) procedures between January 2011 and December 2012. Mortality prediction was done using the online calculator from the site www.euroscore.org. The calibration of the EuroSCORE II model was analysed using the Hosmer–Lemeshow test and discrimination was analysed by plotting receiver operating characteristic curves (ROC) and calculating area under the curve (AUC). The analysis was done in the total sample, CABG, valve surgery and in mixed procedures.

Results:

The overall observed mortality was 5.7% in the total sample, 6.6% in CABG, 4.2% in valve surgeries and 10.2% in mixed procedures whereas the predicted mortality was 2.9%, 3.1%, 2.4%, 5.1% in total sample, CABG, valve surgery and mixed procedure, respectively. The significance (P value) of Hosmer–Lemeshow test was 0.292, 0.45, 0.56 and 1 for the total sample, CABG, valve surgery and mixed procedure, respectively, indicating good calibration. The AUC of ROC was 0.76, 0.70, 0.83 and 0.78 for total sample, CABG, valve surgery and mixed procedure, respectively.

Conclusion:

Mortality of the sample was under-predicted by EuroSCORE II. Calibration of the EuroSCORE II model was good for total sample as well as for all surgical subcategories. Discrimination was good in the total sample and in the mixed procedure population, acceptable in CABG patients and excellent in valve surgeries.

INTRODUCTION

Risk stratification in cardiac surgery has special importance in this modern era, as it helps in identifying high-risk cases and thus facilitates optimal allocation of resources. Therefore there has been a constant search for an ideal model of risk assessment and stratification for cardiac surgery. The present practice is to develop a risk stratification model in a particular population and then validate it in the same as well as in other populations. This may not give accurate results because of difference in patient demographics, treatment modalities, techniques and practice of medicine in various countries. European System for Cardiac Operative Risk Evaluation I (EuroSCORE I) was a commonly used risk stratification model in cardiac surgery.[1] It was modified so as to improve it's power to predict mortality. The newer modified EuroSCORE II identified 18 factors as risk predictors for mortality.[2] External validation of EuroSCORE II has been carried out and published from many countries. The earlier studies validating EuroSCORE II in India were from the urban population and had a higher percentage of patients undergoing coronary artery bypass and thus may not be representative of the general Indian population and valve surgery patients.[3,4] The present study was carried out to validate this European scoring system in general adult Indian patients undergoing on-pump CABG and valve surgery.

METHODS

After obtaining Ethics Committee Clearance, this retrospective study was carried out on patients who were operated between January 2011 and December 2012. All adults who underwent following procedures were included. (1) Isolated coronary artery bypass graft (CABG), (2) valve surgeries, (3) mixed procedures (CABG + valve). Patients with age <18 years, patients undergoing surgery for congenital heart disease, off-pump CABG, thoracic aortic procedures or procedures not listed in the original EuroSCORE II database were excluded from the study. Data acquisition was done from the electronic database of the department of cardiothoracic surgery, and details on the variables of EuroSCORE II were entered into Microsoft Excel sheet. Mortality if any was noted. Patients having missing data on any of the EuroSCORE II variables were also excluded from the study.

Mortality prediction was done for all patients using EuroSCORE II calculator from the site www.euroscore.org. Calibration and discrimination of EuroSCORE II were analysed in the total sample, isolated CABG, isolated valve surgery and mixed CABG with valve procedures.

The quantitative variables were expressed as a mean ± standard deviation. The qualitative variables were expressed as absolute frequencies and percentages. Hosmer–Lemeshow test was used for assessment of calibration of the EuroSCORE. This test compares the observed versus predicted mortality. If the test is significant the calibration is said to be poor. Non-significant test means good calibration. Discrimination refers to the capacity of a model to distinguish high-risk patients from low-risk patients. Discrimination was seen by plotting receiver operating characteristics (ROC) curves and calculating area under curve (AUC). Discriminative power is considered to be excellent if AUC is >0.80, good if >0.75 and fair (acceptable) if >0.70. Both the calibration and discrimination test were repeated for total sample, isolated CABG patients, valve surgeries and mixed procedures. All the statistical analysis was done using SPSS version 20 software (2011, IBM, Armonk, NY, United States of America).

RESULTS

The patient recruitment chart is shown in Figure 1. A total of 1084 patients with age >18 years were operated during the study period. Out of these, 64 patients underwent off-pump CABG, 38 patients were operated for congenital cardiac conditions, 10 patients had surgery on thoracic aorta and thus were excluded from this study. Insufficient data led to the exclusion of another 61 patients [Figure 1]. The final analysis included 911 patients who underwent either isolated CABG, valve surgery or mixed procedures (CABG + valve). CABG constituted 47.8%, valve surgeries accounted for 46.8% and mixed procedures for 5.4% patients. Isolated mitral valve surgery was done in 27.9%, isolated aortic valve surgery in 7.4%, double valve replacement in 9.99% patients. Tricuspid valve procedures either in isolation or in combination with other valves accounted for the rest (1.6%).

Figure 1.

Chart showing patient recruitment. PAH – Pulmonary arterial hypertension; CLD – Chronic lung disease; CCS IV – Canadian Cardiology Society Class IV angina

The details of EuroSCORE II predictors are given in Table 1. The mean age in our study was 49.37 ± 13.4 years, 56.9 ± 8.9 years, 41.19 ± 12.73 years and 54.88 ± 11.36 years in the total sample, CABG, valve surgeries and mixed procedures, respectively. Our dataset consisted of 33.5% females.

Table 1.

Details of European System for Cardiac Operative Risk Evaluation II risk variables

The predicted and observed mortality in the total sample and also in the isolated CABG sample, valve surgeries and in the mixed procedure, sample is listed in Table 2. EuroSCORE II underpredicted mortality in all the surgical subgroups. The data were also divided according to the predicted risk, into very low risk (<1%), low risk (1%–2.99%), moderate risk (3%–4.99%) and high risk (>5%). The risk category wise observed and predicted mortality is tabulated in Table 3. The result of Hosmer–Lemeshow goodness of fit test and the area under the curve (AUC) of ROC curves presenting discrimination of total sample, and the subgroups are tabulated in Table 4. The ROC curves are shown in Figure 2. EuroSCORE II test showed a good calibration in the total sample as well in all the subgroups. The AUC of ROC curves was 0.76 for total sample, 0.70 in CABG group, 0.83 for valve group and 0.78 for mixed group patients. The corresponding confidence intervals are tabulated in Table 4.

Table 2.

Surgery category wise observed and predicted death in the study cohort

Table 3.

Risk category wise observed and predicted mortality

Table 4.

Calibration and discrimination of European System for Cardiac Operative Risk Evaluation II

Figure 2.

Receiver operating characteristic curves with area under curve values. (a) Receiver operating characteristic curve for total sample, (b) receiver operating characteristic curve for coronary artery bypass graft, (c) receiver operating characteristic curve for Valve surgery, (d) receiver operating characteristic curve for mixed procedures. ROC – Receiver operating characteristic; AUC – Area under curve

DISCUSSION

The results of our study show that EuroSCORE II had a good calibration in all surgical subgroups, however, discrimination was good for the total sample and mixed procedures, fair for CABG patients and excellent for valve surgeries.

EuroSCORE II was published in 2011, as the earlier versions (additive and logistic EuroSCORE) had a tendency to overestimate the mortality.[2] EuroSCORE II database was created by collecting data from 22,381 patients (16,828 as developmental set and 5553 as validation set) across 154 units in 43 countries. Out of these, only 4 units were from India. It is understandable that the demographics, patient profile and contemporary surgical techniques and practices are different in the Indian subcontinent as compared to the rest of the world. Thus, the EuroSCORE database population is unlikely to be representative of Indian demographics. Because of similar concerns external validation has been carried out and published from UK, Spain, Italy Finland, Turkey and China.[5,6,7,8,9,10,11,12,13,14,15] In India, two earlier studies have attempted validation of EuroSCORE II. One study was from Western India and the other from North India.[3,4] These studies were carried out in urban centres and had a predominance of CABG surgery. The present study, however, is more likely to be representative of the general population as it was carried out in a government hospital catering to the patients from lower socioeconomic strata. Further, we must emphasise that our study sample had nearly equal number of CABG and valve procedures similar to that of the original EuroSCORE II database.

Age was found to be a significant predictor of mortality from 60 years onwards in the EuroSCORE II analysis.[2] However, the mean age in our dataset was significantly lower as compared to EuroSCORE II [Table 1] Females constituted 33.5% of our sample which is similar to EuroSCORE II database (30.9%), whereas in the study by Borde et al. on the only 19.89% patients were female.[3] We also observed that more male patients (80.5%) underwent isolated CABG whereas 47.8% females were operated for valve surgeries. The percentage of patients undergoing CABG and valve procedures was also comparable to EuroSCORE II dataset.

Number of patients having poor mobility was higher in our data set as compared to EuroSCORE II. We also observed that most patients (29 out of 56) presenting with poor mobility had a history of cerebrovascular accident (CVA), had mitral valve disease of rheumatic origin with left atrial appendage (LAA) clot and underwent mitral valve surgery. The presence of LAA clot is a well-known cause for CVA. In the Indian subcontinent, the major cause of valvular heart disease is still rheumatic in origin as opposed to the Western population where the degenerative valvular lesions are predominant.[16,17] The paucity of rheumatic mitral valve disease with LAA clot in the western population could be the cause for a lower number of patients with poor mobility in EuroSCORE II dataset. Further, patients with long-standing rheumatic mitral disease with LAA clot often have large left atrium (LA), and many of them require LA reduction plasty. Factors like presence LAA clot, LA size have not been given weightage in the EuroSCORE II. However, they can be of significant importance in the Indian context.

The number of patients undergoing redo surgery was also significantly higher in our study cohort. 5.3% of patients in the study cohort had a history of previous cardiac surgery as compared to only 2% in the study by Borde et al.[3] The incidence of chronic lung disease in our population was less than half of the EuroSCORE II population [Table 1]. This could possibly be attributed to the younger age of patients in our sample.

Insulin dependent diabetes mellitus (DM) has been identified as a risk factor in EuroSCORE II database, but it does not take into account the duration of DM, which can have a significant impact on the nature of disease in coronary arteries. The Indian population has a genetic predilection for DM, and early onset of DM is common. In our experience, most patients from lower socioeconomic strata have a long duration of uncontrolled diabetes at admission and often have diffuse disease in the coronary arteries. The nature of disease in the coronaries can be of significant impact on the success of revascularisation and post-operative outcome,.

Good fit of a model in the total surgical sample may not mean good fit in the individual surgical categories. This is essentially because of difference in the etiopathogenesis of various cardiac conditions. Possibly because of these concerns, some authors have analysed calibration and discrimination of EuroSCORE II separately in surgical subcategories.[4,5,9] The earlier studies on Indian population did not validate the score in various surgical categories. Thus, the present sample was also analysed by surgical categories. The Hosmer–Lemeshow goodness of fit test showed a good calibration in the total sample, CABG patients, valve surgery and mixed procedures. Discrimination was good in the total sample and the mixed procedure population, acceptable in CABG patients and excellent in valve surgeries. Our study showed that EuroSCORE II under predicts mortality in most surgical subgroups. When the risk was tabulated category wise, we found that mortality was well predicted in the very low-risk group, but it was under-predicted in low, moderate- and high-risk groups [Tables 2 and 3]. The previous study on Indian population by Borde et al. reported a slight under-prediction in low-risk groups (<2%) and over prediction in moderate- and high-risk groups (2%–5% and >5%, respectively). Pillai et al. also observed overestimation of mortality using EuroSCORE II. However, we must emphasise the fact that most patients in the above two studies underwent CABG. Valvular disease of rheumatic origin is still prevalent in India. These patients are often from lower socioeconomic status, malnourished and poorly preserved, which can lead to higher mortality. This could be the reason for the difference in prediction capability of their study and our study. In a vast country like India, regional differences in patient demographics, dietary habits, age at onset of disease and disease severity are likely to be present. This can significantly affect the outcome. Thus, we strongly feel that there is a need for multicentric study including centres from all regions of the country to validate the model. It may also be beneficial to make modification in the EuroSCORE II by incorporating variables that are important in the Indian context.

Poulis et al. observed the potential flaws in the EuroSCORE II dataset in terms of sampling time error. They found that mortality was different in different seasons. As data of EuroSCORE II were collected only over a 12-week period, it is unlikely to be reflective of the year round mortality. Our data were spread across multiple seasons.

Some concern has been raised by researchers regarding the validity of EuroSCORE II in retrospective models.[18] However, it holds true only for studies which recruited patients operated before the sampling of EuroSCORE II database, i.e., May-July 2010. As our patients were operated between January 2011 and December 2012, they should represent contemporary cardiac surgery practices similar to EuroSCORE database and it is validity should not be affected by it is retrospective nature.

CONCLUSION

EuroSCORE II has a good calibration in total cardiac surgery sample as well as in CABG, valve surgery and mixed procedure categories. Discrimination was excellent in valve surgery category. However, fair (acceptable) discrimination was found in CABG patients. EuroSCORE II underpredicted mortality in all the surgical subgroups. When studied in risk wise category, its predictive power was very good in a very low-risk group, but underpredicted mortality in other risk groups. Although the model fit of EuroSCORE II was good in our sample, we feel that there is a need for multicentric study, including centres from all regions of India to validate the model.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.