Association of Comorbidities With Postoperative In-Hospital Mortality

Supplemental Digital Content is available in the text

Abstract

The purpose of this article is to evaluate the American Society of Anesthesiologists Physical Status (ASA PS) and the Charlson comorbidity index (CCI) for the prediction of postoperative mortality.

The ASA PS has been suggested to be equally good as the CCI in predicting postoperative outcome. However, these scores have never been compared in a broad surgical population.

We conducted a retrospective cohort study in a German tertiary care university hospital. Predictive accuracy was compared using the area under the receiver-operating characteristic curves (AUROC). In a post hoc approach, a regression model was fitted and cross-validated to estimate the association of comorbidities and intraoperative factors with mortality. This model was used to improve prediction by recalibrating the CCI for surgical patients (sCCIs) and constructing a new surgical mortality score (SMS).

The data of 182,886 patients with surgical interventions were analyzed. The CCI was superior to the ASA PS in predicting postoperative mortality (AUROC_CCI 0.865 vs AUROC_ASAPS 0.833, P < 0.001). Predictive quality further improved after recalibration of the sCCI and construction of the new SMS (AUROC_SMS 0.928 vs AUROC_sCCI 0.896, P < 0.001). The SMS predicted postoperative mortality especially well in patients never admitted to an intensive care unit.

The newly constructed SMS provides a good estimate of patient's risk of death after surgery. It is capable of identifying those patients at especially high risk and may help reduce postoperative mortality.

INTRODUCTION

Approximately 230 million surgical procedures are performed every year worldwide.¹ Postoperative mortality has been decreasing for decades due to continuous advances in hygiene, standard of care, and risk stratification, enabling clinicians to anticipate and counteract possible complications. Despite these improvements, a recent prospective international multicenter cohort study revealed postoperative mortality to be as high as 4% and therefore higher than expected.² Moreover, the authors found that 3 of 4 patients who died during their hospital stay had never been admitted to an intensive care unit (ICU).²

Besides the surgical procedure itself and associated complications, the patients’ comorbidities entail risks of postoperative mortality and morbidity.^3–8 The most common way to evaluate a surgical patient's comorbidities in clinical routine is the American Society of Anesthesiologists Physical Status (ASA PS) Classification. By contrast, the Charlson comorbidity index (CCI) is more frequently used in clinical research.⁹ While the ASA PS is a grade 6 severity estimate score and fairly easy to use, the CCI is a more complex score of 19 weighted comorbidity items. Studies in small and selected research populations have repeatedly suggested that the simpler ASA PS can be as good as the CCI in predicting postoperative morbidity and mortality.^10–12 Despite the common practice of using the ASA PS in the perioperative clinical setting, there has never been a large study in a broad surgical population to compare these 2 scores with reference to predictability of postoperative mortality.

We therefore conducted a single-center retrospective analysis at 2 campuses of a tertiary care university hospital to investigate whether either the ASA PS or the CCI is a better predictor of postoperative in-hospital mortality in surgical patients. In a post hoc exploration, we tried to further improve predictability: we calibrated the CCI for surgical patients and constructed a surgical mortality score (SMS) also including intraoperative factors.

METHODS

Patients

The appropriate ethics committee waived the requirement for informed consent (EA1/007/13) and the trial was registered at ClinicalTrials.gov (NCT01810133) prior to data collection. In our department, the treatment of every patient is documented on a paper-based anesthesia case log that contains machine-readable elements. These paper case logs are subsequently scanned. The scanned image files are stored alongside the machine-readable elements in a database. All patients with a digitalized electronic anesthesia record between January 2006 and December 2011 were eligible for inclusion. Ambulatory cases were excluded and only cases with complete data were analyzed.

Data Source

We extracted patient data from 2 sources and combined them in 1 MySQL-database (MySQL Community Server; Oracle, Redwood City, CA): the ASA PS, the surgical discipline, priority of the surgery (elective, urgent, or emergency), localization of the surgery, and whether the patient received any intraoperative transfusion was extracted from the electronically archived anesthesia protocols Medlinq (Medlinq Softwaresysteme, Hamburg, Germany). Age, gender, International Classification of Diseases (ICD)-coded diagnoses, and in-hospital death were extracted from the hospital data management system (SAP, Walldorf, Germany). If the patient underwent >1 surgical procedure during his hospital stay, only the data regarding the first surgical intervention was included in the analysis. The CCI was calculated based on the ICD diagnoses routinely documented in the in-hospital data management system.¹³

Statistical Analyses

Frequencies are reported as number and percentages with 95% confidence intervals (95% CIs) when appropriate, continuous variables are presented as median and interquartile ranges (IQRs) if normality was ruled out by histograms or quantile–quantile plots. Continuous variables were neither transformed nor grouped for analyses. Categorical data were compared using Fisher exact test, continuous data using the Mann–Whitney U test, and the Kruskal–Wallis test as indicated. Exact testing was conducted whenever applicable. All calculations were conducted using R 3.0 for MacOS (The R Foundation for Statistical Computing; http://www.r-project.org/). The probability of a type I error <1% was considered to be statistically significant.

Regression Modeling and Model Validation

The association of variables with postoperative mortality was estimated fitting a binary logistic regression model. Variable selection for this model was conducted by cross-validation as follows: the study population was randomly divided into 10 samples. For each of these samples, a binary logistic regression model with backward selection by Akaike information criterion was fitted. Variables that were selected in ≥8 of the 10 fitted models and did not correlate with other variables (r² ≥ 0.6) were then used to fit a binary logistic regression model in the complete study population. For the calibration of the CCI as well as for weighting the SMS variables, the regression coefficients (betas) were added to a sum.

Cross-validation was not repeated for further models presented in this study. For the linear regression model fitted with SMS as dependent variable, SMS was log-transformed because of skewness.

Nomogram

In order to visualize the association of the variables with the outcome and offer a simple paper-based possibility to calculate the probability of postoperative in-hospital mortality, a nomogram was constructed with methods described by Harrell.¹⁴

Receiver-Operating Characteristic Analysis

We conducted receiver-operating characteristic (ROC) curve analyses to compare the prognostic accuracy of the ASA, CCI, CCI for surgical patients (sCCI), and the newly constructed SMS for postoperative mortality with the software package developed by Robin et al,¹⁵ and ROC curves were compared using the integrated DeLong method of the package to evaluate differences in the areas under the ROC curves (AUROC).

RESULTS

Study Population

We identified 230,034 electronically accessible anesthesia records during the study period, out of which 34,345 were anesthesia records of the same patient who underwent surgery >1 time. The data of 2827 outpatient cases and 10,010 cases with incomplete records were excluded from the analysis. In total, the data of 182,886 cases were analyzed (Figure 1). Patients’ median age was 47 and 47% of the study population was female. The most frequent comorbidities were malignancy of any kind (13%), diabetes without complication (10.9%), and renal disease (6.8%). The majority of patients underwent elective surgery (78.1%); the mean duration of the surgical procedure was 61 minutes (IQR 25–116 minutes). Most patients underwent traumatology/orthopedic surgery (18.4%), general surgery (15.5%), and gynecological surgery (10.2%; Table 1). A total of 2301 patients died during their hospital stay (1.3%; Table 2) and the median hospital length of stay was 5 days (IQR 3–10 days). Survivors had a median hospital length of stay of 5 days (IQR 3–10 days) and nonsurvivors of 25 days (IQR 11–51 days).

FIGURE 1.

Flow chart of data acquisition leading to analyzed cases.

TABLE 1.

Surgical Disciplines and In-Hospital Mortality

TABLE 2.

Characteristics of the Study Population (n = 182,886)

Association of Comorbidities With Postoperative Mortality

Age, gender, priority of surgery, location of surgery, duration of surgery, intraoperative transfusion, and all CCI items were considered to be potentially associated with postoperative mortality. Out of all considered variables, 2 items of the CCI, “hemiplegia or paraplegia” and “myocardial infarction (MI),” as well as duration of surgery were not selected into the model by the validation method described previously. The final model is presented in Table 3.

TABLE 3.

Surgical mortality score^∗

Age and gender were mildly associated with higher mortality. Patients with urgent surgical procedures were 3-fold likely to die, with emergency procedures 6-fold likely to die compared with the patients undergoing elective procedures. Patients with intraabdominal, intrathoracic, or intracranial procedures were 1.3, 1.7, and 1.8-fold likely to die compared with those without the respective procedures. The odds of dying were 2-fold when receiving an intraoperative transfusion. The CCI items most strongly associated with postoperative mortality were “moderate or severe liver disease” with 10-fold odds of death (original CCI weight 3), “metastatic solid tumor” with 6-fold odds of death (original CCI weight 6), “acquired immune-deficiency syndrome or human immunodeficiency virus” with 4-fold odds of death (original CCI weight 6), and “congestive heart failure” with 4-fold odds of death (original CCI weight 1). The remaining CCI items were associated with a 1.5 to 2-fold odds of death (original CCI weights 1 or 2).

Prediction of Postoperative Mortality

The quality of prediction of the CCI was superior to the ASA PS (AUROC_CCI 0.865 [95% CI 0.859–0.871] vs AUROC_ASAPS 0.833 [95% CI 0.826–0.840]; P < 0.001; Figure 2). As shown in Table 3, the weights of the original CCI differed from the independent risk attributed to the CCI items. We therefore modified the CCI by adding up the regression coefficients of the respective comorbidities resulting in a calibrated sCCI. This modified sCCI showed a better predictability than the original CCI (AUROC_sCCI 0.896 [95% CI 0.889–0.903] vs AUROC_CCI 0.865 [95% CI 0.859–0.871]; P < 0.001; Figure 2).

FIGURE 2.

Receiver-operating characteristic (ROC) curves for the prediction of postoperative in-hospital mortality by different scores: the ASA Physical Status (AUROC_ASAPS 0.83), the original Charlson comorbidity index (CCI; AUROC_CCI 0.87), the Charlson CCI recalibrated for surgical patients (sCCI; AUROC_sCCI 0.90), and the surgical mortality score (SMS) composed of the sCCI and additional intraoperative variables (AUROC_SMS 0.93). All ROC curves differ significantly in pairwise comparison (P < 0.001). ASA PS = American Society of Anesthesiologists Physical Status, AUROC = area under the receiver-operating characteristic curves.

We than created the new SMS. In addition to the CCI variables, surgery-associated factors, priority and location of surgery, as well as intraoperative transfusion were considered variables with association to in-hospital death. History of MI as well as paraplegia and hemiplegia were excluded from the final model by the cross-validation process. The resulting SMS (Table 3) predicted postoperative in-hospital mortality to be even better (AUROC_SMS 0.928 [95% CI 0.923–0.933] vs AUROC_sCCI 0.896 [95% CI 0.889–0.903]; P < 0.001; Figure 2). A logistic regression model was fitted to assess the validity of the SMS. The SMS remained independently associated with in-hospital death in multivariate analysis including age, gender, ASA PS, priority, location, and duration of surgery, hospital length of stay, and ICU admission (Table 4).

TABLE 4.

Binary Logistic Regression Model for In-Hospital Mortality Including the New SMS (n = 182,886)

For the regression model that served as a basis for the construction of the SMS, we also provide a nomogram as a paper-based alternative to manually estimate the probability of in-hospital death (Figure 3).

FIGURE 3.

Nomogram for predicting in-hospital mortality in surgical patients. Find the age, gender, priority, and location of surgery, and the comorbidities of the patient in the nomogram. Then draw a vertical line to the scale in the top row to determine the corresponding points for each item. Now add all points, find the sum in the second to last row (total of points), and draw a vertical line down: the intersection marks the probability of in-hospital death for this patient.

Prediction of Postoperative Mortality in Patients With and Without ICU Admission

Approximately one-third of all included patients were admitted to the ICU after surgery (35%; 47,386 of 182,886). Patients who had never been admitted to the ICU and presented an elevated risk of in-hospital death were detected especially well by the SMS (AUROC_neverICU 0.90 [95% CI 0.77–0.88 vs 0.93] vs AUROC_ICU 0.84 [95% CI 0.83–0.85]; P < 0.001; Figure 4). Patients never admitted to the ICU were younger, had a lower ASA PS, had more often elective surgery, had less often intracranial, intrathoracic, or intraabdominal surgery, less comorbidities, a shorter hospital stay, and died less often (0.1% vs 4.4%; P < 0.001 for all variables; see Table 5, Supplemental Content, http://links.lww.com/MD/A215, which describes the study population divided by ICU admission). Admission to ICU remained independently associated with a higher SMS in multivariate analysis (see Table 6, Supplemental Content, http://links.lww.com/MD/A215, which shows the results of a linear regression model for the new SMS including ICU admission as independent variable).

FIGURE 4.

Receiver-operating characteristic (ROC) curves for prediction of postoperative in-hospital mortality in (A) patients never admitted to the intensive care unit (ICU) and (B) patients admitted to the ICU during their hospital stay by the American Society of Anesthesiologists Physical Status (ASA PS), the Charlson comorbidity index (CCI), the Charlson CCI for surgical patients (sCCI), and the surgical mortality score (SMS).

DISCUSSION

In this retrospective cohort study in a broad surgical population of 182,886 patients at a German tertiary care university hospital, we demonstrated the CCI being superior in predicting postoperative mortality compared with the ASA PS. In a post hoc approach, we improved the prediction of postoperative mortality by validating the relevant items of the CCI and recalibrating them. We constructed a new SMS by including intraoperative variables and could further improve the prediction. The newly calibrated CCI and the SMS were especially suitable to predict postoperative in-hospital mortality in patients who were never admitted to an ICU during their hospital stay. This was despite the fact that these patients were younger, had a lower ASA PS, had more often elective surgery, had less often intracranial, intrathoracic, or intraabdominal surgery, less comorbidities, and a shorter hospital stay.

The prediction of postoperative mortality is important for the patient's individual perioperative risk assessment. Previous studies have often been limited by investigating special surgical populations (eg, prancreatectomy, hip fracture, or cardiac surgery). A recent systematic review by Moonesinghe et al¹⁶ found only 27 studies conducted since 1980 examining risk prediction in broad surgical populations. From these studies, only 2 investigated the CCI (both calculated the CCI from ICD diagnoses),^17,18 only 4 investigated the ASA PS,^19–22 and none compared both in the same population. Although ASA PS has been shown to have varying quality of predicting postoperative mortality (AUROC 0.63–0.90), our results for the CCI are remarkably similar to an Australian multicenter national database analysis of 2.5 million patients. Sundararajan et al¹⁸ demonstrated that the CCI calculated from ICD diagnoses had a good predictive quality for in-hospital death with an AUROC of 0.855, in line with our results. Although we conducted this analysis in only 1 hospital, this similarity indicates good generalizability of our findings. Besides that, the SMS can be fine adjusted in hospitals with fairly different patient populations.

Surprisingly, preexisting MI was not associated with patients’ outcome in our analysis. When cross-validating variable selection for the final model, MI did not reach significance in any of the 10 random samples. Medical care for patients with MI has constantly improved over the last decades (increasing reperfusion therapy), which has led to better short and long-term outcomes.²³ In addition, these patients are likely to be assessed thoroughly before elective surgery.²⁴ It is most likely that MI patients in our sample had adequately recanalized coronary arteries and received a full cardiac assessment before undergoing surgery.

There are scores other than the 2 examined in this article that could be used to predict postoperative mortality but all of the possible scores have disadvantages. The ASA PS has a high interrater variability,^25–27 the CCI is fairly complex, as are the physiological and operative severity score for the enumeration of mortality and morbidity (POSSUM) and P (Portsmouth)-POSSUM scores. The latter, in addition, include variables of high uncertainty (eg, blood loss) and variables that may change quickly and greatly (systolic blood pressure, white blood cell count, hemoglobin, or potassium). Temporary changes of these variables may therefore—temporarily—underestimate or overestimate the risk of death. The established SMS we present in this study renders the possibility to estimate the risk of death after surgery on the basis of age, gender, priority, and localization of surgery, the need for intraoperative transfusions, and comorbidities. Although a few of these variables may change during the course of the hospital stay, they only change once and the SMS can be adjusted reflecting these changes in the risk of death for the patient.

These days (and independently from the computerization of our world), patients are overdocumented. At our institution, a patient's comorbidities are documented at least 4 times: when the surgeon takes the medical history for planning the surgery, when the patients fill out a commercially available paper questionnaire prior to seeing the anesthesiologist in our preoperative clinic, when the anesthesiologist documents the comorbidities with the patient's history on the preanesthesiological assessment form, and when the medical documentation assistant records them as ICD-10 codes in the hospital administration system for the billing department. The calculation of our SMS may appear rather complicated for everyday clinical use but computerization in hospitals further progresses and less is documented on paper. Besides the possibility to use a mobile or web-based application for this purpose, there is an even more promising idea: the possibility to automatically calculate the SMS. If patients filled out their questionnaires electronically,^28–30 or either the surgeon, the anaesthesiologists, or the medical documentation assistant gathered and recorded their data digitally on a regular bases, one of these sources could easily be used to calculate individual patients’ risks. In addition, a system similar to that could even autocalibrate itself to stay accurate in the future.

We also demonstrated that the SMS predicts postoperative in-hospital death especially well in patients never admitted to the ICU. As Pearse et al² reported in the European Surgical Outcome Study, 75% of patients who die after surgery have never been admitted to an ICU. Although the ratio is different in our study population, the calculation of the patients’ risk of death with the SMS may be an appropriate tool to identify those patients at high risk and take appropriate measures to counteract. We therefore also provided a paper-based way to calculate the risk of postoperative death with the nomogram. What measures to take (regularly scheduled vital signs on the normal ward, telemetry, intermediate care unit admission, or even ICU admission) and when to take them? These questions will have to be answered in future research and may very well vary between countries and even hospitals of different levels of care.

LIMITATIONS

We investigated a large and very broad surgical patient population and constructed a valid risk prediction regression model that is unlikely to be overfitted. Regardless, this study has also limitations: although prospective cohort studies usually gather data of better quality, retrospective studies are superior in describing the reality of clinical practice and therefore provide increased generalizability because analyses are conducted on less selected populations.³¹ Based on data from a single tertiary care university hospital, generalizability of our data for other hospital types with other patient populations may be limited.

CONCLUSIONS

Taking comorbidities into account improves the prediction of postoperative mortality. It can further be improved by adding surgery-associated variables. The resulting SMS is an easy way to calculate the patients’ individual risk. With medical data being progressively recorded digitally, an automated calculation of the SMS could be implemented in different kinds of medical software applications. The SMS may also be beneficial when identifying patients at especially high risk and may even aid in reducing postoperative mortality, if the appropriate measures for these patients will be taken.