Developing a Simpler Prognosticating Tool: Comparing the Combined Assessment of Risk Encountered in Surgery Score with Deyo-Charlson Comorbidity Index and The American Society of Anesthesiologists Physical Status Score in Predicting 2 years Mortality after Hip Fracture Surgery

Abstract

Background: The use of risk stratification tools in identifying high-risk hip fracture patients plays an important role during treatment. The aim of this study was to compare our locally derived Combined Assessment of Risk Encountered in Surgery (CARES) score with the the American Society of Anesthesiologists physical status (ASA-PS) score and the Deyo–Charlson Comorbidity Index (D-CCI) in predicting 2-year mortality after hip fracture surgery. Methods and Material: A retrospective study was conducted on surgically treated hip fracture patients in a large tertiary hospital from Jan 2013 through Dec 2015. Age, gender, time to surgery, ASA-PS score, D-CCI, and CARES score were obtained. Univariate and multivariable logistic regression analyses were used to assess statistical significance of scores and risk factors, and area under the receiver operating characteristic (ROC) curve (AUC) was used to compare ASA-PS, D-CCI, and CARES as predictors of mortality at 2 years. Results: 763 surgically treated hip fracture patients were included in this study. The 2-year mortality rate was 13.1% (n = 100), and the mean ± SD CARES score of surviving and demised patients was 21.2 ± 5.98 and 25.9 ± 5.59, respectively. Using AUC, CARES was shown to be a better predictor of 2-year mortality than ASA-PS, but we found no statistical difference between CARES and D-CCI. A CARES score of 23, attributable primarily to pre-surgical morbidities and poor health of the patient, was identified as the statistical threshold for “high” risk of 2-year mortality. Conclusion: The CARES score is a viable risk predictor for 2-year mortality following hip fracture surgery and is comparable to the D-CCI in predictive capability. Our results support the use of a simpler yet clinically relevant CARES in prognosticating mortality following hip fracture surgery, particularly when information on the pre-existing comorbidities of the patient is not immediately available.

Introduction

Acute hip fracture is a major global health concern in many countries, accounting for close to 20% of all osteoporotic fractures.¹ In a recent Asian Federation of Osteoporosis Society study published by Cheung et al.,² it was projected that the number of cases will double to an estimated 2.5 million by the year 2050. Hip fracture will remain an important public health issue owing to the increasing burden placed on the healthcare system.³ Previous studies estimate the 1-year mortality rate following a hip fracture to range from 10 to 40%,^4‒7 and compared to the general population, the excess annual mortality remains higher for periods of up to 10 years.⁸ However, a recent study in the Singapore population showed that absolute mortality following hip surgery decreased significantly by 21% in 2006 to 2011 and by 40% in 2012 to 2017 compared to 2000–2005.⁹

The use of medical comorbidities for identifying high-risk surgical candidates and predicting peri-operative complications is important during the course of treating a hip fracture patient. The American Society of Anesthesiologists physical status (ASA-PS) classification system, first introduced in 1941 and subsequently revised to the current version in 2014, categorizes the physical status of preoperative patients into 6 gradings. Patients with a higher ASA-PS grading are at increased risk of medical complications following hip fracture surgery.¹⁰ However, the ASA-PS grading system is largely limited to systemic diseases and may over-simplify the pre-morbid status of a patient.

The Deyo–Charlson Comorbidity Index (D-CCI) evaluates a patient’s health status based on a weighted scale of 17 comorbidities. The original index with 19 categories—first developed in 1987 and subsequently modified to 17 categories in 1992—has been used as a validated clinical evaluation tool to determine the baseline comorbidity load of a patient prior to surgery.^11,12 Taking into consideration both systemic and localized diseases, the D-CCI score was superior to the ASA-PS in predicting post-operative mortality in a study conducted on a broad surgical population.¹³ A higher D-CCI score was indicative of a poorer pre-morbid health status and was associated with increased risk of long-term mortality after hip fracture surgery.^11,14‒16 We showed in our previous study that the D-CCI was the dominant predictive factor for mortality at 2 years follow-up.¹⁷ More importantly, while differences in survival associated with sex and ethnicity decreased with time, differences in mortality associated with higher Charlson Comorbidity Index (CCI) scores among others actually increased on long-term follow-up.⁹

A surgical risk calculator was developed locally and validated clinically in Singapore in 2018. Using nine clinically available investigation results, the Combined Assessment of Risk Encountered in Surgery (CARES) surgical risk calculator provides an objective measure for evaluating the risk of 30-day mortality and need for post-operative ICU among surgical patients in Singapore.¹⁸ Compared to ASA-PS, the CARES score performed better in terms of area under the receiver operating characteristic (ROC) curve (AUC) in predicting short-term mortality.

The aim of this study was to compare ASA-PS, D-CCI, and CARES as predictors of long-term risk of mortality. We hypothesize that our locally derived surgical risk stratification calculator is a simple yet clinically viable tool for predicting risk of 2-year mortality following hip fracture surgery

Methods

This study is a retrospective analysis of prospectively collected data from Jan 2013 and Dec 2015 at a large tertiary hospital in Singapore. Patients above 60 years of age who had undergone surgical fixation or hemiarthroplasty for traumatic hip fractures were included in the study. Patients with periprosthetic fractures, pathological fractures, or those treated conservatively were excluded. Patient-related variables of interest collected and analyzed included age, gender, time to surgery, ASA-PS, D-CCI, and CARES scores.

The age-adjusted D-CCI score was calculated based on 17 comorbid conditions, each assigned a weight of 1 to 6 according to its impact on mortality. Taking into account the effect of age, each decade after 40 years was assigned one additional point.¹⁹ The ASA-PS score was determined by the consultant anesthetist at the time of surgery. The CARES score was calculated using nine preoperative variables: age, gender, ASA status, surgical priority, surgical risk, red cell distribution width (RDW), presence of anemia, ischemic heart disease, and congestive heart failure, each weighed according to their rank score.¹⁸ Time to surgery was calculated as time elapsed from hospital admission to start of surgery. The threshold for delay in surgery was taken as greater than 48 hours. Table 1 shows the criteria used in each of the 3 scoring systems.

Table 1.

Comparing CARES, D-CCI, and ASA-PS Scoring System.

CARES	D-CCI	ASA-PS
Age Gender Surgical risk Anemia status Red Cell distribution width Ischemia heart disease Congestive heart disease American Society of Anesthesiologists classification Priority of surgery Each of the variables are weighted according to their rank score	1 Myocardial infarction  Congestive heart failure  Peripheral vascular disease  Cerebrovascular disease  Dementia  Chronic pulmonary disease  Rheumatologic disease  Peptic ulcer Disease  Mild liver disease  Diabetes without complications 2 Hemiplegia or paraplegia  Renal disease  Diabetes with complications  Any malignancy 3 Mod /severe liver disease 6 Metastatic solid malignancy  Acquire immunodeficiency disease For each decade >40 years of age, a score of 1 is added to the total D-CCI score	1 Normal healthy 2 Mild systemic disease 3 Severe systemic disease 4 Constant threat to life 5 Moribund 6 Declared brain dead

CARES = Combined Assessment of Risk Encountered in Surgery, D-CCI = Deyo–Charlson Comorbidity Index, ASA-PS = the American Society of Anesthesiologists physical status.

Univariate logistic regression analysis was used to assess the effects of variables recorded at baseline on risk of mortality at 2 years post-surgery. Factors with P-value <.20 in the univariate analysis were entered into a multivariate logistic regression incorporating a forward stepwise selection algorithm with significance level to enter and stay of .05 and .10, respectively. To compare ASA-PS, D-CCI, and CARES as predictors of 2-year mortality, ROC curves were obtained for each and area under the curve compared between pairs of risk matrices. Youden’s rule was used to identify a statistically optimal CARES cut-off defining a “high-risk” threshold beyond which the patient’s baseline comorbidities posed greater risk of mortality. All analyses were performed using SAS v9.4 software (SAS Inc., Cary, NC, USA). Statistical significance was set at P < .05.

Results

A total of 763 hip fracture patients met the inclusion criteria and were included in the analysis. The mean ± SD age was 77.9 ± 8.1 years, and the overall mortality rate at 2 years was 13.1% (n = 100). Study demographics are summarized in Table 2.

Table 2.

Summary of Patient Demographics and Comorbidity Scores by Alive or Dead at 2 years Post-Surgery.

Variable	Total	Alive (n = 663)	Dead (n = 100)	P-value
Age	763	77.6 ± 8.02	80.0 ± 8.7	.008
Gender Male Female	221 542	173 490	48 52	<.001
Surgical delay No Yes	241 522	223 440	18 82	.002
American Society of Anesthesiologists (mean) 1 2 3 4	4 542 211 6	2.26 ± .46 4 488 168 3	2.49 ± .56 0 54 43 3	<.001
Deyo–Charlson Comorbidity Index (mean) 0-2 3-4 5-6 >6	50 340 248 125	4.55 ± 1.74 49 318 207 89	6.3 ± 2.56 1 22 41 36	<.001
Combined Assessment of Risk Encountered in Surgery score <10 10-20 20-30 >30	0 354 339 70	21.2 ± 5.98 0 334 282 47	25.9 ± 5.59 0 20 57 23	<.001

Bold font indicates statistical significance (P < .05).

We found a significant difference in age between those surviving (mean 77.6 ± 8.02 years) and those dead at the 2-year follow-up (mean 80.0 ± 8.73 years). The female-to-male ratio was 3:1 with significantly higher mortality in females. 31.6% (n = 241) of patients underwent surgery within 48 hours of admission to the hospital, and univariate analysis showed that delay in surgery was an independent predictor of mortality.

Patient baseline health conditions measured using CARES were classified into 4 risk groups: Low (0–10), Low–Moderate (11–20), Moderate–high (21–30), and High (>30). From the Low–Moderate (6%) rate, mortality rates approximately doubled with successive increases to Moderate–High (17%) and then to High (33%) (Figure 1).

Figure 1.

Death: Total ratio at 2 years follow-up after hip fracture surgery.

CARES mean scores differed significantly between patients alive at 2 years post-surgery (6.3 ± 2.56) and those who died (25.9 ± 5.59), and similarly for D-CCI (4.55 ± 1.74 vs 6.3 ± 2.56) and ASA (2.26 ± .46 vs 2.49 ± .56) (all P <.001) (Table 2).

Multivariable logistics regression analysis identified gender, D-CCI, and CARES as independent predictors of 2-year mortality (Table 3) and were also selected using a stepwise selection algorithm (Table 4). Odds ratios from the multivariable logistic regression model incorporating these factors as the main risk predictors of 2-year mortality were D-CCI (OR = 1.34, 95% CI: 1.20-1.50; P < .001), CARES (OR = 1.08, 95% CI: 1.03-1.12; P=.001), and gender (OR = .53, 95% CI: .33-.85; P=.008) (Table 4).

Table 3.

Multivariable Logistics Regression Analysis of Factors Affecting 2 years Mortality After Hip Fracture Surgery.

Factor	Odds ratio	95% confidence interval		P-value
Surgical delay	1.46	.82	2.59	.200
American Society of Anesthesiologists	1.48	.95	2.30	.084
Gender	.49	.30	.79	.003
Age	1.02	.99	1.05	.253
Deyo–Charlson Comorbidity Index	1.30	1.16	1.46	<.001
Combined Assessment of Risk Encountered in Surgery	1.06	1.02	1.11	.005

Bold font indicates statistical significance (P < .05).

Table 4.

Multivariable Logistic Regression Analysis Incorporating a Stepwise Selection Algorithm (significance levels: enter=.05, stay=.10) in the Analysis of Risk Factors.

Entered	Contributing factor	Odds ratio	95% confidence interval		Wald chi-square	P-value
1	Deyo–Charlson Comorbidity Index	1.34	1.20	1.50	27.82	<.001
2	Combined Assessment of Risk Encountered in Surgery	1.08	1.03	1.12	11.92	.001
3	Gender	.53	.33	.85	7.02	.008

Bold font indicates statistical significance (P < .05).

AUC as a measure of predictive capability was as follows: ASA-PS (AUC = .606, 95% CI: .553-.658), D-CCI (AUC = .696, 95% CI: .646-.747), and CARES (AUC = .681, 95% CI: .631-.731). D-CCI (P = .007) and CARES (P = .017) were both significantly better predictors of 2-year mortality than ASA-PS; however, the difference between CARES and D-CCI was not statistically significant (P = .607). (Figure 2).

Figure 2.

Comparisons of ROC curves for the American Society of Anesthesiologists physical status, Deyo–Charlson Comorbidity Index, and CARES in predicting 2-year mortality.

Youden’s rule for obtaining a statistically optimal threshold delineating “low” vs “high” risk of 2-year mortality identified a CARES score of 23 as the cut-off with scores ≥23 indicating “high” risk of 2-year mortality. Sensitivity and specificity of the CARES score at the 23 cut-off were 74% and 60%, respectively; positive predictive value (PPV) and negative predictive value (NPV) given the 13% mortality rate was 22% and 93%, respectively. The Youden cut-off for D-CCI was 6, with sensitivity, specificity, PPV, and NPV of 59%, 76%, 27%, and 92%, respectively. At the ASA-PS Youden cut-off of 3, sensitivity, specificity, PPV, and NPV were 46%, 74%, 21%, and 90%, respectively.

Discussion

The 2-year mortality rate in this study was 13.1%, and we found that an increase in CARES score was associated with worsening survivorship. The use of CARES in predicting 2-year mortality was comparable to D-CCI and significantly better than ASA-PS. We also reported that a CARES score of 23, attributable primarily to pre-surgical morbidities and poor health of the patient, was identified as the statistical threshold for “high” risk of 2-year mortality.

Identification of patients at increased risk of complications after hip fracture surgery is important as it will allow clinicians to improve peri-operative management when treating this group of patients. Decision-making regarding treatment and peri-operative surgical management are often guided by the patient’s comorbidities assessed via various health matrices such as ASA-PS and D-CCI.^20,21

The ASA-PS classification system was first introduced in 1941 by Saklad et al. and has since been revised multiple times to improve objectivity and inter-rater variability. Despite the latest revised scoring scheme and the addition of case vignettes designed to improve grading accuracy, the ASA-PS classification relies on not only comorbidity presence but on severity as well in determining a grade. Nevertheless, inter-rater reliability and disparities in ASA-PS assessments still exist, with greater inaccuracies among non-anesthesia trained clinicians who frequently underestimate severity compared to their anesthesia trained colleagues.^22‒24 Despite these issues, the ASA-PS classification system is still being commonly used in orthopaedic trauma settings. Patients in higher ASA-PS classes have been found to be strongly associated with peri-operative medical complications following hip fracture surgery and at increased risk of post-operative mortality.^10,25‒28 In our study, we found ASA-PS to be a significant risk predictor of 2-year mortality in univariate analysis but failed to reach statistical significance in the presence of D-CCI, CARES, and other risk factors in multivariable analysis.

D-CCI evaluates a patient’s comorbid load score based on a weighted summation of existing medical conditions and has been shown to be a good preoperative indicator for both short- and long-term mortality in the elderly with hip fracture.^14,15,29,30 Rather than relying upon the evaluating clinician’s assessment of comorbid severity, D-CCI is an objective clinical evaluation tool aimed at improving inter-rater reliability. However, information on a patient’s pre-existing medical condition may not be readily available in many settings. At the same time, reliably coded comorbidities in administrative data may be inaccurate and often under-reported.^31,32 Nevertheless, having so stated, D-CCI remains a useful evaluation tool in the peri-operative management of surgical patients, given its high inter-rater reliability.^33,34 In our previously published study, the age-adjusted D-CCI was found to be the dominant risk predictor affecting mortality after hip fracture surgery.¹⁷

The importance of developing a reliable and objective risk stratification tool for surgical patients is crucial in the prognostication and management of complications after surgery. The poor inter-rater reliability of the ASA-PS and the D-CCI reliance on information regarding pre-existing medical conditions has given rise to an unaddressed need for a clinical tool that uses more readily available data. The CARES score used in this study was locally validated and found to perform better than the ASA-PS classification in prognosticating early post-surgical mortality and the need for intensive care unit stay. Patients with a higher CARES score have a higher comorbid load, are in poorer health, and may also be at higher risk of long-term mortality. In comparisons of the CARES score and ASA-PS grading, we demonstrated in the present study that CARES was the better risk predictor for long-term mortality. In addition, the comparable AUC in predicting 2-year mortality after hip fracture surgery between D-CCI and CARES suggests that in a situation where comprehensive pre-existing medical information is not available, the simpler CARES scoring system may serve as an equally good risk prognostication tool. CARES, with its inclusion on surgical complexity as a scoring factor, may also be more applicable to estimating survivorship of patients beyond the peri-operative period.

Using the Youden rule, we identified a CARES a “high”-risk threshold based on a patient’s pre-existing medical condition for prognosticating the 2-year mortality risk following hip fracture surgery. Although a CARES score of ≥23 was suggestive of a poorer outcome after surgery, it should not preclude a patient from having surgery after a traumatic hip fracture.

Strength and Limitation

The strength of this study lies in the large cohort with a high follow-up rate. Of the 775 surgically treated hip fracture patients with complete data on their preoperative ASA-PS, CARES, and D-CCI scores, our lost-to-follow-up rate was only 1.5% (n = 12). This new and clinically validated risk stratification tool has addressed some of the shortcomings seen in both the ASA-PS and D-CCI classification systems. In addition, because of a more detailed range of baseline CARES scores captured in our study, we were able to identify a threshold at which baseline comorbidities play a more important role in affecting mortality.

This was a retrospective observational study investigating the risks of 2-year mortality. Other long-term complications including functional disability, reduced quality of life, or deterioration of health status were not evaluated. The reliability of the ASA-PS grading and CARES scores could not be independently assessed because they were assigned and recorded at the time of surgery. The D-CCI scores were calculated retrospectively using patient health records and documented pre-existing medical history.

Conclusion

In this study, we showed that the CARES score is an important risk predictor for long-term mortality after hip fracture surgery and is comparable to D-CCI. CARES may be used in conjunction with other risk stratification tools and particularly when information on the pre-existing comorbidities of the patient is not immediately available. The CARES threshold score of 23 reflects an important threshold in the prognostication of surgically treated hip fracture patients.

ORCID iD

Eric Wei Liang Cher https://orcid.org/0000-0002-4781-965X