Abstract
Introduction
Frailty is a risk factor for adverse postoperative outcomes. We aimed to test the performance of a prospectively-validated frailty measure, the Risk Analysis Index (RAI) in vascular surgery patients and delineate the additive impact of procedure complexity on surgical outcomes.
Methods
We queried the 2007–2013 American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) database to identify six major elective vascular procedure categories (carotid revascularization, abdominal aortic aneurysm [AAA] repair, suprainguinal revascularization, infrainguinal revascularization, thoracic aortic aneurysm [TAA] repair, and thoracoabdominal aortic aneurysm [TAAA] repair). We trained and tested logistic regression models for 30-day mortality, major complications and prolonged length of stay (LOS). The first model, “RAI”, used the RAI alone; “RAI-Procedure (RAI-P)” included procedure category (e.g., AAA repair) and procedure approach (e.g., endovascular); “RAI-Procedure Complexity (RAI-PC)” added outpatient versus inpatient surgery, general anesthesia use, work relative value units (RVUs), and operative time.
Results
The RAI model was a good predictor of mortality for vascular procedures overall (C-statistic 0.72). The C-statistic increased with the RAI-P (0.78), which further improved minimally, with the RAI-PC (0.79). When stratified by procedure category, the RAI predicted mortality well for infrainguinal (0.79) and suprainguinal (0.74) procedures, moderately well for AAA repairs (0.69) and carotid revascularizations (0.70), and poorly for TAAs (0.62) and TAAAs (0.54). For carotid, infrainguinal, and suprainguinal procedures, procedure complexity (RAI-PC) had little impact on model discrimination for mortality, did improve discrimination for AAAs (0.84), TAAs (0.73), and TAAAs (0.80). While the RAI model was not a good predictor for major complications or LOS, discrimination improved for both with the RAI-PC model.
Conclusions
Frailty as measured by the RAI was a good predictor of mortality overall after vascular surgery procedures. While the RAI was not a strong predictor of major complications or prolonged LOS, the models improved with the addition of procedure characteristics like procedure category and approach.
INTRODUCTION
Frailty is a medical syndrome of diminished strength, endurance, and reduced physiologic reserve that increases one’s vulnerability for disability and death in the presence of a stressor. A growing body of surgical literature shows that frailty is an independent risk factor for poor outcomes1–3 and failure-to-rescue from postoperative complications.4 There are many frailty measurement tools,2 but the assessment burden often precludes their implementation in busy surgical clinics. The Risk Analysis Index (RAI) is a frailty screening tool based on the accumulated deficits model,5 which was initially developed in a veteran population.6 It was subsequently recalibrated and externally validated in two large surgical databases: Veterans Affairs Surgeons and the American College of Surgeons National Surgical Quality Improvement Program (VASQIP/ACS-NSQIP), and in a prospective cohort of surgical veterans.7 The RAI is a prospectively administered, patient-facing tool that takes less than two minutes to perform. Frailty as defined by the RAI is associated with 30-, 180-, and 365-day postoperative mortality,1 and in a quality improvement study, routine preoperative frailty screening with the RAI was associated with reduced institutional surgical mortality.1
Frailty can be unmasked after exposure to a stressor, such as surgery.8 While much of frailty research focuses on the direct impact of frailty on outcomes, we know less about the effect or magnitude of a surgical procedure stressor on adverse outcomes in frail patients. Vascular surgery patients represent an ideal group to study this phenomenon as there are a many different procedure categories, ranging from lower surgical risk (carotid revascularizations) to those with higher risk (thoracoabdominal aortic aneurysm repairs [TAAAs]), as well as less invasive (endovascular) and more invasive (open) approaches to treat the same pathologies. Vascular patients have a higher prevalence of frailty,2 and to our knowledge, the RAI has not yet been evaluated for associations with vascular postsurgical outcomes nor performance as a standalone risk score in vascular surgery patients.
The aim of this study was to assess the association of frailty, as measured by the RAI, in vascular surgery patients with 30-day mortality, major complications and prolonged hospital length of stay (LOS). We sought to determine the predictive ability of the RAI on the three outcomes of interest, and to determine the impact of procedure complexity on each model’s predictive accuracy. Our hypotheses were that the RAI would be associated with poor surgical outcomes and have a robust predictive ability for 30-day mortality, major complication and prolonged LOS.
METHODS
1.1. Data
We queried the 2007–2013 ACS-NSQIP Participant Use Files (PUF) for six major vascular surgery categories by Current Procedural Terminology (CPT) code (Appendix A). The procedure categories included AAA repair, carotid revascularization, infrainguinal revascularization, suprainguinal revascularization, thoracic aortic aneurysm (TAA) repair, and TAAA repair. Emergency cases were excluded (ACS-NSQIP variable EMERGNCY: Yes). The covariates serving as procedure characteristics included procedure category (e.g., AAA repair), procedure approach (endovascular versus open) as identified by CPT code, outpatient versus inpatient surgery status, general anesthesia use, work relative value units (RVUs),9,10 and operative time in minutes (ACS-NSQIP variables INOUT, ANESTHES, WORKRVU, OPTIME).
Our study outcomes of interest were 30-day mortality (ACS-NSQIP variable YRDEATH, “−99” if alive at 30 postoperative days), major complications, and prolonged LOS. We defined major complications as any occurrence of deep incisional wound infection, organ space wound infection, dehiscence, pneumonia, reintubation, pulmonary embolism, ventilator dependence for >48 hours, acute renal failure, stroke, cardiac arrest requiring cardiopulmonary resuscitation, myocardial infarction, sepsis, septic shock, reoperation, and bleeding requiring transfusion (ACS-NSQIP variables WNDINFD, ORGSPCSSI, DEHIS, OUPNEUMO, REINTUB, PULEMBOL, FAILWEAN, OPRENAFL, CNSCVA, CDARREST, CDMI, OTHSYSEP, OTHSESHOCK, RETURNOR, and OTHBLEED).11 We defined prolonged LOS as postoperative LOS (ACS-NSQIP variable TOTHLOS minus HTOODAY) ≥75th percentile for each procedure approach and category (e.g., endovascular AAA repair ≥ 3 days versus open AAA repair ≥ 9 days).
This study was deemed exempt by the institutional review boards at Emory and Stanford Universities.
1.2. Frailty/RAI
The RAI score includes age, sex, disseminated cancer, weight loss, renal failure, congestive heart failure, dyspnea, functional status, cognitive impairment, and living status. Using previously described methods, we calculated the RAI score for each record in our sample using NSQIP preoperative variables (Appendix B).12–19 After 2012, ACS-NSQIP stopped collecting the variables for the cognitive impairment score including previous coma, impaired sensorium, and prior stroke events (ACS-NSQIP variables IMPSENS, COMA, and CVA). Missing data for these variables were scored as not having any cognitive impairment in order to yield more conservative estimates. These were rare occurrences even in years where the data were collected and the effects on model discrimination and calibration were determined to be minimal by comparing models based on data before and after 2012. If any other RAI variables were missing, such as age or sex, the record was excluded. Once the RAI was calculated, patients were classified as Non-Frail (RAI<30), Frail (RAI≥30), and Very Frail (RAI≥35), as per prior validation studies.7
1.3. Analyses
Demographics, patient characteristics, and procedure characteristics were summarized for the entire cohort as well as stratified by procedure category and approach (endovascular versus open). Chi-square tests were calculated to compare categorical variables and ANOVA tests were calculated to compare continuous variables between procedure category groups. We summarized the three outcomes of interest (mortality, major complications, and prolonged postoperative LOS) by RAI score as a continuous variable as well as in the categories Non-Frail, Frail, and Very Frail. We also binned the RAI into 5-unit categories to visualize trends. We assessed three different logistic regression models for each dichotomous outcome: mortality, major complication, and prolonged postoperative LOS. The first model, “RAI” tested the RAI score only; the second model, “RAI-P” (RAI-Procedure) included the RAI, procedure category, and procedure approach; the third model, “RAI-PC” (RAI-Procedure Complexity) included the RAI as well as pre- and postoperative procedural characteristics (procedure category, procedure approach, outpatient/inpatient surgery status, general anesthesia use, RVUs, and operative time).
We randomly split the cohort into a training set (70% of observations) and a validation set (30% of observations). We used chi-square tests to compare demographics, procedure characteristics, and outcomes across the training and validation sets to confirm they were comparable (Appendix C). All three logistic regression models were created on the training set and then tested for discrimination and calibration using the validation sets. Discrimination, or each model’s ability to predict the occurrence of the outcome, was assessed by calculation of C-statistics, also known as the area under the curve (AUC). A C-statistic of 0.5 would indicate the model’s predictive ability is no better than random chance, and a C-statistic of 1.0 demonstrates the model perfectly discriminates an occurrence versus non-occurrence. Generally, C-statistics can be interpreted as: excellent (0.9–1.0), good (0.8–0.89), fair (0.7–0.79), poor (0.6–0.69), or no discriminatory capacity (0.5–0.59).20,21 C-statistics and 95% confidence intervals were calculated and Receiver Operating Characteristic (ROC) curves were generated to assess model discrimination for the entire test data set as well as within each procedure category and procedure approach. We also generated model calibration plots to determine how well each model’s predicted probabilities align with the observed outcome rates in different groups of patients according to level of frailty.22,23 Calibration plots illustrate the predicted probabilities from the logistic regression models developed from the training dataset as well as the observed outcome rates and 95% confidence intervals from the validation dataset. We used Stata v.15 (College Station, TX: StataCorp LLC) for all data analyses.
RESULTS
We included 139,569 elective vascular surgery patients from the 2007–2013 ACS-NSQIP database. The demographics and baseline characteristics of the entire cohort stratified by procedure category are shown in Table I. The mean RAI for the entire cohort was 24.5 (SD 5.7); mean RAI ranged from 22.4 (SD 6.3) for suprainguinal revascularizations to 25.8 (SD 4.7) for AAA repairs. The highest proportion of patients had RAI scores between 25 and 29 (34.7%, Figure I). In the total cohort, 14.5% of patients undergoing procedures were Frail (RAI≥30) and 4.8% were Very Frail (RAI≥35). Of the procedure categories, patients undergoing infrainguinal revascularizations represented the largest percentage of Frail patients (17.8%), followed by TAAs (17.1%), and carotid revascularizations (15.3%); chi-square p<0.001. Similarly, the procedure categories with the largest proportions of Very Frail patients were infrainguinal revascularizations (7.5%), TAAs (6.1%), and AAAs (4.8%); p<0.001 [Table I]. Patients with RAI≥30 were more likely to undergo an endovascular compared to an open procedure (26.7% versus 22.7%, p<0.001) (Figure I).
Table I. Demographics and baseline characteristics for the entire study cohort, stratified by procedure category.
Abbreviations: Abdominal Aortic Aneurysm (AAA); Thoracic Aortic Aneurysm (TAA); Thoracoabdominal Aortic Aneurysm (TAAA); Body Mass Index (BMI); Congestive Heart Failure (CHF); Activities of Daily Living (ADL); American Society of Anesthesiologists (ASA). All chi-square and ANOVA tests comparing baseline characteristics among the procedure groups are statistically significant, p<0.001.
| Total Sample N = 139,569 | AAA N = 25,307 | Carotid N = 51,978 | Infrainguinal N = 42,078 | Suprainguinal N = 15,535 | TAA N = 1,841 | TAAA N = 2,830 | |
| Mean (SD) | Mean (SD) | Mean (SD) | Mean (SD) | Mean (SD) | Mean (SD) | Mean (SD) | |
| Age | 70.0 (10.6) | 73.3 (8.9) | 71.2 (9.5) | 68.3 (11.5) | 65.0 (11.3) | 68.8 (12.9) | 70.9 (10.1) |
| Preoperative BMI (Missing N=1,964) | 27.9 (5.9) | 28.1 (5.7) | 28.4 (5.9) | 27.7 (6.1) | 26.7 (5.9) | 27.8 (6.1) | 27.5 (5.5) |
| N (%) | N (%) | N (%) | N (%) | N (%) | N (%) | N (%) | |
| Ethnicity (Missing N=6,736) | |||||||
| Not Hispanic | 128,181 (94.1) | 23,498 (98.0) | 48,406 (96.9) | 37,780 (94.8) | 14,115 (96.9) | 1,736 (97.4) | 2,646 (97.9) |
| Hispanic | 4,652 (3.5) | 488 (2.0) | 1,550 (3.1) | 2,066 (5.2) | 446 (3.1) | 46 (2.6) | 56 (2.1) |
| Race | |||||||
| White | 114,568 (82.1) | 21,522 (85.0) | 45,334 (87.2) | 31,610 (75.1) | 12,335 (79.4) | 1,345 (73.1) | 2,422 (85.6) |
| Black | 11,801 (8.5) | 1,204 (4.8) | 2,257 (4.3) | 6,302 (15.0) | 1,589 (10.2) | 290 (15.8) | 159 (5.6) |
| Other/Unknown | 13,200 (9.5) | 2,581 (10.2) | 4,387 (8.4) | 4,166 (9.9) | 1,611 (10.4) | 206 (11.2) | 249 (8.8) |
| Male Sex | 90,054 (64.5) | 20,266 (80.1) | 30,946 (59.5) | 26,819 (63.7) | 8,978 (57.8) | 1,026 (55.7) | 2,019 (71.3) |
| Diabetes | |||||||
| Non-Insulin | 23,124 (16.6) | 3,084 (12.2) | 9,621 (18.5) | 7,764 (18.5) | 2,190 (14.1) | 182 (9.9) | 283 (10.0) |
| Insulin | 18,264 (13.1) | 788 (3.1) | 5,364 (10.3) | 10,160 (24.2) | 1,819 (11.7) | 57 (3.1) | 76 (2.7) |
| CHF | 2,396 (1.7) | 363 (1.4) | 578 (1.1) | 1,096 (2.6) | 286 (1.8) | 40 (2.2) | 33 (2.2) |
| Dyspnea | |||||||
| Moderate Exertion | 23,064 (16.5) | 5,121 (20.2) | 8,362 (16.1) | 5,985 (14.2) | 2,621 (16.9) | 411 (22.3) | 564 (19.9) |
| At Rest | 1,835 (1.3) | 396 (1.6) | 561 (1.1) | 575 (1.4) | 229 (1.5) | 42 (2.3) | 32 (1.1) |
| Renal Failure | 4,423 (3.2) | 355 (1.4) | 631 (1.2) | 2,876 (6.8) | 440 (2.8) | 79 (4.3) | 42 (1.5) |
| ADL Status | |||||||
| Partial Dependence | 8,658 (6.2) | 793 (3.1) | 2,138 (4.1) | 4,352 (10.3) | 1,141 (7.3) | 133 (7.2) | 101 (3.6) |
| Total Dependence | 990 (0.7) | 111 (0.4) | 174 (0.3) | 477 (1.1) | 165 (1.1) | 33 (1.8) | 30 (1.1) |
| Impaired Cognition | 8,625 (6.2) | 816 (3.2) | 5,211 (10.0) | 1,841 (4.4) | 559 (3.6) | 97 (5.3) | 101 (3.6) |
| Cancer | 522 (0.4) | 128 (0.5) | 124 (0.2) | 158 (0.4) | 87 (0.6) | 7 (0.4) | 18 (0.6) |
| Weight Loss | 1,436 (1.0) | 330 (1.3) | 260 (0.5) | 459 (1.1) | 278 (1.8) | 55 (3.0) | 54 (1.9) |
| Dependent Living | 5,529 (4.0) | 636 (2.5) | 1,452 (2.8) | 2,322 (5.5) | 789 (5.1) | 193 (10.5) | 137 (4.8) |
| ASA Class (Missing N=834) | |||||||
| 1-No Disturb / 2-Mild | 10,584 (7.6) | 1,595 (6.3) | 3,993 (7.7) | 3,556 (8.6) | 1,218 (7.9) | 67 (3.7) | 155 (5.5) |
| 3-Severe | 101,869 (73.4) | 17,882 (70.7) | 40,278 (77.6) | 29,970 (72.2) | 11,021 (71.8) | 1,028 (56.0) | 1,690 (59.8) |
| 4-Life Threat / 5-Moribund | 26,282 (18.9) | 5,806 (23.0) | 7,640 (14.7) | 8,004 (19.3) | 3,107 (20.3) | 742 (40.4) | 983 (34.8) |
| Risk Analysis Index (RAI) Score | |||||||
| RAI, Mean (SD) | 24.5 (5.7) | 25.8 (4.7) | 24.6 (5.0) | 24.4 (6.5) | 22.4 (6.3) | 24.3 (6.8) | 24.8 (5.3) |
| RAI<30 [Non-Frail], N (%) | 119,273 (85.5) | 21,442 (84.7) | 45,379 (87.3) | 34,599 (82.2) | 13,841 (89.1) | 1,527 (82.9) | 2,485 (87.8) |
| RAI≥30 [Frail], N (%) | 20,296 (14.5) | 3,865 (15.3) | 6,599 (12.7) | 7,479 (17.8) | 1,694 (10.9) | 314 (17.1) | 345 (12.2) |
| RAI≥35 [Very Frail], N (%) | 6,719 (4.8) | 983 (3.9) | 1,679 (3.2) | 3,159 (7.5) | 678 (4.4) | 111 (6.1) | 109 (3.9) |
Figure I.
Distribution of Risk Analysis Index (RAI) in the entire cohort (top) and RAI score distribution when stratified by procedure type (endovascular vs open, bottom).
In Table II, we summarize procedure characteristics and outcomes for the study cohort, stratified by procedure category. There were 26,536 major complications in the entire cohort (19.0% of patients). Patients undergoing TAAAs had the highest proportion of patients with 30-day complications (48.7%), followed by TAAs (30.7%), and suprainguinal revascularizations (29.0%); chi-square p<0.001. Patients undergoing carotid procedures had the lowest proportion of 30-day complications (8.2%). As with major complications, mortality was highest for TAAAs (6.9%), TAAs (5.1%), and suprainguinal revascularizations (2.7%); chi-square p<0.001. The frequencies and proportions of each major complication type, stratified by procedure type are shown in Appendix D.
Table II. Procedure characteristics and outcomes for the entire study cohort, stratified by procedure category.
Abbreviations: Abdominal Aortic Aneurysm (AAA); Thoracic Aortic Aneurysm (TAA); Thoracoabdominal Aortic Aneurysm (TAAA); Relative Value Units (RVUs); Length of Stay (LOS); Risk Analysis Index (RAI). All chi-square and ANOVA tests comparing procedural characteristics and outcomes among the procedure groups are statistically significant, p<0.001.
| Total Sample N = 139,569 | AAA N = 25,307 | Carotid N = 51,978 | Infrainguinal N = 42,078 | Suprainguinal N = 15,535 | TAA N = 1,841 | TAAA N = 2,830 | |
| Procedure Characteristics / Outcomes | Mean (SD) | Mean (SD) | Mean (SD) | Mean (SD) | Mean (SD) | Mean (SD) | Mean (SD) |
| Postoperative Length of Stay (Missing N=83) | 3.9 (5.9) | 4.1 (6.1) | 2.0 (3.1) | 5.0 (6.2) | 5.9 (7.5) | 6.5 (8.3) | 9.9 (12.0) |
| Work RVUs | 22.4 (7.2) | 25.9 (5.3) | 19.6 (0.04) | 22.0 (7.0) | 23.9 (9.2) | 31.0 (4.6) | 36.1 (24.6) |
| Operative Time, Minutes (Missing N=15) | 162.1 (96.6) | 168.9 (87.7) | 115.0 (47.9) | 193.2 (107.2) | 201.9 (114.0) | 174.4 (114.7) | 276.1 (145.4) |
| N (%) | N (%) | N (%) | N (%) | N (%) | N (%) | N (%) | |
| Procedure Type | |||||||
| Open | 107,077 (76.7) | 5,587 (22.1) | 51,403 (98.9) | 34,899 (82.9) | 12,950 (83.4) | 89 (4.8) | 2,149 (75.9) |
| Endovascular | 32,492 (23.3) | 19,720 (77.9) | 575 (1.1) | 7,179 (17.1) | 2,585 (16.6) | 1,752 (95.2) | 681 (24.1) |
| Surgery Status | |||||||
| Outpatient | 6,932 (5.0) | 372 (1.5) | 1,288 (2.5) | 3,767 (9.0) | 1,473 (9.5) | 15 (0.8) | 17 (0.6) |
| Inpatient | 132,637 (95.0) | 24,935 (98.5) | 50,690 (97.5) | 38,311 (91.1) | 14,062 (90.5) | 1,826 (99.2) | 2,813 (99.4) |
| General Anesthesia | 118,277 (84.7) | 22,768 (90.0) | 43,978 (84.6) | 33,781 (80.3) | 13,239 (85.2) | 1,749 (95.0) | 2,762 (97.6) |
| Prolonged LOS (Missing N=83) | 45,814 (32.8) | 7,385 (29.2) | 18,512 (35.6) | 13,631 (32.4) | 5,014 (32.3) | 507 (27.6) | 765 (27.1) |
| 30-day Major Complication | 26,536 (19.0) | 4,967 (19.6) | 4,274 (8.2) | 10,842 (25.8) | 4,510 (29.0) | 565 (30.7) | 1,378 (48.7) |
| 30-day Mortality, Total | 2,482 (1.8) | 502 (2.0) | 447 (0.9) | 831 (2.0) | 413 (2.7) | 94 (5.1) | 195 (6.9) |
| 30-day Mortality, Non-Frail* | 1,424 (1.2) | 324 (1.5) | 279 (0.6) | 363 (1.1) | 244 (1.8) | 66 (4.3) | 148 (6.0) |
| 30-day Mortality, Frail* | 1,058 (5.2) | 178 (4.6) | 168 (2.6) | 468 (6.3) | 169 (10.0) | 28 (8.9) | 47 (13.6) |
| 30-day Mortality, Very Frail* | 597 (8.9) | 80 (8.1) | 82 (4.9) | 297 (9.4) | 94 (13.9) | 18 (16.2) | 26 (23.9) |
The denominator for each of these subcategories is different based on the operation performed and frailty category.
The training and test groups were similar in demographics, procedure characteristics and outcomes (Appendix C). There was excellent calibration as shown in the calibration plot (Figure II) comparing the predicted probability of 30-day mortality versus the observed mortality rate per RAI score. At all RAI levels, the observed mortality and 95% CI overlaps with the predicted mortality. The overall 30-day mortality was 2,482 (1.8%). Mortality at 30 days increased with increasing RAI score, where 12.1% of patients with RAI 40–44 died, and 18.8% of patients with RAI≥45 died (Figure II). When studied across procedure categories, mortality continued to be substantially higher for Frail and Very Frail patients (Table II). For suprainguinal revascularizations, 30-day mortality for Frail patients increased nearly 4-fold compared to the total cohort (2.7% versus 10.0%, p<0.001). Mortality was at least 4 times greater for Very Frail vs. Non-Frail patients within each procedure category, except for TAAs and TAAAs, which was 3 times greater (5.1% versus 16.2% for TAAs and 6.9% versus 23.9% for TAAAs; both p <0.001).
Figure II.
Calibration plot demonstrating the predicted probability versus the observed mortality rate per Risk Analysis Index (RAI) score.
The discrimination of our RAI model and the RAI-PC model for 30-day mortality, major complication, and prolonged LOS are demonstrated in Figure III. Alone, the RAI was a robust predictor of mortality for all combined vascular procedures (C-statistic 0.72). When stratified by procedure category, the RAI model showed fair discrimination for carotid (0.70), AAA repairs (0.69), infrainguinal (0.79), and suprainguinal revascularizations (0.74). In contrast, the RAI model showed poor discrimination for mortality for TAAs (0.62) and TAAAs (0.54). The RAI-PC model, however, demonstrated robust discrimination for mortality across all procedure categories, with C-statistics ranging from 0.72 (carotid revascularizations) to 0.84 (AAA repairs). While the RAI model was a poor predictor of major complications and prolonged LOS, the procedure complexity in the RAI-PC model improved the discrimination for major complications in all procedures combined (0.75), as well as for AAA repairs (0.78), suprainguinal revascularizations (0.73), TAAs (0.70), and TAAAs (0.75). The full model also increased the discrimination for prolonged LOS in infrainguinal (0.75) and suprainguinal revascularizations (0.78), as well as TAAs (0.73), and TAAAs (0.72).
Figure III.
Forest plot comparing the C-statistics for Risk Analysis Index (RAI)-only model performance versus the RAI-Procedure Complexity (RAI-PC) model for the three outcomes of interest (mortality, major complication, and prolonged length of stay [LOS]), stratified by procedure category. The RAI-PC model adjusted for procedural characteristics: procedure category (e.g., AAA repair), procedure type (endovascular versus open), outpatient versus inpatient surgery status, general anesthesia use, work relative value units, and operative time. Abbreviations: Abdominal Aortic Aneurysm (AAA); Thoracic Aortic Aneurysm (TAA); Thoracoabdominal Aortic Aneurysm (TAAA).
When the 30-day mortality models were stratified by both procedure category and procedure approach (endovascular versus open), the RAI model demonstrated moderate to excellent discrimination (C-statistic range 0.70 to 0.90) with the exception of carotid stents (0.66), endovascular TAAs (0.62), and both endovascular and open TAAAs (0.69 and 0.57, respectively); Table III. Despite modest performance of the RAI in the carotid stent and thoracic/thoracoabdominal aneurysms, the models all improved once additional procedure characteristics were added, demonstrating again that the RAI-PC model was a robust predictor of 30-day mortality. For endovascular TAAA procedures, the discrimination improved from 0.69 in the RAI model to 0.98 in the RAI-PC model; for open TAAAs, the discrimination improved from 0.57 in the RAI model to 0.74 in the RAI-PC model. The effect of procedure characteristics on the 30-day mortality models’ discrimination is demonstrated in Figure IV. There was little difference between in the discrimination of the RAI-P model versus the RAI-PC model. While the RAI-only model was a good predictor of mortality (0.72), the RAI-P (0.78) and RAI-PC (0.79) models were more robust predictors.
Table III. C-Statistics comparing the Risk Analysis Index (RAI)-only model discrimination versus the RAI-Procedure Complexity (RAI-PC) model including procedure characteristics for 30-day mortality, stratified by procedure category (e.g., AAA repair) and procedure type (endovascular versus open). The RAI-PC model adjusted for procedure category, procedure type, outpatient versus inpatient surgery status, general anesthesia use, work relative value units, and operative time.
Abbreviations: Peripheral Artery Disease (PAD), Abdominal Aortic Aneurysm (AAA), Thoracic Aortic Aneurysm (TAA), Thoracoabdominal Aortic Aneurysm (TAAA).
| Procedure Category | Endovascular Surgery N = 32,492 | Open Surgery N = 107,077 | ||
|---|---|---|---|---|
| C-statistic RAI [95% CI] | C-statistic RAI-PC [95% CI] | C-statistic RAI [95% CI] | C-statistic RAI-PC [95% CI] | |
| All Procedures | 0.719 [0.67–0.77] | 0.805 [0.77–0.84] | 0.723 [0.70–0.75] | 0.795 [0.78–0.81] |
| AAA | 0.722 [0.66–0.78] | 0.840 [0.80–0.88] | 0.713 [0.65–0.77] | 0.765 [0.71–0.82] |
| Carotid | 0.656 [0.39–0.92] | 0.761 [0.47–1.00] | 0.703 [0.65–0.75] | 0.724 [0.68–0.77] |
| Infrainguinal | 0.759 [0.67–0.85] | 0.796 [0.71–0.88] | 0.792 [0.76–0.82] | 0.791 [0.76–0.82] |
| Suprainguinal | 0.903 [0.82–0.99] | 0.893 [0.79–1.00] | 0.728 [0.68–0.78] | 0.772 [0.73–0.82] |
| TAA | 0.615 [0.46–0.77] | 0.742 [0.63–0.86] | * | * |
| TAAA | 0.685 [0.08–1.00] | 0.982 [0.95–1.00] | 0.565 [0.48–0.65] | 0.740 [0.68–0.81] |
There were too few observations in the open TAA procedure category to test model performance.
Figure IV.
Receiver Operating Characteristic (ROC) curve comparing a model with the Risk Analysis Index (RAI)-only model versus the RAI-Procedure (RAI-P) model, and the RAI-Procedure Complexity (RAI-PC) model. The RAI-P model adjusted for procedure category (e.g., abdominal aortic aneurysm) and procedure type (e.g., endovascular abdominal aortic aneurysm repair). The RAI-PC adjusted for procedure category and procedure type, as well as the procedural characteristics (outpatient versus inpatient surgery status, general anesthesia use, work relative value units, and operative time).
DISCUSSION
In this study, we provide evidence that the RAI is a useful measure of frailty in vascular surgery patients and showed that the RAI can predict postoperative mortality across six major elective vascular procedure categories. When analyzed by procedure category or procedure approach, the RAI was a robust predictor for mortality and major complications in the setting of suprainguinal and infrainguinal revascularizations for peripheral arterial disease (PAD) and endovascular AAA repair, and a modest predictor of postoperative mortality for open AAA and CEA. While the RAI was not a strong predictor of mortality in TAA or TAAA repairs, or for major complications or prolonged length of stay for all categories, the addition of procedural complexity greatly improved the overall discrimination of the mortality models in TAA and TAAA patients and prolonged length of stay for all procedures suggesting the importance of the magnitude of the stressor. The lack of change in RAI-PC model performance for major complications suggests other patient characteristics and peri-procedural care may contribute more to morbidity from vascular procedures than patient frailty or the surgery itself.
As expected, mortality rates increased as the RAI score increased both for all vascular surgery patients, and across all six major vascular surgery categories. The differences in mortality rates between Frail patients compared to the Non-Frail patients were particularly striking. In this study, the 30-day mortality probability was 1.8% among all vascular procedures, 5.2% in Frail patients (RAI≥30), and 8.9% in Very Frail patients (RAI≥35). For the highest frailty scores, 30-day mortality for all vascular procedures was prohibitively high, and should give patients and surgeons pause before considering operative intervention. For specific procedure categories, mortality for the higher RAI scores were even higher (TAAA, TAA, Suprainguinal) suggesting the importance of considering frailty prior to offering surgery. Though only 4.8% of our study cohort was Very Frail, routine preoperative frailty screening in vascular clinics can easily identify this high-risk group, thereby minimizing interruptions to workflow.1 Novel care pathways and resources can then be allocated to this small, but high-risk group, including shared decision-making support, referral to geriatrics or palliative care services, and pre-habilitation.
The RAI proved to be a strong discriminator of postoperative mortality after vascular procedures in our study. Since frailty is defined by an individual’s response to a stressor, understanding the dose or magnitude of stress that is experienced by the patient with each approach of surgery are vital in our ability to adequately describe patient risks and benefits. In our earlier frailty research, we have shown that frail patients are vulnerable to adverse outcomes in both low-risk and high-risk procedures.11 It is known that the short-term mortality after endovascular procedures is often better than open procedures,24 and this is likely in part due to the immediate physiologic demands placed on patients from a more invasive, open surgery. In contrast, patients who underwent an endovascular AAA repair were shown to have a significantly lower hypermetabolic stress response and nutritional deterioration.25
However, frail patients have similar long-term mortality trajectory beyond the initial postoperative period for AAA repair.26 In our study, frailty was a better predictor of mortality in open versus endovascular procedures, suggesting a greater role for frailty in this minimally invasive subgroup. The open aneurysm (AAA/TAA/TAAA) patients had lower RAI scores than endovascular aneurysm repairs, indicating a pattern of careful patient selection for the more “stressful” procedures. This stringent selection process or “eyeball test” likely led to a more homogenous group of robust patients undergoing aneurysm repairs. Furthermore, the addition of procedural characteristics to the RAI models for aneurysm procedures led to improved model performance, thus providing additional support to our hypothesis that operative stress modulates the impact of frailty on perioperative mortality. However, the RAI-P and RAI-PC models were essentially equal predictors for mortality (Figure IV), suggesting that using pre-operative characteristics only (RAI, procedure category, and procedure approach) may be enough for robust mortality prediction. In contrast, the RAI was an equally strong predictor of postoperative mortality after extremity revascularization procedures, suggesting a smaller role for procedural characteristics and a higher frailty impact. Procedural characteristics also improved the discrimination for prolonged LOS. We suspect that frailty had little impact on LOS without considering procedural complexity because the procedures included in the study may have highly standardized patterns of care. For example, a carotid endarterectomy will remain in the hospital approximately one day compared to a patient who has an open bypass for PAD that would require a longer length of stay for monitoring and rehabilitation. In addition to operative time and general anesthesia use, RVUs can be a surrogate for procedure complexity.9 Though the use of RVUs alone as a predictor of surgical effort and complexity is controversial,10 combined with the other procedure characteristics, RVUs have been shown to improve predictive models.27
The RAI predicted postoperative mortality well, but was a poor predictor of major complications in vascular surgery patients. Even after procedure characteristics were included in the model, it remained a weak predictor of complications, suggesting a limited role of procedural stress/magnitude on complication occurrence. Although frailty is associated with increased complications,4 frailty alone may not be a good predictor of morbidity in vascular surgery patients due to a higher frailty burden compared to other specialties. There may be also be other patient factors such as specific comorbidities (diabetes), BMI or anemia that are bigger contributors to specific complications, such as infections. Perioperative care protocols such as intensive care unit use, venous thromboembolism prophylaxis, and infection control strategies may have stronger impact on complications. Other risk calculators have performed better at predicting specific complications, such as the Vascular Study Group of New England Cardiac Risk Index (VSGNE CRI) for adverse outcomes after carotid endarterectomy.28 This is likely because the VSGNE CRI was specifically designed to evaluate carotid endarterectomy rather than being applied to multiple procedures within a specialty. Broader risk calculators such as the P-POSSUM have also performed poorly at predicting complications in vascular surgery patients.29,30 One explanation might be that frailty as a geriatric concept is a stronger marker of the end of life rather than a predictor of short-term outcomes, and this may be reflected in the suboptimal performance of the RAI in predicting short term procedural complications.
Overall, we recommend applying the RAI as a preoperative screening tool to identify high-risk patients who are vulnerable to poor outcomes, as shown by a growing body of literature.1,4,31–33 The RAI is based on the deficits accumulation model of frailty as advanced by Rockwood, but is in fact a weighted score. Other frailty measures in vascular literature include the modified Frailty Index (mFI)34,35, the Clinical Frailty Scale (CFS),5,36 and morphometrics.37,38 Each has its own strength and purpose, but when compared to the RAI, they have known limitations. The mFI has not been prospectively validated,4,11,31 and the CFS is not as granular as the RAI with regard to functional status. Also, the CFS includes morphometrics requiring CT scans or other time-intensive metrics.2 In comparison, the RAI incorporates more domains relevant to frailty and frailty-related outcomes such as nutrition, living status, and cognition,3,33,39 and uses patient-reported metrics of functional status. An additional benefit is that the RAI does not rely on any imaging, lab parameters, or specialized phenotypic instruments. Since its application only takes 1–2 minutes, the RAI has been successfully incorporated into a system-wide screening of surgical populations with a resultant decrease in surgical mortality.1,6,40 Patient classified as Frail or Very Frail by the RAI could then have more detailed discussions about not only surgical treatments, but also alternative treatment strategies and goals of care. For example, the RAI can identify patients with limited life expectancy who may not live long enough to experience the benefits of preventive surgery such as carotid endarterectomy for asymptomatic carotid disease. Screening for frailty can also help in shared decision-making around timing of operative intervention, where the risks of surgery are much higher than the risks of surveillance, e.g. endovascular AAA repair for small aneurysms in frail patients. While RAI did not predict complications as well, frail patients experiencing major complications after complex vascular procedures, both open and endovascular, can result in failure to rescue (death) or institutionalization.11,41 The RAI could essentially introduce a “surgical pause” in the shared decision-making process, leading to the introduction of advanced directives, post-discharge planning, and the role for rehabilitation for this high-risk patient subgroup. Future research is needed to measure the impact of frailty screening in vascular care pathways and quality improvement.
This study has limitations, which are inherent to the nature of a retrospective, database design. We did not have access to center-level or surgeon-level identification, so we were unable to account for clustering observations within hospital. Center- and surgeon-level characteristics were also unavailable, such as whether the procedure was performed at a high-volume center, which might have improved the model. However, the PCs that were ultimately chosen did appear to improve the model significantly. While the ACS-NSQIP dataset includes observations from a wide range of hospitals, most hospitals that contribute data are large academic centers, therefore our results may not be generalizable to smaller community or rural hospitals. Also, the RAI version used in this study is intended for application in registry data. While the prospective version has high correlation with the retrospective version in surgical patients, and both were validated in surgical populations up to 180 and 365 days,1 further testing within vascular surgery patients may be needed.
CONCLUSIONS
In conclusion, frailty as measured by the RAI should be considered during preoperative evaluation for vascular surgery patients as it is a robust predictor of postoperative mortality. The RAI performs better in patients undergoing suprainguinal and infrainguinal PAD revascularizations (open and endovascular) as a well as EVAR, with modest performance in carotid and open AAA repair patients. Procedure characteristics were utilized as surrogate markers for operative stress and helped to improve model performance in TAA and TAAA patients. Further study is needed to determine the relative contributions of frailty as measured by the RAI and various procedural characteristics in predicting long-term survival and outcomes for vascular surgery patients.
Supplementary Material
ACKNOWLEDGEMENTS
This work was supported by the U.S. Department of Health and Human Services (R03-AG050930-02).
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
REFERENCES
- [1].Hall DE, Arya S, Schmid KK, et al. Association of a Frailty Screening Initiative With Postoperative Survival at 30, 180, and 365 Days. JAMA Surg. 2017;152(3):233–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [2].Kraiss LW, Beckstrom JL, Brooke BS. Frailty assessment in vascular surgery and its utility in preoperative decision making. Semin Vasc Surg. 2015;28(2):141–7. [DOI] [PubMed] [Google Scholar]
- [3].Makary MA, Segev DL, Pronovost PJ, et al. Frailty as a predictor of surgical outcomes in older patients. J Am Coll Surg. 2010;210(6):901–8. [DOI] [PubMed] [Google Scholar]
- [4].Arya S, Kim SI, Duwayri Y, et al. Frailty increases the risk of 30-day mortality, morbidity, and failure to rescue after elective abdominal aortic aneurysm repair independent of age and comorbidities. J Vasc Surg. 2015;61(2):324–31. [DOI] [PubMed] [Google Scholar]
- [5].Rockwood K, Song X, MacKnight C, et al. A global clinical measure of fitness and frailty in elderly people. CMAJ. 2005;173(5):489–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [6].Hall DE, Arya S, Schmid KK, et al. Development and Initial Validation of the Risk Analysis Index for Measuring Frailty in Surgical Populations. JAMA Surg. 2017;152(2):175–82. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [7].Arya S, Varley P, Youk A, et al. Recalibration and External Validation of the Risk Analysis Index: A Surgical Frailty Assessment Tool. Ann Surg. 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [8].Neupane I, Arora RC, Rudolph JL. Cardiac surgery as a stressor and the response of the vulnerable older adult. Experimental gerontology. 2017;87(Pt B):168–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [9].Nguyen KT, Gart MS, Smetona JT, et al. The Relationship Between Relative Value Units and Outcomes: A Multivariate Analysis of Plastic Surgery Procedures. Eplasty. 2012;12(500–10). [PMC free article] [PubMed] [Google Scholar]
- [10].Shah DR, Bold RJ, Yang AD, et al. Relative value units poorly correlate with measures of surgical effort and complexity. J Surg Res. 2014;190(2):465–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [11].Shah R, Attwood K, Arya S, et al. Association of Frailty With Failure to Rescue After Low-Risk and High-Risk Inpatient Surgery. JAMA Surg. 2018;153(5):e180214. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [12].American College of Surgeons National Surgical Quality Improvement Program. User Guide for the 2007 ACS NSQIP Participant Use Data File. https://www.facs.org/~/media/files/quality%20programs/nsqip/ug07.ashx.
- [13].American College of Surgeons National Surgical Quality Improvement Program. User Guide for the 2012 ACS NSQIP Participant Use Data File. https://www.facs.org/~/media/files/quality%20programs/nsqip/ug12.ashx.
- [14].American College of Surgeons National Surgical Quality Improvement Program. User Guide for the 2013 ACS NSQIP Participant Use Data File. https://www.facs.org/~/media/files/quality%20programs/nsqip/2013_acs_nsqip_puf_user_guide.ashx.
- [15].American College of Surgeons National Surgical Quality Improvement Program. User Guide for the 2014 ACS NSQIP Participant Use Data File. https://www.facs.org/~/media/files/quality%20programs/nsqip/nsqip_puf_userguide_2014.ashx.
- [16].American College of Surgeons National Surgical Quality Improvement Program. User Guide for the 2011 ACS NSQIP Participant Use Data File. https://www.facs.org/~/media/files/quality%20programs/nsqip/ug11.ashx.
- [17].American College of Surgeons National Surgical Quality Improvement Program. User Guide for the 2010 ACS NSQIP Participant Use Data File. https://www.facs.org/~/media/files/quality%20programs/nsqip/ug10.ashx.
- [18].American College of Surgeons National Surgical Quality Improvement Program. User Guide for the 2009 ACS NSQIP Participant Use Data File. https://www.facs.org/~/media/files/quality%20programs/nsqip/ug09.ashx.
- [19].American College of Surgeons National Surgical Quality Improvement Program. User Guide for the 2008 ACS NSQIP Participant Use Data File. https://www.facs.org/~/media/files/quality%20programs/nsqip/ug08.ashx.
- [20].Fischer JE, Bachmann LM, Jaeschke R. A readers’ guide to the interpretation of diagnostic test properties: clinical example of sepsis. Intensive Care Med. 2003;29(7):1043–51. [DOI] [PubMed] [Google Scholar]
- [21].Hosmer DW, Lemeshow S. Applied Logistic Regression. New York: John Wiley & Sons, Inc.; 2000. [Google Scholar]
- [22].Alba AC, Agoritsas T, Walsh M, et al. Discrimination and Calibration of Clinical Prediction Models: Users’ Guides to the Medical Literature. JAMA. 2017;318(14):1377–84. [DOI] [PubMed] [Google Scholar]
- [23].Van Calster B, Nieboer D, Vergouwe Y, et al. A calibration hierarchy for risk models was defined: from utopia to empirical data. J Clin Epidemiol. 2016;74:167–76. [DOI] [PubMed] [Google Scholar]
- [24].Paravastu SC, Jayarajasingam R, Cottam R, et al. Endovascular repair of abdominal aortic aneurysm. Cochrane Database Syst Rev. 2014(1):Cd004178. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [25].Salartash K, Sternbergh WC 3rd, York JW, et al. Comparison of open transabdominal AAA repair with endovascular AAA repair in reduction of postoperative stress response. Ann Vasc Surg. 2001;15(1):53–9. [DOI] [PubMed] [Google Scholar]
- [26].George EL, Trickey AW, Brooke BS, et al. Variation in Center-Level Frailty Burden and Its Impact on Long-Term Survival in Patients Undergoing Repair for Abdominal Aortic Aneurysms. J Vasc Surg. 2019;In Press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [27].Merkow RP, Bentrem DJ, Cohen ME, et al. Effect of cancer surgery complexity on short-term outcomes, risk predictions, and hospital comparisons. J Am Coll Surg. 2013;217(4):685–93. [DOI] [PubMed] [Google Scholar]
- [28].Eslami MH, Rybin D, Doros G, et al. An externally validated robust risk predictive model of adverse outcomes after carotid endarterectomy. J Vasc Surg. 2016;63(2):345–54. [DOI] [PubMed] [Google Scholar]
- [29].Teixeira IM, Teles AR, Castro JM, et al. Physiological and Operative Severity Score for the enumeration of Mortality and Morbidity (POSSUM) System for Outcome Prediction in Elderly Patients Undergoing Major Vascular Surgery. J Cardiothorac Vasc Anesth. 2018;32(2):960–7. [DOI] [PubMed] [Google Scholar]
- [30].Kuhan G, Abidia AF, Wijesinghe LD, et al. POSSUM and P-POSSUM overpredict mortality for carotid endarterectomy. Eur J Vasc Endovasc Surg. 2002;23(3):209–11. [DOI] [PubMed] [Google Scholar]
- [31].Ehlert BA, Najafian A, Orion KC, et al. Validation of a modified Frailty Index to predict mortality in vascular surgery patients. J Vasc Surg. 2016;63(6):1595–601 e2. [DOI] [PubMed] [Google Scholar]
- [32].Morisaki K, Yamaoka T, Iwasa K, et al. Influence of frailty on treatment outcomes after revascularization in patients with critical limb ischemia. J Vasc Surg. 2017;66(6):1758–64. [DOI] [PubMed] [Google Scholar]
- [33].Wang J, Zou Y, Zhao J, et al. The Impact of Frailty on Outcomes of Elderly Patients After Major Vascular Surgery: A Systematic Review and Meta-analysis. Eur J Vasc Endovasc Surg. 2018;56(4):591–602. [DOI] [PubMed] [Google Scholar]
- [34].Rockwood K, Andrew M, Mitnitski A. A comparison of two approaches to measuring frailty in elderly people. The journals of gerontology Series A, Biological sciences and medical sciences. 2007;62(7):738–43. [DOI] [PubMed] [Google Scholar]
- [35].Mitnitski AB, Mogilner AJ, Rockwood K Accumulation of deficits as a proxy measure of aging. TheScientificWorldJournal. 2001;1:323–36. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [36].Donald GW, Ghaffarian AA, Isaac F, et al. Preoperative frailty assessment predicts loss of independence after vascular surgery. J Vasc Surg. 2018;68(5):1382–9. [DOI] [PubMed] [Google Scholar]
- [37].Hale AL, Twomey K, Ewing JA, et al. Impact of sarcopenia on long-term mortality following endovascular aneurysm repair. Vasc Med. 2016;21(3):217–22. [DOI] [PubMed] [Google Scholar]
- [38].Matsubara Y, Matsumoto T, Aoyagi Y, et al. Sarcopenia is a prognostic factor for overall survival in patients with critical limb ischemia. J Vasc Surg. 2015;61(4):945–50. [DOI] [PubMed] [Google Scholar]
- [39].Robinson TN, Eiseman B, Wallace JI, et al. Redefining geriatric preoperative assessment using frailty, disability and co-morbidity. Ann Surg. 2009;250(3):449–55. [DOI] [PubMed] [Google Scholar]
- [40].Melin AA, Schmid KK, Lynch TG, et al. Preoperative frailty Risk Analysis Index to stratify patients undergoing carotid endarterectomy. J Vasc Surg. 2015;61(3):683–9. [DOI] [PubMed] [Google Scholar]
- [41].Arya S, Long CA, Brahmbhatt R, et al. Preoperative Frailty Increases Risk of Nonhome Discharge after Elective Vascular Surgery in Home-Dwelling Patients. Ann Vasc Surg. 2016;35:19–29. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.