Key Points
Question
Is a standardized Frailty Screening Initiative associated with a reduction in 1-year mortality in patients undergoing elective surgery?
Findings
In this quality improvement study of 50 463 patients who underwent major operations, the implementation of a Frailty Screening Initiative using the Risk Analysis Index linked to an electronic health record alert was associated with significantly reduced systemwide, 1-year postoperative mortality from 3.9% to 3.3%. In frail patients specifically, 1-year mortality decreased significantly from 20.2% to 16.0%.
Meaning
These findings indicate that previous studies demonstrating the value of frailty screening can be replicated, demonstrating the validity of routine frailty screening as a method for improving outcomes in surgical patients.
This quality improvement study assesses the association between a standardized Frailty Screening Initiative with changes in 1-year mortality in patients undergoing elective surgery in the US.
Abstract
Importance
Patient frailty is a known risk factor for adverse outcomes following surgery, but data are limited regarding whether systemwide interventions related to frailty are associated with improved patient outcomes.
Objective
To evaluate whether a frailty screening initiative (FSI) is associated with reduced late-term mortality after elective surgery.
Design, Setting, and Participants
This quality improvement study with an interrupted time series analysis used data from a longitudinal cohort of patients in a multihospital, integrated health care system in the US. Beginning in July 2016, surgeons were incentivized to measure frailty with the Risk Analysis Index (RAI) for all patients considering elective surgery. Implementation of the BPA occurred in February 2018. The cutoff for data collection was May 31, 2019. Analyses were conducted between January and September 2022.
Exposures
The exposure of interest was an Epic Best Practice Alert (BPA) used to identify patients with frailty (RAI ≥42) and prompt surgeons to document a frailty-informed shared decision-making process and consider additional evaluation by a multidisciplinary presurgical care clinic or the primary care physician.
Main Outcomes and Measures
The primary outcome was 365-day mortality after the elective surgical procedure. Secondary outcomes included 30-day and 180-day mortality as well as the proportion of patients referred for additional evaluation based on documented frailty.
Results
A total of 50 463 patients with at least 1 year of postsurgical follow-up (22 722 before intervention implementation and 27 741 after) were included (mean [SD] age, 56.7 [16.0] y; 57.6% women). Demographic characteristics, RAI score, and operative case mix, as defined by Operative Stress Score, were similar between time periods. After BPA implementation, the proportion of frail patients referred to a primary care physician and presurgical care clinic increased significantly (9.8% vs 24.6% and 1.3% vs 11.4%, respectively; both P < .001). Multivariable regression analysis demonstrated an 18% reduction in the odds of 1-year mortality (0.82; 95% CI, 0.72-0.92; P < .001). Interrupted time series models demonstrated a significant slope change in the rate of 365-day mortality from 0.12% in the preintervention period to −0.04% in the postintervention period. Among patients triggering the BPA, estimated 1-year mortality changed by −4.2% (95% CI, −6.0% to −2.4%).
Conclusions and Relevance
This quality improvement study found that implementation of an RAI-based FSI was associated with increased referrals of frail patients for enhanced presurgical evaluation. These referrals translated to a survival advantage among frail patients of similar magnitude to those observed in a Veterans Affairs health care setting, providing further evidence for both the effectiveness and generalizability of FSIs incorporating the RAI.
Introduction
The growing population of aging patients within the US presents unique challenges to health care systems, and efforts to design care pathways that specifically address the needs of these patients are of critical importance.1 This is especially true within the field of surgery, where almost two-thirds of all patients are older than 65 years.2,3 Although investigators have long studied the association between aging and adverse surgical outcomes, our understanding of this link has evolved beyond simple chronologic age to the concept of frailty. Frailty defines a syndrome that embraces a more complete picture of a person’s health by considering measures of physical function and cognitive performance in addition to age and organ-specific comorbidity,4,5 and it provides a better estimate of perioperative morbidity and mortality than age alone.6,7 The recognition that frailty is a powerful variable in surgical outcomes has therefore led to the rapid proliferation in both local and national efforts aimed at improving surgical care in these high-risk patients.8,9,10,11
In an effort to facilitate standardized frailty screening within the context of routine clinical encounters, our previous work reported development and validation of the Risk Analysis Index (RAI).12,13 Additional work demonstrated its feasibility for use within routine clinical encounters at both a Veterans Affairs hospital and within a private-sector, multihospital health care system.11,14 In both contexts, the RAI not only provided a rapid, flexible method for preoperative frailty assessment but also proved reliable for estimating postoperative outcomes following elective surgery. Despite this documented success with respect to implementation of routine frailty screening, there remains a critical knowledge gap regarding whether efforts to screen and mitigate frailty-associated risk translate into improved outcomes at the level of the screened population.
We sought to address this gap by evaluating the association of a frailty screening initiative (FSI) program within the electronic health record with the rate of 1-year mortality following elective surgery in a longitudinal cohort. Importantly, the fact that our design leveraged preoperatively assessed frailty measurements later linked to postoperative outcomes provided an opportunity to use a robust, longitudinal, interrupted time series design to isolate the effect of the frailty intervention from existing secular trends in 1-year mortality.
Methods
Study Context and Data Source
The University of Pittsburgh Medical Center (UPMC) is a 42-hospital Integrated Delivery and Financing System serving both private practice physicians and multispecialty, academic physician groups, primarily focused at 5 hospitals within the greater Pittsburgh, Pennsylvania region. The Wolff Center for Quality and Safety at UPMC derived an analytic data set for this quality improvement study by linking existing clinical and administrative data from the electronic health record as part of an ongoing quality improvement project.14 The study was reviewed and approved as a quality improvement project by the UPMC Quality Review Committee (Protocol #986). Informed consent was not obtained because the frailty screening was done as part of routine clinical care. Analysis of the data was done with approval from the institutional quality improvement committee. Elements from the Standards for Quality Improvement Reporting Excellence 2.0 (SQUIRE) were used in development of the intervention and this report.15
Study Population
We identified a longitudinal cohort of all patients with an RAI value recorded in an outpatient surgery clinic who subsequently underwent a major surgical procedure within 270 days of RAI assessment. This time frame was chosen to balance the possibility that patient frailty at the time of operation changed significantly from when it was measured at their clinic visit, but also to allow an adequate sample size for analysis. To address the potential that this time frame may influence results, sensitivity analyses were performed for the primary analyses using RAI measurements within 90 and 180 days. Major surgery was defined as a Clinical Procedural Terminology (CPT) code eligible for abstraction and inclusion within the American College of Surgeons’ National Surgical Quality Improvement Program (NSQIP).16 The NSQIP eligibility criterion was used because all previous validation studies within our institution were completed within this same cohort. The cutoff of May 31, 2019, was used to allow for a full year of potential follow-up for all patients at the time of analysis.
Exposure
The implementation of a systemwide FSI using the RAI served as the primary exposure in this study. Detailed descriptions of the development and validation of the RAI12,13 as well as implementation at UPMC have been published previously.14 Briefly, the RAI is a weighted score derived from answers to 14 questions encompassing all 4 elements within a consensus definition of frailty developed within the National Institutes of Aging.17 The overall initiative was implemented in 2 discrete parts. The first phase of implementation began in July 2016 when all clinics within 9 surgical specialties (thoracic surgery, orthopedic surgery, plastic surgery, general surgery, cardiac surgery, neurosurgery, otolaryngology, urology, and obstetrics/gynecology) were asked to complete RAI assessments on all new and preoperative patients, recording the values in the electronic health record (Epic Systems). After an initial phase-in period, surgeons were incentivized to complete an RAI score for 80% of targeted patient encounters by making a portion of their salary contingent on meeting this benchmark. During this initial period, clinicians were asked to acknowledge the RAI score through an Epic Best Practice Advisory (BPA) but were not prompted to take any action. Beginning in July 2017, additional BPAs alerted clinicians of any frail patients with an RAI of 42 or greater, which served as the exposure of interest in this study.
The frailty-specific BPA first asked clinicians to indicate whether surgery was contemplated as potentially beneficial; if so, they were further prompted to take action regarding the patient’s potential frailty by selecting 1 or more of the following options: (1) attest that they had documented a frailty-informed shared decision-making process in their clinic note or referred the patient for further frailty assessment and mitigation by either (2) the patient’s primary care physician (PCP) or (3) an interdisciplinary preoperative evaluation clinic called the Center for Perioperative Care (CPC). These frailty-specific BPAs were initially optional, but after the observation that a subset of clinicians was not taking action based on frailty, the BPAs became mandatory in February 2018. This was enforced by enabling software rules that prevented clinicians from closing encounters until the BPA was addressed.
Outcomes and Covariables
One-year mortality from the date of surgery was the primary outcome for this study, determined through linkage to UPMC’s proprietary vital status file that draws from a variety of sources including the Social Security Death Index. Thirty- and 180-day mortality were included as secondary outcomes. We chose 1-year mortality as the primary outcome because previous studies demonstrate a strong link between RAI and late-term mortality, with more modest association with 30-day mortality.13 In addition, 1-year mortality represents an outcome particularly relevant in the context of frail patients because the trajectory of declining health in frail patients may span months,18 and the consequences of an acute insult such as surgery are therefore best measured over longer intervals.
To address confounding differences in patient populations between time periods, we also collected baseline information on patient age, sex, race, ethnicity, body mass index (BMI), and RAI score. Race and ethnicity represent patient self-report as extracted from the electronic health record; they were included in the analysis given prior literature demonstrating that race and ethnicity can be associated with surgical outcomes, and these are therefore included as potential covariates. Operative procedure relative value units (RVUs) and Operative Stress Score (OSS)19 were collected to control for changes in case mix across the intervention periods.20
Statistical Analysis
Analyses were conducted between January and September 2022. After inspecting the completeness and distribution of data, we summarized descriptive statistics of the frequency and proportion of demographic characteristics and specified outcomes, stratified by time before and after the initiation of the BPAs. Calculations exclude missing values. Univariate comparisons were made with likelihood-ratio χ2 tests for categorical variables and Wilcoxon rank sum tests for continuous variables. Multivariable analyses used logistic regression as well as forward/backward stepwise regression to determine significant variables, estimating the absolute and relative changes in mortality with 95% CIs. Final multivariable models for mortality included time period (before vs after intervention), age, log-transformed BMI, procedure RVU, sex, race, ethnicity, OSS, and frailty (RAI <42 vs ≥42). Interaction terms were considered but were not significant and were therefore not included in the final models. A multivariable time-series analysis was run using generalized linear models based on the Gaussian distribution with an identity link function. A wash-in period was not specified because the onset of exposure was immediate based on when it was enabled within the electronic health record. In addition, the practice of frailty measurement was well established at the time of implementation. Autocorrelation and partial autocorrelation plots examined lag terms, but none were found to be significant. The size of the unmeasured confounder required to obviate the observed association (eg, E-value) was computed using an online calculator.21,22 A 2-sided P < .05 was used to indicate statistical significance. All other analyses were conducted using Stata, version 17 (StataCorp LLC).
Results
Cohort Characteristics and Outcomes
From July 1, 2016, to May 31, 2019, we identified 50 463 patients with major surgery within 270 days of preoperative RAI assessment. Characteristics of the cohort are presented in Table 1, stratified by time before and after BPA initiation. For the overall cohort, the mean (SD) age was 56.7 (16.0) years; 57.6% were female and 42.4% were male; 9.0% were Black, 87.7% were White, and 3.3% were of other race (including 16 distinct American Indian, Asian, and Pacific Islander racial categories as well as those who declined to specify race); and 0.7% were Hispanic or Latino, 92.5% were not Hispanic or Latino, and 6.8% did not have a specified ethnicity. Characteristics of cohorts for each period were similar with regard to age, sex, and BMI. Importantly, RAI was also similar, and the proportion of frail patients in each cohort was not significantly different (4.7% before intervention and 4.9% after intervention; P = .73), suggesting that the prevalence of surgical patients with frailty did not change. Case mix based on OSS was similar, with OSS 2 and 3 procedures being the most common in both periods. The univariate comparison for OSS did indicate a statistically significant difference in expected frequency; this is likely related to overall sample size and not a practical difference. However, given this finding, OSS was included in subsequent multivariable analyses. After BPA implementation, the proportion of frail patients referred to a PCP or the CPC increased significantly (9.8% vs 24.6% and 1.3% vs 11.4%, respectively; both P < .001). Unadjusted mortality rates were lower after BPA initiation for 30-day (0.9% vs 0.7% [P = .08]), 180-day (2.5% vs 2.1% [P = .003]), and 365-day mortality (3.9% vs 3.3% [P = .001]).
Table 1. Baseline Characteristics, Preoperative Management, and Postoperative Mortality Stratified by Time Before and After BPA Initiation.
| No. (%) | P valuea | |||
|---|---|---|---|---|
| Overall (n = 50 463) | Before BPA (n = 22 722) | After BPA (n = 27 741) | ||
| Baseline demographic characteristics | ||||
| Age, mean (SD), y | 56.7 (16.0) | 56.5 (16.1) | 56.9 (15.9) | .004 |
| Sex | ||||
| Male | 21 373 (42.4) | 9684 (42.6) | 11 689 (42.1) | .27 |
| Female | 29 090 (57.6) | 13 038 (57.4) | 16 052 (57.9) | |
| Race | ||||
| Black | 4545 (9.0) | 2029 (8.9) | 2516 (9.1) | <.001 |
| White | 44 238 (87.7) | 19 827 (87.3) | 24 411 (88.0) | |
| Otherb | 1680 (3.3) | 866 (3.8) | 814 (2.9) | |
| Ethnicity | ||||
| Not specified | 3428 (6.8) | 1642 (7.2) | 1786 (6.4) | .002 |
| Hispanic or Latino | 329 (0.7) | 152 (0.7) | 177 (0.6) | |
| Not Hispanic or Latino | 46 706 (92.5) | 20 928 (92.1) | 25 778 (92.9) | |
| BMI, mean (SD) | 29.3 (6.0) | 29.2 (5.9) | 29.3 (5.9) | .14 |
| RAI score, median (IQR)c | 21 (14-30) | 20 (13-29) | 21 (15-31) | <.001 |
| Frail patients (RAI ≥42) | 2444 (4.8) | 1077 (4.7) | 1367 (4.9) | .73 |
| Operative management and outcomes | ||||
| OSSd | ||||
| 1 | 3013 (8.1) | 1247 (7.5) | 1766 (8.6) | <.001 |
| 2 | 20 257 (54.5) | 8947 (54.1) | 11 310 (54.8) | |
| 3 | 12 639 (34.0) | 5738 (34.7) | 6901 (33.4) | |
| 4 | 1241 (3.3) | 590 (3.6) | 651 (3.2) | |
| 5 | 42 (0.1) | 21 (0.1) | 21 (0.1) | |
| RVU, median (IQR) | 11.4 (6.5-17.6) | 11.5 (6.6-17.5) | 11.2 (6.4-17.6) | .02 |
| Mortality | ||||
| 30-d | 390 (0.8) | 193 (0.9) | 197 (0.7) | .08 |
| 180-d | 1173 (2.3) | 578 (2.5) | 595 (2.1) | .003 |
| 365-d | 1797 (3.6) | 878 (3.9) | 919 (3.3) | .001 |
Abbreviations: BMI, body mass index (calculated as weight in kilograms divided by height in meters squared); BPA, Best Practice Advisory; OSS, Operative Stress Score; RAI, Risk Analysis Index of Frailty; RVU, relative value unit.
P values are from the Kruskal-Wallis rank test or Wilcoxon rank sum test for continuous variables and from the likelihood ratio χ2 test for categorical variables.
Other race includes Alaska Native, American Indian, Chinese, Filipino, Guam/Chamorro, Hawaiian, Indian (Asian), Japanese, Korean, other Asian, Other Pacific Islander, Samoan, Vietnamese, other, declined, and not specified. None of these was large enough to justify independent analysis.
Scores range from 0-81, with higher scores indicating increased frailty.
Scores range from 1 (very low stress) to 5 (very high stress).
Compliance With the Frailty Initiative
There were 1367 frail patients with an RAI of 42 or greater for whom the frailty-specific BPA fired, asking the surgeon to indicate whether surgery was contemplated as potentially beneficial (Table 1). Surgeons acknowledged this BPA in every case (eg, 100% compliance), indicating a potential benefit of surgery in 768 cases for which the surgeon was further prompted to select 1 or more of the interventions (eg, frailty-informed shared decision-making or referral to either the PCP or CPC). Of these, 590 (74.2%) received at least 1 intervention.
Association of Frailty Initiative Implementation With Overall Mortality
To determine the association of BPA implementation with 30-, 180- and 365-day mortality, we constructed multivariable logistic regression models controlling for both demographic and operative factors (Table 2). As expected, patient- and procedure-specific factors such as age, race, increasing BMI, RAI scores, RVUs, and OSS were independently associated with increased odds of mortality. However, after controlling for these confounding factors, we identified 21% to 24% reductions in the odds of mortality for patients who underwent surgery in the post-BPA period, but the reduction was statistically significant only at 180 days (OR, 0.78; 95% CI, 0.67-0.91) and 365 days (OR, 0.82; 95% CI, 0.72-0.92; P < .001 for both).
Table 2. Association Between BPA Implementation and Mortality, Controlling for Baseline and Intraoperative Covariates in Multivariable Logistic Regression.
| Covariate | Mortality | |||||
|---|---|---|---|---|---|---|
| 30-d | 180-d | 365-d | ||||
| OR (95% CI) | P value | OR (95% CI) | P value | OR (95% CI) | P value | |
| Intervention | ||||||
| Before BPA | 1 [Reference] | NA | 1 [Reference] | NA | 1 [Reference] | NA |
| After BPA | 0.78 (0.60-1.01) | .07 | 0.78 (0.67-0.91) | <.001 | 0.82 (0.72-0.92) | .001 |
| Age | 1.05 (1.04-1.06) | <.001 | 1.04 (1.04-1.05) | <.001 | 1.05 (1.04-1.05) | <.001 |
| Sex | ||||||
| Female | 1 [Reference] | NA | 1 [Reference] | NA | 1 [Reference] | NA |
| Male | 1.52 (1.16-1.99) | .003 | 1.43 (1.22-1.66) | <.001 | 1.45 (1.28-1.65) | .003 |
| Race | ||||||
| White | 1 [Reference] | NA | 1 [Reference] | NA | 1 [Reference] | NA |
| Black | 1.80 (1.18-2.75) | .006 | 1.36 (1.03-1.79) | .03 | 1.37 (1.09-1.71) | .006 |
| Othera | 1.41 (0.69-2.89) | .34 | 1.40 (0.95-2.07) | .09 | 1.17 (0.82-1.66) | .38 |
| Ethnicity | ||||||
| Not Hispanic or Latino | 1 [Reference] | NA | 1 [Reference] | NA | 1 [Reference] | NA |
| Not specified | 1.12 (0.68-1.86) | .65 | 1.23 (0.93-1.63) | .15 | 1.33 (1.06-1.68) | .01 |
| Hispanic or Latino | Omittedb | NA | 0.62 (0.16-2.42) | .49 | 0.61 (0.20-1.89) | .40 |
| BMI (log) | 0.61 (0.28-1.32) | .21 | 0.30 (0.20-0.46) | <.001 | 0.41 (0.29-0.58) | <.001 |
| Frail (RAI ≥42) | 3.44 (2.46-4.80) | <.001 | 4.41 (3.63-5.36) | <.001 | 4.05 (3.44-4.77) | <.001 |
| OSS | ||||||
| 1 | 1 [Reference] | NA | 1 [Reference] | NA | 1 [Reference] | NA |
| 2 | 0.93 (0.50-1.72) | .82 | 0.96 (0.70-1.31) | .78 | 0.96 (0.75-1.24) | .76 |
| 3 | 3.35 (1.79-6.28) | <.001 | 2.23 (1.59-3.13) | <.001 | 2.06 (1.57-2.72) | <.001 |
| 4 | 6.74 (2.96-15.34) | <.001 | 4.12 (2.55-6.72) | <.001 | 3.84 (2.58-5.72) | <.001 |
| 5 | 8.10 (0.72-90.89) | .09 | 8.98 (2.69-29.91) | <.001 | 6.46 (2.17-19.20) | .001 |
| RVU | 0.98 (0.95-1.00) | .02 | 0.97 (0.96-0.99) | <.001 | 0.98 (0.97-0.99) | <.001 |
Abbreviations: BMI, body mass index; BPA, Best Practice Advisory; NA, not applicable; OR, odds ratio; OSS, Operative Stress Score; RAI, Risk Analysis Index of Frailty; RVU, relative value unit.
Other race includes Alaska Native, American Indian, Chinese, Filipino, Guam/Chamorro, Hawaiian, Indian (Asian), Japanese, Korean, other Asian, Other Pacific Islander, Samoan, Vietnamese, other, declined, and not specified. None of these was large enough to justify independent analysis.
Hispanic or Latino ethnicity was omitted because of lack of model convergence.
We further investigated the robustness of these results by using interrupted time series analysis to control for secular trends in postoperative mortality using generalized linear model regression. In these models, the dependent variable was mortality rate, and explanatory variables were time (month), an indicator of whether the surgery was before or after BPA initiation, and an interaction term representing the number of months after BPA initiation. Results of these models allow for determination of whether there was an immediate level change in mortality rate before and after intervention as well as whether there was a change in mortality rate trends. Results from these analyses are presented in Table 3 and the Figure.
Table 3. Changes in Mortality Before and After Best Practice Advisory (BPA) Initiation From Multivariable Gaussian Regressiona.
| Mortality | Level change (95% CI) | P value | Trend change (95% CI) | P value |
|---|---|---|---|---|
| 30-d | −0.22 (−0.48 to 0.05) | .12 | −0.02 (−0.05 to 0.01) | .06 |
| 180-d | −0.77 (−1.41 to −0.13) | .02 | −0.08 (−0.15 to 0.00) | .06 |
| 365-d | −0.36 (−0.95 to 0.25) | .23 | −0.16 (−0.22 to −0.09) | <.001 |
Confidence intervals estimated from a multivariable Poisson regression model of mortality. Models for 180- and 365-day mortality included a lag term. The coefficient for level change assesses whether mortality decreased at the time of BPA initiation. The coefficient for trend change assesses the slope of the line after BPA initiation.
Figure. Interrupted Time Series Analysis.
Plots demonstrate 30-, 180-, and 365-day mortality rates before and after Best Practice Alert (BPA) implementation. Rates are expressed as deaths per 100 procedures. Implementation is noted by the vertical line (February 2018). The solid lines are actual mortality rates, and the dotted orange trend lines represent estimates based on generalized linear model regression. The dotted blue trend lines represent a counterfactual situation where the preintervention trend would continue into the postintervention period. Comparison between the postintervention trend line and the counterfactual trend line represents the association of BPA implementation with mortality.
Prior to BPA implementation, 1-year mortality was increasing at a rate of 0.12% per month. After BPA implementation, this rate began decreasing by 0.04% per month, representing a significant change in mortality trend (trend change, −0.16% per month; 95% CI, −0.22% to −0.09% per month). This finding indicates that BPA implementation was associated with 1-year mortality reduction independent of secular trends. Nonsignificant reductions in 30-day and 180-day mortality were also noted. Sensitivity analyses were performed using RAI values within 90 and 180 days of surgery as opposed to 270 days to address whether the time range influenced results. We identified similar results in both of these analyses, with significant associations with 365-day mortality reduction when using both 90- and 180-day thresholds for RAI measurement (−0.07% per month and −0.06% per month, respectively) (eTable and eFigure in the Supplement).
Influence of Frailty-Specific BPAs on Frail Patients
Documentation of shared decision-making and referral to the CPC and/or PCP increased significantly in frail patients following implementation of the BPA. Documentation of shared decision-making increased from 8.8% to 34.9% (P < .001) in frail patients after BPA administration, and referral to the CPC increased from 1.2% to 6.3% (P < .001).
To isolate the influence of the frailty-specific BPAs triggered for frail patients with an RAI of 42 or greater, we performed a subgroup analysis on frail patients undergoing surgery in the postintervention period. In this analysis, we constructed multivariable models identical to our initial analysis with the inclusion of a binary variable representing whether surgeons chose to act on the reported frailty score by selecting at least 1 of the proposed interventions (eg, shared decision-making or referral to either a PCP or CPC). This analysis demonstrated a significant reduction in the estimated absolute rate of mortality among very frail patients with marginal effect sizes ranging from −2.7% (95% CI, −3.6% to −1.9%) to −4.2% (95% CI, −6.0% to −2.4%) (Table 4). The improvements correspond to a substantial survival advantage after adjusting for confounders (eg, adjusted odds ratio [aOR], 2.24 [95% CI, 1.24-4.04] for 30-day survival; E-value, 3.91 [lower confidence interval, 1.79]). Improving surgeon awareness is a valuable result of FSIs; these data also suggest that providing multidisciplinary resources for frail patients in the perioperative period is critical to the success of FSIs.
Table 4. Absolute and Relative Mortality of Frail Patients (Risk Analysis Index of Frailty ≥42) With Whom Clinician Did or Did Not Intervene, Estimated From Multivariable Logistic Regression.
| Mortality | |||
|---|---|---|---|
| 30-d | 180-d | 365-d | |
| Absolute mortality rate, % | |||
| Provider did not intervene | 5.1 | 14.4 | 20.2 |
| Provider intervened | 2.3 | 10.9 | 16.0 |
| Marginal effect size (95% CI) | −2.7 (−3.6 to −1.9) | −3.5 (−5.3 to −1.9) | −4.2 (−6.0 to −2.4) |
| Relative mortality, OR (95% CI)a | |||
| Mortality reduction | 0.45 (0.25 to 0.81) | 0.73 (0.53 to 0.98) | 0.75 (0.58 to 0.97) |
| Survival advantage | 2.24 (1.24 to 4.04) | 1.38 (1.02 to 1.87) | 1.33 (1.03 to 1.73) |
Abbreviation: OR, odds ratio.
Mortality reduction and survival advantage are inverse values representing the same concept and estimate the association of intervention (referral for additional evaluation and/or shared decision making) with the odds of mortality following elective surgical procedure.
Discussion
Interest in the importance of frailty as a estimate of poor health outcomes for aging patients has rapidly grown over the past 2 decades. Within surgical populations specifically, the association of frailty with adverse outcomes across a broad range of surgical specialties is well documented. Spurred by this understanding, surgeons and hospitals seek to identify mechanisms that not only identify frail patients at risk for poor surgical outcomes but also to develop interventions that may mitigate this risk. The goal of this study was to describe the association of an FSI with late-term postoperative mortality. After controlling for patient characteristics, we found that there was a significant reduction in the odds of 180- and 365-day mortality following implementation of our BPA-based FSI. We further confirmed that these findings were independent of secular trends by using interrupted time series analysis. Finally, we were able to identify changes in surgeon behavior after implementation, with a significant increase in the number of frail patients receiving either additional documentation regarding shared decision-making or referral for additional preoperative evaluation.
The design and implementation of this intervention provided a unique opportunity to leverage longitudinal data with robust confounding control23 and use an interrupted time series design, adding to the robustness of the analysis. After analyzing outcomes for more than 50 000 patients with preoperatively assessed RAI values, we found that implementation of a frailty-specific BPA was associated with clinically and statistically significant reductions in mortality for all patients, but especially for the frail patients for whom the surgeons chose to act by selecting 1 of 3 light-touch interventions. Importantly, we identified not only an immediate reduction in mortality but also a persistent trend in mortality reduction in the months following implementation. These observations demonstrate that the interventions were both effective and sustainable, 2 key characteristics in determining the utility of quality improvement efforts within hospital systems.
The importance of these findings is enhanced by the fact that they demonstrate reproducibility of prior work, which is a critical consideration when evaluating the validity findings related to specific interventions.24,25 Our initial experience with an RAI-based FSI in the Veterans Affairs health system demonstrated a large survival advantage among frail patients after adjusting for patient-level covariates (aOR, 2.87; 95% CI, 1.98-4.16 for 180-day mortality). The findings from this private-sector, multihospital health care system (aOR, 1.38; 95% CI, 1.02-1.87) remain substantial after even more robust control for both patient and procedure-level confounding. Although numerous studies have linked prospectively measured frailty with perioperative outcomes, it is unknown whether these efforts ultimately influenced patient outcomes.26 Similarly, although use of dedicated perioperative pathways for aging and frail patients have become more commonplace, less is understood regarding their association with patient outcomes. For example, a Perioperative Optimization of Senior Health effort identified reduced length of stay, perioperative complication, and readmissions in a cohort of 183 patients.27 Despite promising results, that study represented a highly selected patient population at a smaller scale. Our results fill an important gap in the literature by both demonstrating the value of routine frailty screening in reducing postoperative mortality and providing evidence for the feasibility of these efforts in high-volume clinical settings. While ongoing randomized clinical trials may provide additional evidence to support widespread adoption of more intensive frailty-focused interventions, our current observations suggest that even a straightforward frailty assessment and clinical prompt has the potential to provide substantial benefit in routine clinical practice.
In addition to identifying a reduction in mortality generally, we were also able to identify a benefit in frail patients specifically. In a subgroup analysis of frail surgical patients, we found that acting on a patient’s frailty status—by referral to either a PCP or our perioperative clinic—was associated with a 3.5% reduction in 180-day mortality and 4.2% reduction in 365-day mortality when compared with frail patients for whom no additional action was taken. This observation is critical for 2 reasons. First, it suggests that frail patients have modifiable risk factors that can be addressed prior to surgery. If this were not the case, it may call into question the utility of designing clinical pathways specifically aimed at frail patients. Second, it provides insight into the mechanism of action for the observed improvements. In our original FSI study, it remained unclear whether observed mortality reduction may be an example of the Hawthorne effect, in which improvement was a result of increased awareness of patient frailty and not due to a specific intervention. In other words, it is possible that surgeons became aware of an institutional effort to improve outcomes for frail patients and therefore modified their behavior in ways outside the FSI—which were unmeasured in the analysis—ultimately leading to mortality reduction. However, given the association between mortality reduction in very frail patients and additional evaluation prompted by the BPA in the current study, it is reasonable to conclude that improvements associated with FSI are not driven solely by those unmeasured changes in behavior. Although surgeons’ awareness of institutional efforts to improve outcomes for frail patients may influence their behavior leading to overall improvement, such as more careful selection of frail patients to undergo surgical procedures, formal pathways targeted at frail patients clearly provided added benefit.
Limitations
Despite compelling results, there are limitations to this study that must be acknowledged. Our design leverages longitudinal data collected before surgery and uses robust controls for confounding. We also quantified the size of the unmeasured confounder required to obviate our findings. However, in the absence of a randomized clinical trial, we are unable to draw conclusions regarding the causal effect of this intervention and concede that we may have failed to control for unobserved confounders. Interrupted time series analysis allows us to control for secular trends in the data and potentially draw conclusions based on an intervention implemented at a precise time point. It does not control for the possibility that a separate change occurred at the same time and accounts for the mortality reduction. To our knowledge, however, there were no system-level interventions enacted within this period that may confound our results. One method of addressing this issue would be to apply a difference-in-differences approach. While that would be ideal, it would require existence of a comparable health care system where frailty was measured but no intervention was implemented. Unfortunately, this comparator does not exist. We are also unable to determine the precise components of the additional evaluation provided by PCPs or the perioperative clinic that account for the observed benefits in frail patients. However, this study provides important background on the importance of completing trials that identify and standardize the most cost-effective package of interventions for frail surgical patients. Finally, our experimental design allowed analysis only of patients who eventually had surgery, and we are therefore unable to quantify whether our intervention directly influenced decisions to choose nonoperative approaches. Although changes in patient selection for surgery may provide an additional pathway for the mortality reduction observed in this study, this would not represent an additional—as opposed to alternative—explanation. It is more likely that the additional evaluation triggered by our intervention led to shared patient-provider decisions that led to a choice for nonoperative management. However, given that the proportion of frail patients undergoing surgery before and after BPA implementation were similar, this additional evaluation is also not the sole explanation for mortality reduction. It is also important to consider that these instances of a change in management would be considered a desirable outcome related to FSIs. Although our current study does not allow us to quantify this association, it is the focus of future work.
Conclusions
This quality improvement study builds on previous work reporting the feasibility of routine frailty screening by demonstrating tangible, system-level benefits associated with implementation of an FSI. It also emphasizes the idea that in an aging surgical population, frailty is a critical target for future surgical quality improvement programs. Future efforts should focus on identifying methods to implement frailty screening in a broad range of practice settings as well as identify a suite of interventions best suited to improving outcomes for frail patients prior to surgery.
eTable. Sensitivity Analysis for Changes in 365-Day Mortality Before and After BPA Initiation From Multivariable Gaussian Regression
eFigure. Sensitivity Analysis for Timing of RAI Assessment
References
- 1.Rowe JW, Berkman L, Fried L, et al. Preparing for better health and health care for an aging population: a vital direction for health and health care. NAM Perspect. Published online September 19, 2016. doi: 10.31478/201609n [DOI] [Google Scholar]
- 2.Etzioni DA, Liu JH, Maggard MA, Ko CY. The aging population and its impact on the surgery workforce. Ann Surg. 2003;238(2):170-177. doi: 10.1097/01.SLA.0000081085.98792.3d [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Kwok AC, Semel ME, Lipsitz SR, et al. The intensity and variation of surgical care at the end of life: a retrospective cohort study. Lancet. 2011;378(9800):1408-1413. doi: 10.1016/S0140-6736(11)61268-3 [DOI] [PubMed] [Google Scholar]
- 4.Rodríguez-Mañas L, Féart C, Mann G, et al. ; FOD-CC group (Appendix 1) . Searching for an operational definition of frailty: a Delphi method based consensus statement: the frailty operative definition-consensus conference project. J Gerontol A Biol Sci Med Sci. 2013;68(1):62-67. doi: 10.1093/gerona/gls119 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Sternberg SA, Wershof Schwartz A, Karunananthan S, Bergman H, Clarfield AM. The identification of frailty: a systematic literature review. J Am Geriatr Soc. 2011;59(11):2129-2138. doi: 10.1111/j.1532-5415.2011.03597.x [DOI] [PubMed] [Google Scholar]
- 6.Makary MA, Segev DL, Pronovost PJ, et al. Frailty as a predictor of surgical outcomes in older patients. J Am Coll Surg. 2010;210(6):901-908. doi: 10.1016/j.jamcollsurg.2010.01.028 [DOI] [PubMed] [Google Scholar]
- 7.Robinson TN, Eiseman B, Wallace JI, et al. Redefining geriatric preoperative assessment using frailty, disability and co-morbidity. Ann Surg. 2009;250(3):449-455. doi: 10.1097/SLA.0b013e3181b45598 [DOI] [PubMed] [Google Scholar]
- 8.Diffusion Marketplace . The surgical pause. Updated April 2022. Accessed May 10, 2022. https://marketplace.va.gov/innovations/preoperative-frailty-screening-prehabilitation
- 9.Patient-centered multidisciplinary care for veterans undergoing surgery (PAUSE). ClinicalTrials.gov identifier: NCT05037292. Updated October 20, 2022. Accessed May 10, 2022. https://clinicaltrials.gov/ct2/show/NCT05037292
- 10.Robinson TN, Rosenthal RA. The ACS NSQIP Geriatric Surgery Pilot Project: improving care for older surgical patients. Bull Am Coll Surg. 2014;99(10):21-23. [PubMed] [Google Scholar]
- 11.Hall DE, Arya S, Schmid KK, et al. Association of a frailty screening initiative with postoperative survival at 30, 180, and 365 days. JAMA Surg. 2017;152(3):233-240. doi: 10.1001/jamasurg.2016.4219 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Hall DE, Arya S, Schmid KK, et al. Development and initial validation of the Risk Analysis Index for measuring frailty in surgical populations. JAMA Surg. 2017;152(2):175-182. doi: 10.1001/jamasurg.2016.4202 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Arya S, Varley P, Youk A, et al. Recalibration and external validation of the Risk Analysis Index: a surgical frailty assessment tool. Ann Surg. 2020;272(6):996-1005. doi: 10.1097/SLA.0000000000003276 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Varley PR, Borrebach JD, Arya S, et al. Clinical utility of the Risk Analysis Index as a prospective frailty screening tool within a multi-practice, multi-hospital integrated healthcare system. Ann Surg. 2021;274(6):e1230-e1237. doi: 10.1097/SLA.0000000000003808 [DOI] [PubMed] [Google Scholar]
- 15.Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. SQUIRE 2.0 (Standards for Quality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process. BMJ Qual Saf. 2016;25(12):986-992. doi: 10.1136/bmjqs-2015-004411 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Bilimoria KY, Liu Y, Paruch JL, et al. Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons. J Am Coll Surg. 2013;217:833-842.e423. doi: 10.1016/j.jamcollsurg.2013.07.385 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Walston J, Bandeen-Roche K, Buta B, et al. Moving frailty toward clinical practice: NIA Intramural Frailty Science Symposium summary. J Am Geriatr Soc. 2019;67(8):1559-1564. doi: 10.1111/jgs.15928 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Stow D, Matthews FE, Hanratty B. Frailty trajectories to identify end of life: a longitudinal population-based study. BMC Med. 2018;16(1):171. doi: 10.1186/s12916-018-1148-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Shinall MC Jr, Arya S, Youk A, et al. Association of preoperative patient frailty and operative stress with postoperative mortality. JAMA Surg. 2020;155(1):e194620. doi: 10.1001/jamasurg.2019.4620 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.George EL, Hall DE, Youk A, et al. Association between patient frailty and postoperative mortality across multiple noncardiac surgical specialties. JAMA Surg. 2021;156(1):e205152. doi: 10.1001/jamasurg.2020.5152 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Mathur MB, Ding P, Riddell CA, VanderWeele TJ. Web site and R package for computing E-values. Epidemiology. 2018;29(5):e45-e47. doi: 10.1097/EDE.0000000000000864 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.VanderWeele TJ, Ding P. Sensitivity analysis in observational research: introducing the E-value. Ann Intern Med. 2017;167(4):268-274. doi: 10.7326/M16-2607 [DOI] [PubMed] [Google Scholar]
- 23.VanderWeele TJ, Jackson JW, Li S. Causal inference and longitudinal data: a case study of religion and mental health. Soc Psychiatry Psychiatr Epidemiol. 2016;51(11):1457-1466. doi: 10.1007/s00127-016-1281-9 [DOI] [PubMed] [Google Scholar]
- 24.Goodman SN, Fanelli D, Ioannidis JPA. What does research reproducibility mean? Sci Transl Med. 2016;8(341):341ps12. doi: 10.1126/scitranslmed.aaf5027 [DOI] [PubMed] [Google Scholar]
- 25.Zwaan RA, Etz A, Lucas RE, Donnellan MB. Making replication mainstream. Behav Brain Sci. 2017;41:e120. doi: 10.1017/S0140525X17001972 [DOI] [PubMed] [Google Scholar]
- 26.Aucoin SD, Hao M, Sohi R, et al. Accuracy and feasibility of clinically applied frailty instruments before surgery: a systematic review and meta-analysis. Anesthesiology. 2020;133(1):78-95. doi: 10.1097/ALN.0000000000003257 [DOI] [PubMed] [Google Scholar]
- 27.McDonald SR, Heflin MT, Whitson HE, et al. Association of integrated care coordination with postsurgical outcomes in high-risk older adults: the Perioperative Optimization of Senior Health (POSH) Initiative. JAMA Surg. 2018;153(5):454-462. doi: 10.1001/jamasurg.2017.5513 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eTable. Sensitivity Analysis for Changes in 365-Day Mortality Before and After BPA Initiation From Multivariable Gaussian Regression
eFigure. Sensitivity Analysis for Timing of RAI Assessment