Measurement and Validation of Frailty as a Predictor of Outcomes in Women Undergoing Major Gynecologic Surgery

Abstract

Objectives

Frailty is the loss of physical or mental reserve that impairs function, often in the absence of a defined comorbidity. Our aim was to determine if a modified frailty index correlates with morbidity and mortality in patients undergoing hysterectomy.

Design

Retrospective cohort study.

Setting

Hospitals across the U.S. participating in the National Surgical Quality Improvement Program (NSQIP).

Sample

Patients that underwent hysterectomy from 2008-2012.

Main Outcome Measure

Wound infection, severe complications, and mortality.

Study Design

A modified frailty index (mFI) was calculated using 11 variables in NSQIP. The associations between the mFI and morbidity and mortality were assessed. Model fit statistics (c-statistics) were utilized to evaluate the ability of the mFI to distinguish outcomes.

Results

A total of 66,105 patients were identified. Wound complications increased from 2.4% in patients with an mFI of 0 to 4.8% in those with an mFI ≥0.5 (P<0.0001). Similarly, severe complications increased from 0.98% to 7.3% (P<0.0001), overall complications rose from 3.7% to 14.5% (P<0.0001) and mortality increased from 0.06% to 3.2% (P<0.0001) for patients with a frailty index of 0 compared to those with an index ≥0.5. Versus chance, the goodness-of-fit c-statistics suggested that the mFI increases the ability to detect wound complications by 11.4%, severe complications by 22.0%, and overall complications by 11.0%.

Conclusions

The mFI is easily reproducible from routinely collected clinical data and predictive of outcomes in patients undergoing hysterectomy. Frailty may be useful in the preoperative risk assessment of women undergoing gynecologic surgery.

Introduction

The elderly population is rapidly expanding in the United States. It is estimated that by 2050, the US population aged 65 and older will reach 83.7 million, nearly double the 2012 estimate of 43.1 million.(1) In gynecologic surgery, an abundance of literature suggests that elderly women are treated differently than their younger counterparts and, in many scenarios, are less likely to undergo surgical interventions.(2-9)

Studies examining the tolerance of elderly women have reported varied findings. While many single institution reports have found that elderly women tolerate surgery well, population-based studies suggest that elderly women undergoing gynecologic surgery are at significantly greater risk for complications and death than their younger counterparts.(10-15) Divergent outcomes among elderly patients have led to a heightened awareness that measures of performance status and function other than chronologic age alone are important predictors of surgical outcomes.(16, 17)

Frailty is an emerging concept in the general medical and surgical literature.(18-21) A consensus conference in December of 2012 led by the International Association of Gerontology and Geriatrics and the World Health Organization defined frailty as “a medical syndrome with multiple causes and contributors that is characterized by diminished strength, endurance, and reduced physiologic function that increases an individual’s vulnerability for developing increased dependency and/or death.”(18, 19) The importance of frailty in predicting surgical outcomes and how best to measure frailty remain less certain.

Despite the potential association between frailty and outcomes for patients undergoing gynecologic surgery, relatively few studies have examined the concept of frailty.(22, 23) The objective of our study was to perform a population-based analysis to determine if a modified frailty index correlates with morbidity and mortality in patients undergoing major gynecologic surgery. Specifically, we examined whether routinely collected clinical data could be used as a surrogate for frailty and examined the discriminatory ability of an index of frailty to predict adverse outcomes in women undergoing hysterectomy.

Methods

The American College of Surgeons’ National Surgical Quality Improvement Program (NSQIP) database was analyzed.(24) NSQIP is a nationwide database that collects data on surgical patients from hospitals from across the United States. The database was initially developed for benchmarking and quality improvement, and now collects data on over 150 variables from approximately 400 hospitals. NSQIP captures data from the index procedural admission and follows patients for 30 days after surgery. Data is abstracted using a defined sampling schema that collects data from the first 40 cases for a given procedure during 8-day sampling cycles. The sampling is spaced throughout the year to reduce bias in case selection. Data undergoes regular auditing to ensure quality. The Columbia University Institutional Review Board deemed the study exempt.

Patients that underwent hysterectomy for any indication from 2008-2012 were analyzed. The cohort included abdominal, laparoscopic, and vaginal approaches as identified by Current Procedural Terminology (CPT) codes. Indications for hysterectomy included leiomyoma, endometriosis, abnormal menstrual bleeding, benign neoplasms and cysts, pelvic organ prolapse, uterine cancer, cervical cancer, and ovarian cancer.

Clinical and demographic data analyzed included type of hysterectomy (abdominal, laparoscopic, vaginal), performance of concomitant procedures, race (white, black, other, unknown), procedure setting (inpatient or outpatient), age (<40, 40-49, 50-59, 60-69, ≥70), body mass index (BMI, kg/m², normal <25, overweight 25-29.9, obese ≥30), length of stay, American Society of Anesthesiology (ASA) class, functional status (independent, partially dependent, totally dependent, unknown) and preoperative albumin (<3.5, 3.5-4.0, >4.0, unknown).(24) Medical comorbidities present prior to surgery analyzed included in the analysis were diabetes mellitus (insulin dependent and non-insulin dependent), tobacco use, chronic obstructive pulmonary disease, hypertension, the presence of metastatic cancer, corticosteroid use, weight loss, bleeding disorders, and preoperative transfusion. Postoperative complications recorded include reoperation, surgical site infections (superficial, deep, and organ space), wound dehiscence, pneumonia, pulmonary embolism, deep vein thrombosis, urinary tract infection, transfusion, sepsis, shock, myocardial infarction, acute renal failure, and readmission.

A modified frailty index (mFI) was calculated using 11 variables from the Canadian Study of Health and Aging (CSHA) Frailty Index that were matched to variables in NSQIP (Table 1). The CSHA frailty index has been previously developed to provide a measure of frailty using clinically relevant parameters.(25) The 11 items included diabetes mellitus, functional status, respiratory problems (chronic obstructive pulmonary disease or pneumonia), congestive heart failure, history of myocardial infarction, other cardiac problems (previous percutaneous coronary intervention or coronary surgery or angina), hypertension requiring medication, peripheral vascular disease or resting pain, impaired sensorium, history of transient ischemic attack or cerebral vascular accident, and cerebrovascular accident with neurologic defect.(21, 26, 27) If present, each variable was assigned one point. The total points for each patient were then calculated and divided by the total number of points available (covariates with known values). Each patient’s mFI ranged between 0.0 and 1.0, with increasing mFI implying increased frailty.

Table 1.

Factors included in modified frailty index.

	Domain	Coding
1	Diabetes mellitus	Insulin dependent diabetes mellitus or non-insulin dependent diabetes mellitus
2	Functional status	Partially dependent or total dependent
3	Respiratory problems	Chronic obstructive pulmonary disease or current pneumonia
4	Congestive heart failure	Congestive heart failure
5	Myocardial infarction	Prior myocardial infarction
6	Other cardiac problems	Previous percutaneous coronary intervention or coronary surgery or angina
7	Hypertension	Hypertension requiring medication
8	Peripheral vascular disease	Peripheral vascular disease or resting pain
9	Impaired sensorium	Impaired sensorium
10	Cerebrovascular disease	Transient ischemic attack or cerebrovascular accident
11	Cerebrovascular disease with neurologic deficit	Cerebrovascular disease with deficit

The outcomes of interest were morbidity and 30-day mortality. Thirty-day mortality was defined as death within 30 days of the index procedure. Several measures of morbidity were examined. Severe complications were based on the Clavian class IV categorization as previously described and included septic shock, cardiac arrest, myocardial infarction, pulmonary embolism, need for greater than 48 hours of ventilation, and unplanned re-intubation.(21, 26, 28) Wound complications included any occurrence of a superficial, deep, or organ space surgical site infection. Finally, any complication was based on the occurrence of a Clavian IV complication or wound complication as defined above or pneumonia, acute renal failure, urinary tract infection, cerebrovascular accident or stroke, coma, or deep vein thrombosis or thrombophlebitis.

The clinical and demographic data of the cohort are displayed descriptively. Distributions of the outcomes across mFI scores were compared using χ² tests. Analyses were undertaken using model fit statistics to determine the strength of association between age, functional status, ASA classification, and the modified frailty index. Models were first developed to determine the importance of each factor individually in predicting the outcomes of interest and then all of the measures were included in a single model to determine the ability of all four measures (age, functional status, ASA classification, and mFI index) combined to predict the study outcomes.

The c-statistic is a measure of the ability of a model to classify the outcome of interest and is calculated as the area under the curve of a receiver operating characteristic curve (ROC) evaluating true positive and false positive rates. A c-statistic of 1 indicates that a model perfectly predicts the outcome while a value of 0.5 indicates that the model is no better than chance. The pseudo-R² is a measure of the total variability in the response accounted for by the covariates in the model and is based on the concept of R² which can be estimated from ordinary least squares linear regression. A higher pseudo-R² indicates that the variable included explains more observed variation. The Akaike information criterion (AIC) is a measure of the goodness of fit of a model while accounting for the complexity of the model. A lower AIC is indicative of a greater importance of the variable. The likelihood ratio test (LRT) compares the fit of a null model to a model containing one or more covariate. A higher LRT compared to the null model suggest a greater improvement in fit when the variables are included.

To estimate the ability of a given measure to predict the outcomes of interest, we assumed that the null model was associated with a c-statistic of 0.5. When one or more covariates were examined, we calculated the ability of that covariate to predict the outcome as: (c-statistic of model with one or more variables)/(c-statistic of null model).(29) Sensitivity analyses in which the cohort was limited to elderly patients (≥60 years of age) or to patients who underwent laparoscopic hysterectomy. All analyses were performed with SAS version 9.4 (SAS Institute Inc, Cary, North Carolina). All statistical tests were two-sided and a P-value of <0.05 was considered to denote statistical significance.

Results

A total of 66,105 patients were identified. The cohort included 22,801 (34.5%) women who underwent an abdominal hysterectomy, 31,503 (47.7%) who underwent laparoscopic hysterectomy, and 11,801 (17.9%) who underwent vaginal hysterectomy. There were 13,321 (20.2%) patients who were 60 years of age or older.

The clinical and demographic characteristics of the cohort are displayed in Table 2 and Table S1. The majority of women were hospitalized <4 days after surgery and had an ASA class of 1 (12.8%), 2 (64.8%), or 3 (21.4%). Within the cohort, 6683 (10.1%) underwent hysterectomy for uterine cancer, 1626 (2.5%) for ovarian cancer, and 869 (1.3%) for cervical cancer, while the remainder of the patients had surgery for benign gynecologic diseases. The most frequent underlying comorbidities were hypertension (29.9%) and non-insulin dependent diabetes mellitus (5.2%). Transfusion was required in 4.2% of the cohort and the most common postoperative complications were urinary tract infections (2.8%) and superficial surgical site infections (1.6%).

Table 2.

Clinical and demographic characteristics of the cohort.

		N	(%)
		66,105	(100)
Route of hysterectomy
	Abdominal	22,801	(34.5)
	Laparoscopic	31,503	(47.7)
	Vaginal	11,801	(17.9)
Year of procedure
	2008	4988	(7.6)
	2009	7433	(11.2)
	2010	10,427	(15.8)
	2011	18,273	(27.6)
	2012	24,984	(37.8)
Age
	<40	11,586	(17.5)
	40-49	26,627	(40.3)
	50-59	14,571	(22.0)
	60-69	8139	(12.3)
	≥70	5182	(7.8)
Race
	White	48,020	(72.6)
	Black	8035	(12.2)
	Other	2512	(3.8)
	Unknown	7538	(11.4)
BMI
	Normal	17,412	(26.3)
	Overweight	18,936	(28.7)
	Obese	29,374	(44.4)
	Unknown	383	(0.6)
Length of stay
	0	5420	(8.2)
	1	29,584	(44.8)
	2	15,866	(24.0)
	3	8224	(12.4)
	4	2833	(4.3)
	5	1268	(1.9)
	6	788	(1.2)
	7	489	(0.7)
	≥8	1633	(2.5)
ASA Class
	1	8460	(12.8)
	2	42,828	(64.8)
	3	14,131	(21.4)
	4	620	(0.9)
	5	8	(0.01)
	None	37	(0.06)
	Unknown	21	(0.03)
Functional status
	Independent	65,542	(99.2)
	Partially dependent	413	(0.6)
	Totally dependent	60	(0.1)
	Unknown	90	(0.1)

Adverse outcomes increased with an increasing mFI (Table 3). The rate of Clavian IV complications rose from 0.98% in those with an mFI of 0 to 2.97% for those with an mFI of 0.1-0.19, 2.03% for those with an mFI of 0.2-0.29, 3.74% in women with an mFI of 0.3-0.49, and 7.26% for subjects with an mFI of ≥0.5 (P<0.0001). The wound infection rate was 2.41% in those with an mFI of 0, peaked at 5.21% for women with an mFI of 0.1-0.19 and was 4.84% for patients with an mFI of ≥0.5 (P<0.0001). Perioperative mortality was 0.06% in patients with an mFI of 0, 0.27% in those with an mFI of 0-0.09, 0.23% for an mFI of 0.1-0.19, 0.25% with an mFI of 0.2-0.29, 0.57% for those with an mFI of 0.3-0.49 and 3.23% in women with an mFI ≥0.5 (P<0.0001). The results were largely unchanged in sensitivity analyses in which the cohort was limited to elderly patients (≥60 years of age) or to patients who underwent a laparoscopic procedure.

Table 3.

Association between modified frailty index and adverse perioperative outcomes.

	N	Clavian IV complication	Wound complication	Any complication	Mortality
All patients
0	44,045	0.98	2.41	3.71	0.06
0 to 0.09	9341	1.55	3.04	4.79	0.27
0.1 to 0.19	2555	2.97	5.21	6.69	0.23
0.2 to 0.29	7930	2.03	3.44	5.09	0.25
0.3 to 0.49	2110	3.74	4.98	8.01	0.57
≥0.5	124	7.26	4.84	14.52	3.23
P-value		<0.001	<0.001	<0.001	<0.001
Denominator ≥6 *
0	25,610	0.91	2.40	3.71	0.04
0 to 0.09	9341	1.55	3.04	4.79	0.27
0.1 to 0.19	2555	2.97	5.21	6.69	0.23
0.2 to 0.29	356	4.78	5.90	9.55	1.12
0.3 to 0.49	72	12.50	9.72	19.44	5.56
≥0.5	1	100.00	0.00	100.00	0
P-value		<0.001	<0.001	<0.001	<0.001

^*Denotes that data for ≥6 of the variables in the denominator for the mFI were available.

A series of model fit statistics were then calculated to explore the ability of measures of performance to predict outcomes (Table 4). Models for Clavian IV complications demonstrated that, compared to chance alone, the ability to predict Clavian IV complications was increased by 18.8% based on age, 3.8% based on functional status, 24.0% based on ASA, and 22.0% based on mFI. Predictive ability increased to 34.0% in a model containing all 4 factors. For wound complications, ASA was associated with the greatest increase in predictive ability (18.4%) followed by mFI (11.4%), age (3.2%), and functional status (0.6%). Combining all 4 parameters increased the predictive ability of the model to 23.2%. Finally, when any complication was examined, ASA (15.6%) had the highest increase in predictive ability followed by age (12.2%), mFI (11.0%), and functional status (1.8%). The increased predictive ability of the combined model was 20.4%.

Table 4.

Significance of preoperative measures of performance and functional status on perioperative adverse outcomes.

Outcome	Model	C-Statistic	Increased ability to distinguish outcome beyond chance	Pseudo- R2	AIC	LRT
Clavian IV complications	Age	0.594	18.8%	0.0162	9,413.064	144.21
	Functional status	0.519	3.8%	0.0098	9,467.744	87.53
	ASA	0.620	24.0%	0.0164	9,415.644	145.63
	Modified frailty index	0.610	22.0%	0.0214	9,368.425	190.85
	Combined functional indices	0.670	34.0%	-	9,148.331	-
	Full model	0.730	-	-	8,759.190	-
Wound complications	Age	0.516	3.2%	0.0006	16,965.325	8.90
	Functional status	0.503	0.6%	0.0005	16,964.595	7.63
	ASA	0.592	18.4%	0.0179	16,709.570	268.65
	Modified frailty index	0.557	11.4%	0.0076	16,862.088	114.13
	Combined functional indices	0.616	23.2%	-	16,662.320	-
	Full model	0.680	-	-	16,180.610	-
Any complication	Age	0.561	12.2%	0.0083	23,304.586	163.9
	Functional status	0.509	1.8%	0.0042	23,384.079	82.41
	ASA	0.578	15.6%	0.0150	23,175.429	297.06
	Modified frailty index	0.555	11.0%	0.0082	23,307.873	162.62
	Combined functional indices	0.602	20.4%	-	23,057.360	-
	Full model	0.629	-	-	27,748.430	-

^*Full model includes route of hysterectomy, age, race, BMI, ASA class, functional status, albumin, year of surgery, frailty index, and presence of preoperative conditions (diabetes mellitus, tobacco use, COPD, CHF, hypertension, bleeding disorders, disseminated cancer, weight loss, preoperative transfusion, and steroid use).

^**Combined functional indices includes age, functional status, ASA class, and frailty index

^***Increased ability to distinguish outcome beyond chance was calculated as (c-statistics-0.5)/0.5

Discussion

Main Findings

Our data suggest that a modified frailty index is associated with adverse outcomes for patients undergoing major gynecologic surgery. The frailty index is clinically applicable as it captures variables that can be routinely abstracted from the medical record. The ability to predict adverse outcomes is greatest when age, ASA class, functional status, and the modified frailty index are used in combination.

Strengths and Limitations

We recognize a number of important limitations. Despite the fact that NSQIP employs a rigorous methodology for data collection and identification of postoperative complications, our findings are retrospective and subject to bias and should be confirmed in prospective trials. Second, although our study captures patients from a relatively large number of hospitals, centers that participate in NSQIP may not be representative of the entire universe of hospitals in the United States. Overall, the morbidity of gynecologic surgery is low compared to other higher risk procedures and the differences across frailty scores were relatively modest.(21) Third, data for NSQIP is abstracted by trained registrars. While this results in a high degree of conformity, further work is required to determine the validity of data abstraction and mFI calculation by clinicians in routine care settings. Fourth, further work incorporating the mFI and preoperative factors as well as planned surgical complexity may further allow risk stratification and warrants further consideration. Lastly, the mFI is unable to measure the physical phenotype of frailty through capture of such variables as weakness and decreased physical activity.(26) While mFI may be associated with outcome, further study examining how to incorporate this metric into shared decision making is needed.

Interpretation

There is growing recognition that there is widespread variation in the risk of adverse outcomes for patients with the same chronologic age who undergo surgery. The difference in risk appears to be due in large part to differences in underlying functional status. A number of different risk assessment and prediction tools have emerged over the years to help guide clinicians(16) The American Society of Anesthesiologists (ASA) physical status classification system is a subjective assessment of preoperative risk that is one of the most widely used risk assessment tools. It is a simple system that classifies patients into one of six categories and helps identify patients that may benefit from more intensive preoperative evaluation and care.(16) Other systems include the Acute Physiology and Chronic Health Evaluation (APACHE-II), the Physiologic and Severity Score for the Enumeration of Mortality and Morbidity (POSSUM), the Goldman Cardiac Risk Index, and the Prognostic Nutritional Index.(17)

Frailty is an emerging concept in the medical and surgical literature. A number of models to measure aspects of frailty have been proposed and have been correlated with adverse outcomes.(18, 25, 30) The deficit model consists of adding together an individual’s number of impairments and conditions to create a Frailty Index as previously described.(25, 30) The CSHA system is a more extensive model and incorporates multiple domains of function and was used in our study to calculate the mFI.(30) Prior studies have demonstrated that the mFI can be constructed from routinely collected data and is predictive of surgical outcomes for a number of procedures.(17, 21-23, 31, 32) Increasing mFI has been associated with increased morbidity and mortality for procedures in general surgery,(17, 31) vascular surgery,(21) and colorectal surgery.(33)

The use of the frailty index in the gynecologic surgery literature is limited. One study by Courtney-Brooks and colleagues examined a small cohort of women who underwent a major abdominal surgery for a gynecologic malignancy and used a phenotypic frailty model to categorize each participant. They found that 30-day surgical complications increased with frailty score (24% versus 67% for women who were not frail versus those that were frail).(22) Our data suggests that frailty is associated with adverse outcomes in women who undergo hysterectomy for gynecologic cancers as well as in women with benign gynecologic diseases.

Given that a number of instruments are available to assess frailty, an important consideration is how these measures can complement one another. An analysis of the mFI in vascular surgery looked at a number of other variables in NSQIP, including ASA class, and found that the mFI was the strongest predictor of mortality.(21) A review of different instruments available to measure functional status undertaken in 2011 concluded that the frailty index was widely applicable both clinically and for research.(34) In our analysis, we noted that the ability to predict adverse outcomes was increased when four factors (age, ASA class, functional status, and mFI) were combined. Thus, a combined model, which still captures routinely collected data and would be easily applicable to both the clinical and research setting, may the best model for further study.

Conclusion

In sum, these data suggest that frailty is associated with adverse outcomes in women undergoing major gynecologic surgery. The combination of frailty, age, and functional status is associated with a higher predictive value for morbidity than any of the variables alone. Given that the mFI can be calculated from routinely collected clinical data, these findings suggest that frailty may be a useful preoperative measure in patients who are candidates for hysterectomy.