Exploring Clinically Meaningful Changes for the Frailty Index in a Longitudinal Cohort of Hospitalized Older Patients

Abstract

Background

Clinically meaningful change (CMC) for frailty index (FI) scores is little studied. We estimated the CMC by associating changes in FI scores with changes in the Clinical Frailty Scale (CFS) in hospitalized patients.

Methods

The Serious Outcomes Surveillance Network of the Canadian Immunization Research Network enrolled older adults (65+ years) admitted to hospital with acute respiratory illness (mean age = 79.6 ± 8.4 years; 52.7% female). Patients were assigned CFS and 39-item FI scores in-person at admission and via telephone at 1-month postdischarge. Baseline frailty state was assessed at admission using health status 2 weeks before admission. We classified those whose CFS scores remained unchanged (n = 1,534) or increased (n = 4,390) from baseline to hospital admission, and whose CFS scores remained unchanged (n = 1,565) or decreased (n = 2,546) from admission to postdischarge. For each group, the CMC was represented as the FI score change value that best predicted one level CFS change, having the largest Youden J value in comparison to no change.

Results

From baseline to admission, 74.1% increased CFS by ≥1 level. From admission to postdischarge, 61.9% decreased CFS by ≥1 levels. A change in FI score of 0.03 best predicted both one-level CFS increase (sensitivity = 70%; specificity = 69%) and decrease (sensitivity = 66%; specificity = 61%) in comparison to no change. Of those who changed CFS by ≥1 levels, 70.9% (baseline to admission) and 72.4% (admission to postdischarge) changed their FI score by at least 0.03.

Conclusions

A clinically meaningful change of 0.03 in the frailty index score holds promise as a benchmark for assessing the meaningfulness of frailty interventions.

Frailty is the degree of vulnerability after experiencing a stressor, due to age-related decrements in multiple physiological systems (1). Severely frail people experience prolonged multisystem health complications and require more health care resources than their nonfrail counterparts. In many people, frailty can be prevented or successfully managed (2,3). Addressing frailty systematically is paramount for guiding patient care plans and informing public policy.

One approach for assessing this common age-associated state, the frailty index (FI), uses the proportion of deficits accrued in a set of health-related deficits to encapsulate an individual’s degree of vulnerability (4), providing a continuous score from 0 to 1. Any FI should encompass information about a variety of physiological systems, as well as manifestations of their single or combined deficiencies (eg, in cognition, function, mobility). There is no requirement that FIs always contain the same items or are measured in the same way; the FI pertains to a specific setting, while still being general enough to reflect an individual’s overall health. The FI must include a minimum of 30 items to robustly assess the multifactorial nature of frailty. It has shown strong utility for predicting risk of mortality (5–8), functional decline (7–9), and use of healthcare resources (10–12). The FI has been applied to research practice around the world, including an electronic FI which has been gaining traction in routine care in England (13) and the United States (14). In this context, defining the clinically meaningful change (CMC) for the FI can help patients better understand changes in their health and clinicians gauge the efficacy of frailty interventions.

The International Conference on Frailty and Sarcopenia Research Task Force recently commented that for frailty clinical studies, clinically meaningful outcome measures are needed to monitor disease progression and efficacy of interventions, and to design future clinical trials (15). A CMC refers to the difference in a continuous measure that best predicts the smallest clinically meaningful change in a strongly associated measure of the same outcome (16–18). The Clinical Frailty Scale (CFS) (19) is a useful benchmark for establishing CMC for frailty tools because it uses a clinician’s judgment of the patient’s frailty level and has distinct, graded categories. Change in a CFS score is meant to indicate that a health care professional has evaluated a patient’s health as being significantly different from their last evaluation (20). The convenience and utility of the CFS have led to its implementation internationally, and like the FI, the CFS strongly predicts adverse outcomes (21–23). Here, our objective was to explore the CMC in the FI by associating changes in the continuous FI scores with changes in the CFS.

Methods

Data Source

The Serious Outcomes Surveillance (SOS) Network of the Canadian Immunization Research Network (CIRN) collects longitudinal health data from older adults (aged 65+ years) admitted to hospital with acute respiratory illness. Each influenza season, CIRN SOS records real-time data on patient health status from 10 to 45 hospitals across Canada. Details regarding original data collection and ethics approval have been described previously (24,25). Secondary analysis for this study was exempt from research ethics board review in accordance with section 4.2.3 of the Nova Scotia Health Authority Standard Operating Procedure #NSHA-REB-SOP-4-001 (revised September 2017).

Sample

Here, we analyzed CIRN SOS health data collected during three influenza seasons (2011/2012, 2012/2013, and 2013/2014) for 7,254 patients aged 65 or older. Data were collected upon admission to hospital (also collected details on baseline functioning 2 weeks prior) and via telephone interview at 30 days postdischarge (“postdischarge”). Our two subgroups of interest were patients who were in the same or poorer health state upon admission compared to baseline and those who maintained or improved their health between admission and postdischarge (Figure 1).

Figure 1.

Flowchart

In total, 1,274 patients from baseline to admission analyses and 2,614 patients from admission to postdischarge analyses were excluded due to missing either CFS or FI scores; 507 patients died between admission and 30 days postdischarge. We also excluded 13 patients from the baseline to admission analyses, and 128 patients from admission to postdischarge analyses with CFS scores of 9 (“Terminally ill”) due to the unlikely possibility of change at follow-up. Similarly, eight patients with CFS scores of 1 (“Very fit”) at admission were excluded as these patients could not improve further between admission and postdischarge (Figure 1). We removed an additional 14 individuals who became less frail from baseline to admission, and 385 individuals who became frailer from admission to postdischarge. The 385 patients excluded from admission to postdischarge were older, less frail, and had a longer length of stay than those who maintained or improved CFS, but had similar proportions of females included (see Supplementary Table S1). These individuals were removed to better address the purpose of this paper, to look at the minimal frailty change expected for a typical hospitalization. It is generally expected that most people who develop acute illness are worse at admission compared to baseline and better at postdischarge compared to admission. Even so, some people (here 8.6%) are worse at postdischarge compared to admission due to a variety of reasons. These individuals, which were excluded here, should be included when examining recovery and changes in frailty during hospitalization. Final samples were 5,924 for baseline to admission analyses and 4,111 for admission to postdischarge analyses (Figure 1).

Clinical Frailty Scale

CFS scores initially ranged from 1 (“Very fit”) to 7 (“Severely frail”) (18); later, levels 8 (“Very severely frail”) and 9 (“Terminally ill”) were added. CFS scores were assigned by the SOS Network monitors who interviewed patients at admission, reviewed their medical records, and conducted telephone interviews 1 month after discharge from hospital.

Frailty Index

An FI was constructed using the deficit accumulation method described by Searle et al. (4), comprised of 39 health-related items (eg, cognition, I/ADLs, chronic conditions, medications) calculated for baseline, admission, and 1-month postdischarge (items and methodology described previously (24)). All FI items collected in-person at admission were also collected via telephone interview at postdischarge. Like the CFS, a higher FI score indicates more severe frailty. Theoretically, FI scores can range from 0 to 1; however, it is less common to see scores reflecting the presence of more than two thirds of the deficits being measured (ie, for this FI of 39 items, a submaximal limit of ~26 deficits) (4), and the 99% percentile is consistently less than 0.7. Beyond these observed limits, the rate of deficit accumulation is negligible (26).

Clinically Meaningful Change

To calculate the CMC, we utilized an anchor-based approach where the change in the outcome measure is compared to the smallest possible change in another clinical measure—the anchor—that is theoretically related and strongly correlated to the outcome measure (18,27). For this study, the CMC is represented by the degree of FI change that best predicts the smallest possible (one-level) increase or decrease in CFS compared to stability. It can also be calculated as the amount of frailty change needed to produce at least a small meaningful effect (a Cohen’s d ≥0.2) (28). Since the CMC value can be affected by the direction of patient health changes (18), it was calculated separately for prehospitalization (baseline to admission) and posthospitalization (admission to postdischarge). It is generally expected that a patient will worsen their condition until they have entered the hospital, and improve over the course of their admission.

Statistical Analyses

Descriptive characteristics of the sample are expressed as the mean scores ± standard deviation. Mean FI changes were calculated separately for baseline to admission and admission to postdischarge analyses. Worsened frailty was represented by an increase in CFS score (eg, from 2 to 6; a change of +4), and improved frailty by a decrease in CFS score (eg, from 3 to 1; a change of −2). Two receiver operating characteristic (ROC) curves mapped the sensitivity and 1-specificity values for each FI change value predicting a one-level CFS increase (baseline to admission) or a one-level CFS decrease (admission to postdischarge) in comparison to no change in the CFS. Inherent to the grading of CFS, moving up or down a category is considered the smallest significant change in state. Any CFS changes greater than this are informative but lie at least one level above the minimum. For the purpose of this study, defining the minimum change in FI that maps to the smallest significant change in CFS required excluding CFS changes larger than one level from the ROC analyses.

For each group, the CMC was reflected by the FI change value with the largest Youden Index (J) value, representing the point on the ROC curve with the largest vertical distance to the diagonal line of chance, the optimal combination of sensitivity and specificity. This value was compared to the change in FI needed to produce at least a small meaningful effect (a Cohen’s d ≥0.2), calculated as the product of d and the standard deviation of baseline (baseline to admission group) or admission (admission to postdischarge group) FI scores (28). Prevalence of the selected CMC and general mean changes in FI per degree of CFS change were also explored. All analyses were completed using IBM SPSS Statistics Version 25.0.

Results

Baseline to Admission

The first set of analyses comprises 5,924 individuals who increased (N = 4,390; M_Age = 79.7 ± 8.3 years; 52.4% female) or maintained (N = 1,534; M_Age = 79.4 ± 8.46 years; 54.4% female) their CFS scores from baseline to admission (Table 1). Spearman’s rank correlation coefficients between CFS and FI scores at baseline and admission were 0.68 and 0.74 (p < .001), respectively.

Table 1.

Descriptive Characteristics

	Baseline to Admission				Admission to Postdischarge
	Maintained CFS (N = 1,534)		Worsened CFS (N = 4,390)		Maintained CFS (N = 1,565)		Improved CFS (N = 2,546)
Age (Myears ± SD)	79.4 ± 8.5		79.7 ± 8.3		79.4 ± 8.3		78.6 ± 8.1
Median	80.0		80.0		80.0		79.0
% Female (N)	54.4% (834)		52.4% (2,300)		54.2% (849)		53.1% (1,353)
LOS (Mdays ± SD)	9.5 ± 10.4		11.5 ± 13.1		9.7 ± 9.3		10.0 ± 10.5
Median	7.0		8.0		7.0		7.0
FI Change (M ± SD)	0.02 ± 0.04		0.08 ± 0.07		−0.03 ± 0.06		−0.08 ± 0.07
Median	0.00		0.06		−0.02		−0.06
CFS Change (M ± SD)	0.0 ± 0.0		1.4 ± 0.7		0.0 ± 0.0		−1.4 ± 0.6
Median	0.0		1.0		0.0		−1.0
	Baseline	Admission	Baseline	Admission	Admission	Postdischarge	Admission	Postdischarge
FI Score (M ± SD)	0.23 ± 0.14	0.25 ± 0.14	0.22 ± 0.12	0.30 ± 0.13	0.27 ± 0.14	0.24 ± 0.14	0.27 ± 0.12	0.19 ± 0.11
Median	0.21	0.23	0.21	0.28	0.26	0.21	0.26	0.18
CFS Scores (M ± SD)	4.7 ± 1.6	4.7 ± 1.6	4.2 ± 1.4	5.6 ± 1.4	5.0 ± 1.5	5.0 ± 1.5	5.4 ± 1.2	4.0 ± 1.3
Median	4.0	4.0	4.0	6.0	5.0	5.0	5.0	4.0
1 (Very fit)	7	7	37	0	N/A	N/A	N/A	22
2 (Well)	48	48	244	13	31	31	7	147
3 (Managing well)	407	407	1,257	150	296	296	85	875
4 (Vulnerable)	350	350	1,150	896	338	338	612	657
5 (Mildly frail)	183	183	850	978	228	228	650	444
6 (Moderately frail)	269	269	587	1,154	329	329	659	318
7 (Severely frail)	240	240	230	823	299	299	438	83
8 (Very severely frail)	30	30	35	295	44	44	95	0
9 (Terminally ill)	N/A	N/A	N/A	81	N/A	N/A	N/A	N/A

Note: Samples do not include CFS changes >±4 levels due to small sample size (see Figure 1). CFS = Clinical Frailty Scale. FI = Frailty Index; LOS = length of hospital stay; M = mean; SD = standard deviation.

Overall, 25.9% of this sample had no change at CFS score at admission compared to baseline, 51.2% increased by one level, 17.1% by two levels, 4.4% by three levels, 1.3% by four levels, and 0.5% increased more than four levels (maximum seven levels change; Figure 1). On average, patients who worsened CFS by one level had a mean FI change of 0.06 ± 0.06, which increased steadily up to 0.19 ± 0.08 for those who worsened CFS by four levels (Figure 2).

Figure 2.

Mean FI change per level of CFS change. CFS = Clinical Frailty Scale; FI = Frailty Index.

The area under the ROC for predicting one-level worsening in CFS compared to no change was 0.76 (95% confidence interval [CI]: 0.74–0.77). An FI change of 0.03 was the most effective for predicting a one-level increase in CFS, having the largest Youden Index (J) of 0.40 (sensitivity = 70%, specificity = 69%; Table 2 and Supplementary Table S2). The optimal FI change was consistent when stratifying by sex (Table 2). The minimal FI change needed for a Cohen’s d ≥0.2 was 0.025 (0.2 × 0.1247). Approximately two thirds of the participants who worsened CFS by one level had an FI change of 0.03 or greater, and this proportion grew more substantial with each additional CFS level change (Table 3, Supplementary Table S3, and Supplementary Table S4).

Table 2.

Optimal FI Change Values for Predicting One-Level Change in CFS

Baseline to Admission	N with no CFS Change	N with +1 CFS Change	AUC Predicting +1 CFS Change (95% CI)	Change in FI with Highest J	J (Sensitivity, Specificity)
All	1,534	3,035	0.76 (0.74–0.77)	0.026	0.40 (0.70, 0.69)
Males	700	1,417	0.75 (0.73–0.77)	0.026	0.40 (0.69, 0.71)
Females	834	1,618	0.76 (0.75–0.78)	0.029	0.40 (0.65, 0.76)
Admission to Postdischarge	N with no CFS Change	N with -1 CFS Change	AUC Predicting -1 CFS Change (95% CI)	Change in FI with Highest J	J (Sensitivity, Specificity)
All	1,565	1,819	0.68 (0.66–0.70)	−0.032	0.27 (0.66, 0.61)
Males	716	855	0.67 (0.64–0.69)	−0.025	0.25 (0.75, 0.50)
Females	849	964	0.69 (0.66–0.71)	−0.031	0.30 (0.69, 0.61)

Note: Only patients who maintained or changed CFS by 1 level were included in this analysis. AUC = area under the ROC curve; CFS = Clinical Frailty Scale; CI = confidence interval; FI = Frailty Index; J = Youden Index value for the change in FI that best predicts a one-level change in CFS.

Table 3.

Proportion of People Who Changed FI by more than 0.03 Based on Level of CFS Change

Baseline to Admission	CFS Change	N	N (%) Who Changed FI ≥ 0.03
	0	1,534	356 (23.2%)
	+1	3,035	1,899 (62.6%)
	+2	1,013	890 (87.9%)
	+3	263	246 (93.5%)
	+4	79	78 (98.7%)
Admission to Postdischarge	CFS Change	N	N (%) Who Changed FI ≤ −0.03
	0	1,565	623 (39.8%)
	−1	1,819	1,215 (66.8%)
	−2	564	481 (85.3%)
	−3	130	115 (88.5%)
	−4	33	32 (97.0%)

Note: CFS = Clinical Frailty Scale; FI = Frailty Index. CFS change = 0 indicates no change. Percentages reflect the proportion of n within each degree of CFS change whose FI changed by at least +0.03 (baseline to admission) or −0.03 (admission to postdischarge). CFS changes beyond ±4 are not reported due to small sample size.

Admission to Postdischarge

Analyses were also done for 4,111 individuals who maintained (M_Age = 79.4 ± 8.3 years; 54.2% female) or decreased (M_Age = 78.6 ± 8.1 years; 53.1% female) their CFS scores from admission to postdischarge (Table 1). Spearman’s rank correlation coefficients between CFS and FI scores at admission and postdischarge were 0.72 and 0.68 (p < .001), respectively.

Overall, 38.1% of this group had no change of CFS score at postdischarge compared to admission, 44.3% decreased by one level, 13.7% by two levels, 3.2% by three levels, 0.8% by four levels, and 0.2% decreased by more than four levels (maximum six levels change; Figure 1). The mean FI change ranged from −0.06 ± 0.06 for patients who improved CFS by one level, up to −0.18 ± 0.09 for those who improved CFS by four levels (Figure 2).

An ROC where FI changes predicted a one-level decrease in CFS score compared to no change yielded an area under the curve of 0.68 (95% CI: 0.66, 0.70). The optimal FI change coordinate was −0.03, where J = 0.27 (sensitivity = 66%, specificity = 61%; Table 2 and Supplementary Table S2). This value was consistent when stratifying by sex, except that it was marginally smaller for males (Table 2). The minimal FI change needed for a Cohen’s d ≥0.2 was 0.026 (0.2 × 0.1292). Similar to the previous group, roughly two thirds of the participants who improved CFS by one level had an FI change of at least 0.03 with this proportion growing considerably with each additional CFS level change (Table 3, Supplementary Table S3, and Supplementary Table S4).

Discussion

This study showed that a CMC of 0.03 for the FI could identify changes in the level of frailty of hospitalized patients between baseline to admission and from admission to 1-month postdischarge. In an FI including 30 or more items, such as the one used in this study, this translates to at least a one-deficit change (ie, 1/30 or 1/40 = 0.03) indicating that at minimum, a one-deficit change is a significant improvement or deterioration in frailty state.

The FI is often used as a predictor in longitudinal observational studies but rarely as an outcome measure (29,30). The evidence is even more limited for using the FI as an outcome measure in interventional studies (29,31–33). When examined as a research outcome, the FI is usually tested as a continuous variable (29,30,34,35). Though the value of this is important for research purposes, the magnitude of any FI change may not always translate to clinical implications. Only one study (36) has identified what constitutes a clinically meaningful FI change. They identified a small CMC for the FI of 0.03, supporting our findings in this study. A small number of randomized control trials (RCT) have used change in FI as an outcome. The earliest considered any change in FI to be an improvement due to testosterone and nutritional supplementation (34). Others have used a one-deficit change as an appropriate benchmark (20), while others have used a specific change of 0.03 (35,37) to represent significant posttreatment change. An RCT examining the effects of multidomain lifestyle interventions on frailty in patients with diabetes found an approximately one deficit (~0.03 for their 38-item FI) drop upon 1-year follow-up (38). Another RCT targeting cognitive function found a reduction in FI of 0.02 (95% CI 0.02–0.03) after 6 months (39). A study using L-carnitine supplementation found that FI scores changed by 0.02 at 5-week follow-up from baseline, jumping to a difference of 0.04 by 10 weeks (40).

A limitation of this study is that, as is routine care for this clinical setting, baseline and admission medical health history were collected concurrently. It is generally expected that a patient’s condition will be worse in the lead up to hospital admission. While this expectation can lead to greater disparity between baseline and admission frailty scores, we expect that any bias that affects baseline frailty scores affected both CFS and FI in a similar manner. Therefore, while the magnitude of change within each tool can possibly become inflated, it is unlikely that the relationship between them was undermined. Further, the area under the curve for the admission to discharge ROC was 0.68—below the generally accepted level of 0.7. Lower AUCs are common in frailty studies and could be related to the heterogeneity of older adults within clinical settings and the multidomain nature of frailty. Also, a 0.03 change in the FI is not experienced equally at all levels of frailty. How this change manifests among these levels would be valuable to investigate.

Further, we excluded patients who died during hospitalization and a minority of patients who worsened between admission and postdischarge. These patients are important to include when examining the impact of hospitalization on frailty. The CFS increase was similar between the groups who worsened between baseline to admission and those who worsened between admission and postdischarge. Even so, the FI increase was much higher for the baseline to admission CFS worsening group with 70.9% of the patients experiencing an FI increase greater than 0.03; only 35.6% of the patients of the admission to postdischarge CFS worsening group experienced this CMC FI increase. It is even more surprising that 21.8% of this latter worsening group experienced an FI decrease greater than 0.03. CFS and FI measure the same construct but CFS relies on a clinician’s overall impression and FI on a series of examinations. It is possible that other factors not explained by medical tests could affect the subjective assessment of the clinician for this atypical group. Future studies should investigate this further.

Going forward, this CMC should be validated across different samples and FIs. This study followed patients over the course of hospitalization due to illness. More research should be done using the FI as an outcome in exercise and pharmaceutical interventions. Testing the CMC in such studies will further support its utility. Future work may also distinguish the CMC in different genres of FI, such as an FI constructed from objective markers (eg, FI based on laboratory tests).

This study used a cohort of acutely ill individuals who experienced short-term changes in frailty levels. In nonacutely ill older adults, health changes are likely to occur more gradually. Considering that our benchmark of 0.03 corresponds with the average annual rate of FI change for community-dwelling people (26,41), many individuals in our sample experienced changes much larger than that in a shorter period of time, though their net change from baseline to postdischarge may not have been drastic. Future research should replicate this study in nonacutely ill community-dwelling older adults.

The increasing number of frail people poses a great challenge on health care resources. We need to prioritize frailty management, as well as the continuous development and improvement of frailty assessment tools. Using the CMC identified for the FI, a widely used tool in clinical and research settings, to aid with the interpretation of frailty change can add to the effectiveness of frailty interventions and patient care plans and support the FI’s utility in future clinical trials.

Funding

This work was supported by funding from the Fountain Family Innovation Fund of the QEII Health Sciences Centre Foundation. The SOS Network was supported through an investigator-initiated Collaborative Research Agreement with GlaxoSmithKline Biologicals SA, and by the Public Health Agency of Canada and the Canadian Institutes of Health Research to the Canadian Immunization Research Network (CIRN).

Conflict of Interest

M.K.A. reports grant funding from GlaxoSmithKline (GSK), Pfizer, Sanofi, Canadian Institute of Health Research (CIHR), Public Health Agency of Canada (PHAC), and the Canadian Frailty Network, and honoraria from Pfizer, Sanofi, and the Canadian Frailty Network. S.A.M. has received research grants and consultancy fees from GlaxoSmithKline Biologicals SA and Pfizer and has participated in Clinical Trials funded by GSK, Merck, Novartis, Pfizer, and Sanofi Pasteur. K.R. is President and Chief Science Officer of DGI Clinical, which in the last 5 years has contracts with pharma and device manufacturers (Baxter, Baxalta, Shire, Hollister, Nutricia, Roche, Otsuka) on individualized outcome measurement. In 2017, he attended an advisory board meeting with Lundbeck. Otherwise, any personal fees are for invited guest lectures and academic symposia, received directly from event organizers, chiefly for presentations on frailty. He is Associate Director of the Canadian Consortium on Neurodegeneration in Aging, which is funded by the Canadian Institutes of Health Research, and with additional funding from the Alzheimer Society of Canada and several other charities, as well as, in its first phase (2013–2018), from Pfizer Canada and Sanofi Canada. The rest of the authors have no declarations of interest.