Are Performance Measures Necessary to Predict Loss of Independence in Elderly People?

Abstract

Background.

The frailty phenotype (FP) proposed by Fried and colleagues (Fried LP, Tangen CM, Walston J, et al.; Cardiovascular Health Study Collaborative Research Group. Frailty in older adults: evidence for a phenotype. J Gerontol A Biol Sci Med Sci. 2001;56:M146–M156.) requires the administration of performance tests (gait speed, handgrip strength) not always feasible in routine clinical practice. Furthermore, the discriminative capacity of the instrument has been rarely investigated. Aim of this study was to evaluate the discriminative capacity of the FP and compare it with a modified version including only anamnestic information.

Methods.

Data are from 890 participants of the InCHIANTI study without impairment in activities of daily living (ADL) at baseline (mean age 74 years, women 55%). Frailty was defined by (a) the presence of ≥3 criteria of the FP, and (b) having ≥2 criteria of an anamnestic FP (AFP), not including gait speed and handgrip strength. Sensitivity, specificity, positive and negative predictive values (PPV, NPV) were used to evaluate the discriminative capacity of both definitions for incident disability (ie, loss of at least one ADL), incidence of “accelerated” disability (loss of >2 ADL) over a 6-year follow-up, and 5-years mortality.

Results.

FP and AFP yielded a frailty prevalence of 6.4% and 6.5%, respectively; only 32 patients were considered frail by both indices (kappa: .53). For incident disability, FP showed sensitivity = .194, specificity = .963, PPV = .400, and NPV = .903. Similarly, AFP had sensitivity = .129, specificity = .949, PPV = .245, and NPV = .894. Consistent results were found for accelerated disability and mortality.

Conclusions.

In our sample, both FP and AFP showed low sensitivity in identifying older people who would die or develop disability, but they could well discriminate people who would not experience adverse outcomes.

Although a unique definition does not exists, international experts agree that frailty is a multidimensional syndrome characterized by an increased individual’s vulnerability for developing dependency and/or dying when exposed to a stressor (1,2). A significant number of people over the age of 65 years is considered frail: a recent systematic review including 31 studies about frailty found a prevalence of physical frailty ranging from 4% to 17% (mean 9.9%), with a higher prevalence in persons older than 80 years (3). Frailty is considered to confer higher risk of adverse outcomes, including dependency, institutionalization, falls, and mortality (4–7). Therefore, although finite evidence is not yet available, a consensus group consisting of delegates from six major international, European and U.S. societies agreed that health care providers should screen for frailty all people 70 years and older (8). Screening for frailty is indeed noninvasive and may identify potential remedial conditions (8). Nevertheless, to date there is no clear consensus regarding a simple operational definition of frailty able to recognize the syndrome and render it objectively measurable (2,4,7,9–11). Among the proposed definitions, the frailty phenotype (FP) represents one of the most common approaches to identify physical frailty (4). According to this instrument, frailty is considered a pre-disability state (a “physiological precursor” and “etiological factor” in disability), independently of the presence of concomitant diseases, therefore representing a potential useful tool for the initial risk stratification of older people and for preventative interventions (12). However, most research on frailty has studied the FP as a risk factor for disability (hence calculating relative measures of association) while much less attention has been devoted to the prognostic performance of the FP in terms of calibration and discriminative capacity. Strong statistical associations between an outcome and a marker, in fact, do not necessarily imply that the marker can discriminate between persons likely to have the outcome and those who do not (13,14). In confirmation of that, according to literature data, the available frailty scores, including FP, are of limited value for both screening and diagnostic purposes in daily practice (15–17).

In addition, although the FP is based on relatively simple tasks (exhaustion, involuntary weight loss, poor handgrip strength, slow gait speed, and inactivity) (4), assessing performance measures in everyday clinical practice may result sometimes problematic. For instance, the evaluation of muscle strength and gait speed may be difficult, especially in primary care, due to the lack of dynamometers and enough space/time to assess walking speed. Moreover, concomitant disabling conditions may affect the predictive value of the phenotype for negative health-related outcomes (18). On these assumptions, the aims of this study are twofold: first, to evaluate the discriminative capacity of the FP relative to death and incident disability in a population of Italian community-dwelling elderly people; second, to compare the prognostic performance and discriminative capacity of the full index and of a reduced index not including performance measures.

Methods

Data Source

We used data from the InCHIANTI study, which was designed to investigate the factors contributing to the decline of mobility in older persons (19). The participants in the study were randomly selected from the populations of two town areas in the Chianti region: Greve in Chianti and Bagno a Ripoli. The study protocol was approved by the Italian National Institute of Research and Care on Aging ethical committee. The eligible participants were interviewed at their homes by trained study researchers using a structured questionnaire aimed at investigating their health status, their physical and cognitive performance, and other factors possibly related to loss of independence in late life. The interview was followed by a physical examination at the study clinic. Comorbid diseases were ascertained examining clinical history and medical records. Mood status was evaluated using the CES-D scale (20). The first wave of the study started in 1998 and participants were followed-up with evaluations at 3 and 6 years.

Measures of Performance and Disability

Frailty indicators were defined mirroring Fried and colleagues’ criteria (4). Unintentional weight loss was recorded at baseline during medical interview and was defined as a reduction in weight more than 4.5kg in the past 12 months. Exhaustion was defined as a feeling of needing an effort to do everything, and was considered present if the participant reported it for more than 3 or 4 days in the last week. Reduced physical activity was defined as having performed less than 2–4 hours of light exercise per week. Walking speed was evaluated over a 4.57 m course with the patient taking two walks at usual pace. The mean of the two walks was considered, and those with a walking speed below the lowest quintile as adjusted for sex and height were considered slow walkers. Finally, grip strength was measured using a hand-held dynamometer. The average of two measurements was used, and those with a grip strength below the sex and body mass index (quartiles) specific 20th percentile was considered to have low grip strength. A “frailty phenotype” (FP) ranging from 0 to 5 was then calculated counting the number of Fried frailty indicators present. An alternative definition was based only on the “anamnestic” criteria that do not require performance measures (weight loss, exhaustion, and low activity): the “anamnestic” frailty phenotype (AFP) ranging from 0 to 3 was also calculated. The cut-off value to identify frail subjects according to the FP was 3, as suggested in the original article (4). For the AFP, we arbitrarily chose a cut-off of 2, in order to maximize classificatory agreement between FP and AFP.

Disability was evaluated using the basic activities of daily living (ADL): dressing, moving in and out of bed, using the toilet, washing, eating, and control urine and fecal continence (21). Disability was defined as loss of one ADL at follow-up whereas “accelerated” disability as loss of two or more ADL.

Sample Selection

From the original study population we selected participants with age ≥65 years and without disability at baseline (N: 1,039). Thereafter we excluded those without available data on frailty measures (N: 149): these participants were at higher risk for developing disability as they were older and more frequently affected by cardiovascular diseases. Vital status was available for all the remaining participants (N: 890), that were retained for the analysis on mortality. For the analysis on incident disability, we excluded participants with no information at both follow-up Visits 1 and 2, leaving 815 participants available for analysis.

Analytic Approach

To calculate loss of ADL we used data from the 6-year follow-up, that was available for 705 participants, and in the other cases we used data from the 3-year follow-up.

The agreement of FP and AFP in identifying frailty was evaluated using the kappa statistic.

We evaluated the association between presence of FP or AFP with 5-year mortality using Kaplan–Meier curves along with log-rank test, and by calculating incidence rate ratios. The relationship between positive FP or AFP and incidence of disability and “accelerated” disability was evaluated using the cumulative incidence estimated from contingency tables.

We also examined the prognostic capacity of FP and AFP in predicting mortality and disability. The overall discriminative capacity was estimated using the c statistic obtained from Cox proportional hazard models for mortality, and from logistic regression models for disability. This statistic ranges from 0 to 1 and gives the same information of the area under the receiver operating curve, with values close to one indicating good discrimination. The discriminative capacities of a positive FP and AFP were evaluated calculating the probability of having the outcome of interest in those identified as frail (positive predictive value) and of not having the outcome of interest in those identified as not frail (negative predictive value). These probabilities were estimated using the product-limit method for mortality, and from contingency tables for disability.

All the analyses were performed using R 3.1 for Linux (R Foundation for Statistical Computing, Vienna, Austria).

Results

The characteristics of the complete sample and of the groups of participants with FP and AFP are reported in Table 1. The mean age was 73.8 years (SD: 6.6), women were 55.5%, 70.2% in the group with FP and 77.6% in the group with AFP. While 12.1% of the frail people in the AFP group had all three anamnestic criteria positive, none of the participants had all five Fried’s criteria positive. In the group with FP we observed a higher prevalence of hypertension (78.9% vs 70.7% in the AFP group), diabetes mellitus (17.5% vs 6.9%), peripheral artery disease (29.8% vs 19%), and Parkinson’s disease (3.5% vs 1.7%), while renal insufficiency, cancer, and depression were more prevalent in the AFP group (Table 1). The prevalence of positive FP and AFP in the whole sample was 6.4% and 6.5%, respectively, and only 32 patients were classified as frail by both indices (kappa: .53). The prevalence of the frailty indicators in the two groups of participants with FP and AFP are reported in Figure 1. We observed a greater prevalence of exhaustion in the AFP group compared with FP group. Tables 2 and 3 show percentages of positive criteria in frail participants according to both definitions.

Table 1.

Characteristics of Total Population and of Subgroups of Participants With Frailty Phenotype and Anamnestic Frailty Phenotype

	All	Frailty Phenotype	Anamnestic Frailty Phenotype
N	890	57	58
Age, mean (SD)	73.8 (6.6)	80.1 (6.9)	76.9 (6.9)
Gender (F), %	55.5	70.2	77.6
Hypertension, %	66	78.9	70.7
Ischemic heart disease, %	10.8	22.8	20.7
Cerebrovascular disease, %	5.6	7	8.6
Renal insufficiency, %	60.1	71.9	79.3
Parkinson’s disease, %	1.7	3.5	1.7
Peripheral artery disease, %	16.3	29.8	19
Diabetes mellitus, %	11.7	17.5	6.9
Chronic obstructive pulmonary disease, %	10.4	15.8	13.8
Hip or knee osteoarthritis, %	29.4	47.4	48.3
Cancer, %	5.7	5.3	6.9
Depression, %	29.4	61.4	69.0

Figure 1.

Prevalence of frailty indicators in the two groups of participants with frailty phenotype and anamnestic frailty phenotype.

Table 2.

Percentages of Positive Criteria in Frail Participants According to Frailty Phenotype (numbers are row percentages)

	Reduced Physical Activity	Exhaustion	Weight Loss	Reduced Grip Strength	Slow Walking Speed
Reduced physical activity	—	55.6	20.0	68.9	82.2
Exhaustion	73.5	—	35.3	55.9	76.5
Weight loss	52.9	70.6	—	58.8	70.6
Reduced grip strength	75.6	46.3	24.4	—	80.5
Slow walking speed	77.1	54.2	25.0	68.8	—

Table 3.

Percentages of Positive Criteria in Frail Participants According to Anamnestic Frailty Phenotype (numbers are row percentages)

	Reduced Physical Activity	Exhaustion	Weight Loss
Reduced physical activity	—	89.6	25.0
Exhaustion	81.1	—	32.1
Weight loss	54.5	77.3	—

Over 5 years, we observed 106 deaths, with an overall estimated risk of mortality of 12%. Figure 2 shows the survival probability over the follow-up time of people identified as frail using the FP (left panel) or the AFP (right panel); the incidence rate ratios were 4.3 (95% CI: 2.6–6.9), the corresponding values for frailty defined using the AFP were 2.3 (95% CI: 1.3–4.0).

Figure 2.

Mortality in frail and nonfrail people according to the frailty phenotype (left) or the anamnestic frailty phenotype (right).

The overall discriminative capacity of both the FP and AFP with respect to mortality was poor, with c statistics of .37 and .41, respectively. The probability of dying was 36.8% among those with a positive FP, and 24.1% among those with a positive AFP. The probability of surviving was 89.8% among those with a negative FP, and 88.9% among those with a negative AFP.

Figure 3 shows the cumulative incidence of disability and accelerated disability according to different scores of FP and AFP. Compared to people without frailty, a positive FP was associated with a 4.1 greater incidence of disability (95% CI: 2.7–6.2) and with a 2.3 greater incidence of accelerated disability (95% CI: 2.7–8.2), while the corresponding figures for a positive AFP index were 2.3 (95% CI: 1.4–3.9) and 2.6 (95% CI: 1.3–5.2), respectively.

Figure 3.

Incidence of disability and accelerated disability according to different scores of frailty phenotype and anamnestic frailty phenotype.

The FP showed a fair discriminative capacity with respect to both incident disability (c = .708) and incident accelerated disability (c = .696). The discriminative capacity was somewhat lower for AFP, with c of .608 and .602 for incident disability and accelerated disability, respectively.

Table 4 provides sensitivity, specificity, and predictive value of adverse outcomes according to FP and AFP. The probability that a subject with a positive FP would develop disability (positive predictive value, PPV) was 40%, while the probability that a subject with a negative FP would not develop disability (negative predictive value, NPV) was 90.3%; the corresponding values for AFP were 24.5% and 89.4%. The results for incident accelerated disability were similar: for a positive FP, PPV and NPV were 26.7% and 94.3%, respectively, and for a positive AFP the corresponding values were 16.3% and 93.7%.

Table 4.

Sensitivity, Specificity, Positive and Negative Predictive Value of Adverse Outcomes According to Frailty Phenotype and Anamnestic Frailty Phenotype

Adverse Outcome	Sensitivity	Specificity	Positive Predictive Value	Negative Predictive Value
Death
FP	.179	.957	.368	.898
AFP	.129	.947	.241	.889
Disability
FP	.194	.963	.4	.903
AFP	.129	.949	.245	.894
Accelerated disability
FP	.214	.957	.267	.943
AFP	.143	.946	.163	.937

Discussion

Our study confirms that frailty is associated with mortality and incident disability. It also shows that a definition of frailty based only on anamnestic information led to a still clinically significant, albeit weaker, association.

The association between frailty and death has been analyzed in several epidemiological studies, finding an odds ratio between 1.21 and 6.03 in follow-up periods of up to 10 years (4,7,22–25). In our study, the probability of dying was 4.2 times greater in frail subjects and 2.3 times greater in people with AFP. Similarly, several studies described an association between frailty and incident disability in ADL. For instance, adjusted odds ratio of 3.2, 3.15, and 4.4 were found in the Three-City Study (23), in the Women’s Health Initiative Observational Study (26), and in the MacArthur Study of Successful Aging (24), respectively. The results of our study are consistent with those reported in literature as we used the Katz ADL Index as assessment tool for disability and we reported in frail people a 4.1 greater incidence of developing disability (95% CI: 2.7–6.2).

This strong association between frailty and our outcomes of interest does not translate in good prognostic properties. The probability of dying or becoming disabled in frail people is relatively low, and therefore FP cannot be used with confidence when trying to identify people who will experience the outcome. On the other hand, the probability of not developing the outcome in nonfrail people is high, allowing identifying with confidence elderly people who are less likely to experience the outcomes. The literature on the prognostic performance of the FP is scant. The only study that reported data obtained from a European population is the subanalysis of The Toledo Study for Healthy Aging including 1,781 elderly people (15). The researchers found that FP had sensitivity of .24 (95% CI: 0.17–0.82), specificity of .93 (0.92–0.94), PPV of .25 (0.18–0.33), and NPV of .93 (0.91–0.94) with respect to mortality. These data are consistent with the results of our study. Pijpers and colleagues (16) in 2012 reviewed the predictive accuracy of several frailty scores, finding that all reviewed tests have low positive predictive values, but reasonable negative predictive values, concluding that frailty scores can be reliably used to exclude frailty, but are of limited value for both screening and diagnostic purposes. They reported that, with respect to mortality, FP has a PPV of 69% and 17% and a NPV of 64% and 95% (in women and men, respectively). Finally, Woo and colleagues (17) found similar prognostic capacity of FP for death in a cohort of 4,000 elderly Chinese people (among men, sensitivity 16%, specificity 95.3%, PPV 41.8%, and NPV 84.2%). Our study was the first specifically designed to address this issue.

The discrepancy between a strong association and a poor prognostic performance is not surprising. Actually it is very common that scoring systems strongly associated with a given outcome have a poor capacity to identify people who will actually develop the outcome itself (13). In order to consider a marker/scale effective for classifying people according to their future outcome, in fact, we need extremely strong associations between the marker/scale and the outcome of interest, expressed by very high odds ratio values that are rarely seen in epidemiological studies. Our data reinforce the evidence that even risk factors strongly associated with an outcome may be not be good predictors of the outcome itself. Poor prognostic capacity of Fried’s frailty criteria can also derive from the characteristics of the instrument itself. As it is based on the evaluation of signs and symptoms, using the worst population quintile as a reference, FP makes it difficult to establish differences among older people over/under the thresholds and it is not able to capture the continuous gradient from robustness to frailty (15). Moreover, as positive criteria are defined by population-specific cut points, misclassification is possible if data on the distribution of the variables in the community studied are lacking.

With respect to prognostic information, the FP and the AFP had similar performances, lacking sensitivity but having a good NPV. As such, they can reliably identify elderly who are at low risk of developing disability or dying, but are not suitable to detect those at higher risk for these outcomes. Therefore, both instruments are inadequate as screening tools in the routine practice but may be useful to plan follow-up of people identified as at low risk to develop adverse outcomes. In this respect, AFP, that does not require performance test, may be more applicable in the routine practice.

Although FP and AFP showed similar prognostic performance and yielded similar prevalence of frailty, only 32 participants were classified as frail by both indices. While disagreement between FP and other frailty scales has been already discussed in literature (27,28), our data reinforce the evidence that sizeable differences may be present even between a scale obtained from a subset of items and its complete counterpart. It must be noted, indeed, that FP and AFP may actually reflect different “latent constructs,” that is, they may not be both measuring “frailty” defined as a state of increased susceptibility to disability. In line with this hypothesis, the higher prevalence of reported exhaustion among frail people according to AFP may be related to the higher prevalence of depression in this group rather than to frailty. This study has several strengths. First, by providing data on the predictive accuracy of FP, we add evidence to a poorly investigated issue. Moreover, we studied this frailty indicator in a general population representative of the real world, contrary to most studies including the original one (4). Finally, since we demonstrated that a simpler index is as effective as Fried’s FP, our results may help to increase the use of frailty measures in everyday clinical practice.

A limitation of our study is that since we studied only the predictive value of Fried’s criteria, our results cannot be extended to other frailty indicators. Moreover, as we included both robust and prefrail elderly in the “nonfrail” group, sensitivity of the FP might have been underestimated. Including prefrail in the “frail” group would have increased sensitivity and NPV, and decreased specificity and PPV, leaving our conclusions unaffected. Another bias may arise from the fact that NPV and PPV are influenced by the prevalence of the outcome that in our case was relatively low (11% for incident disability). In a scenario with 20% incident disability, the PPV would have been of .56 and the NPV .83, once again leaving our conclusions virtually unaffected. Finally, because of lack of follow-up data we excluded from the analysis some participants (about 10% of the original cohort), who were probably at higher risk for developing disability, since they were older and with a higher prevalence of cardiovascular diseases. This may in part explain the low sensitivity and PPV of the FP we found in our study, although our results are similar to those previously reported in literature.

In conclusion, our study shows that both the FP and the AFP cannot reliably identify people who will die or develop disability at 5 years, while they are equally effective in identifying people who will not experience these outcomes.

Funding

The authors received no specific funding for this work.