Discriminative value of frailty screening instruments in end-stage renal disease

Abstract

Background

Numerous frailty screening instruments are available, but their applicability for identifying frailty in patients with end-stage renal disease (ESRD) is unknown. We aimed to investigate the diagnostic accuracy of three instruments used for frailty screening in an ESRD population.

Methods

The study was conducted in 2013 in a teaching hospital in The Netherlands and included patients receiving haemodialysis, peritoneal dialysis and pre-dialysis care. We determined the sensitivity and specificity of three screening instruments: the Groningen Frailty Indicator (GFI), the Identification of Seniors at Risk–Hospitalized Patients (ISAR-HP) and the Veiligheidsmanagementsysteem (VMS), which is a safety management system for vulnerable elderly patients. The Frailty Index was the gold standard used.

Results

The prevalence of frailty was 37% in a total of 95 participants with ESRD [mean age 65.2 years (SD 12.0), 57% male]. Frailty prevalence in participants ≥65 years of age and <65 years of age was 44% and 28%, respectively (P = 0.11). Sensitivity and specificity for frailty of the GFI were 89% and 57%, respectively; ISAR-HP 83% and 77%, respectively; and VMS 77% and 67%, respectively.

Conclusions

Although the GFI showed the highest sensitivity, it is not yet possible to propose a firm choice for one of these screening instruments or specific items due to the small scale of the study. Since there is a high prevalence of frailty in ESRD patients, translation and testing of the effectiveness of screening using the GFI in the prognostication and prevention of development or deterioration of frailty in this population should be the next step.

Introduction

With a growing ageing population and improved medical care, there is an increasing number of elderly patients with chronic kidney disease (CKD) [1, 2], many of whom progress to end-stage renal disease (ESRD) and become dialysis dependent [1, 2]. More than 5000 patients in The Netherlands and >500 000 patients in the United States undergo dialysis treatment, of whom more than half are ≥65 years of age [2]. The prognosis of older patients with ESRD after starting dialysis is poor; after dialysis initiation, mortality is >35% in patients >70 years, and the mortality rate increases to 50% in patients >80 years [3]. Additionally, older patients are at high risk of impaired functional status and comorbidity [3]. However, prognosis varies substantially among older patients. Therefore, it is important to identify those who are at high risk of mortality and functional decline and those who will benefit from dialysis treatment.

The concept of frailty—a state of low homeostatic reserve leading to high vulnerability for sudden adverse health changes—has recently emerged as a potential predictor of prognosis in the ESRD population [4]. The prevalence of frailty is high in the ESRD population. In addition, frail patients undergoing haemodialysis had 2.6 times higher risk of mortality [95% confidence interval (CI) 1.04–6.49] and 1.4 times higher risk of hospitalization (95% CI 1.00–2.03) compared with non-frail patients, independent of age, gender, comorbidity and disability [4].

A comprehensive geriatric assessment (CGA) is the gold standard to define frailty in patients, but consensus on the exact operational definition of frailty is still lacking. In research settings, frailty is mostly operationalized by the Fried phenotype or the Frailty Index (FI) [5, 6]. Performing a CGA is time-consuming and requires clinical expertise from the investigator and endurance from the patient. Therefore, several screening instruments for frailty have been developed, such as the the Groningen Frailty Indicator (GFI), the Identification of Seniors at Risk–Hospitalized Patients (ISAR-HP) and the Veiligheidsmanagementsysteem (VMS; Dutch safety management system) [7–9]. These instruments aim to select frail patients who would benefit most from CGA by assessing their risk for frailty. They have been primarily developed for use in clinical settings and general populations, but they are also used in research settings and specific patient populations. However, none of these instruments have been specifically developed for or validated for use in an ESRD population [10, 11].

Despite the high prevalence of frailty and the associated risk of adverse outcomes, it would be cost effective to identify those patients at high risk of adverse outcomes, particularly in the presence of time and resources limitations. Therefore, the aim of this study was to determine the diagnostic value, specifically the sensitivity, of three screening instruments for identifying frailty in an ESRD population.

Materials and methods

Study population

This prospective study was conducted at a single dialysis centre in a secondary teaching hospital in The Netherlands between September 2013 and December 2013. Patients >18 years of age receiving chronic haemodialysis, chronic peritoneal dialysis, or pre-dialysis care were included in the study. The pre-dialysis care group consisted of (i) patients with an estimated glomerular filtration rate (eGFR) <20 mL/min/1.73 m² but who did not yet need renal replacement therapy and (ii) patients who were expected to be dependent on renal replacement therapy within a considerable time frame due to a recent decline in kidney function. The study was approved by the Medical Ethics Committee (METC) of the Academic Medical Center Amsterdam (reference number W13_164 # 13.17.0209). Written informed consent was obtained from all participants.

Data collection

At enrolment, trained research staff collected information from medical charts, including comorbidities as measured by the Charlson comorbidity index [12]. The eGFR was calculated using the four-variable Modification of Diet in Renal Disease (MDRD) formula [13]. Participants were asked to complete a questionnaire and they performed a set of function tests supervised by trained research staff (Supplementary data, Table S1). Maximal grip strength (average of three measurements) was measured in the dominant hand using a type 5030J1 Jamar hydraulic dynamometer. Walking speed (normal and rapid pace) was measured as the fastest time of two measurements [14]. Non-recordable grip strength or inability to walk were scored as positive items for all frailty screening instruments.

Frailty assessment

Frailty status was established by a medical student according to the FI [6, 14]. We used 38 variables, consisting of physical, psychological, social and cognitive items but excluding shoulder strength and peak flow measurement, and used cut-off points as suggested by Searle et al. [14] (Supplementary data, Table S1). The FI represented the total score of positive items as a proportion of all items measured. An FI score <0.25 was considered as non-frail and an FI score of ≥0.25 as frail [14].

The discriminative value was assessed for three different frailty screening instruments by comparing with the FI as the gold standard. The GFI consists of 15 items of self-reported limitations grouped in four domains: physical, cognitive, psychological and social. A score of ≥4 is considered as frail (Supplementary data, Table S2) [7]. The VMS for the vulnerable elderly is a Dutch risk assessment tool to identify elderly patients at risk of functional decline both during and after hospital admission [8]. Based on expert opinion, the VMS for the vulnerable elderly consists of 13 risk-related items grouped in four domains: delirium, falls, malnutrition and functional impairment. A positive score in one or more domain(s) is considered as high risk for frailty (Supplementary data, Table S2). The ISAR-HP [9] is based on the ISAR for the Emergency Department. It is a simple four-item risk assessment instrument for functional decline both during and after hospital admission that was developed and validated in adults ≥aged 65 years of age [9, 15, 16]. A score of ≥2 was considered as high risk for frailty (Supplementary data, Table S2). The VMS and ISAR-HP were originally developed for screening the risk for functional decline; they assess domains that are associated with the incidence of frailty and highly correlate with functional decline [17]. In our study, we used both screening instruments for assessing frailty because they are commonly used for this purpose in clinical practice.

Statistical analysis

Descriptive statistics were given for all baseline demographic and clinical data. Baseline characteristics were compared between frail and non-frail patients. Normally and non-normally distributed continuous variables and nominal variables were tested using the Student's t-test, Mann–Whitney U-test or chi-square test. Data from participants were excluded from the analysis of an instrument if they had missing values for >20% per instrument [11]. Sensitivity, specificity and positive and negative predictive values of the three frailty screening instruments were calculated, with the FI as the gold standard. The statistical significance level was set at 0.05. All analyses were performed using Statistical Package for Social Sciences (SPSS) software, version 20.

Results

Study population

Of a total of 144 patients, 95 (66%) participated in this study. Individuals in the included and non-responders groups were similar with respect to age, treatment modality and Charlson comorbidity index. In the included group, there was a lower proportion of females compared with the non-responders group (43% versus 68%, respectively; P = 0.004). The mean age of the 95 participants was 65.2 years (SD 12.0; range 27–88 years). Forty-two participants (44%) were undergoing haemodialysis and 14 (15%) peritoneal dialysis and 39 (41%) were receiving pre-dialysis care.

Frailty

Of the total study population, 37% were considered frail according to the FI (44% of participants >65 years of age and 28% in the younger group) (P = 0.11). As shown in Table1, of the frail people more patients were female (60%) than of the non frail (33%) (P = 0.01). No difference was found in age between frail [mean 66.6 years (SD 13.5); P = 0.39] and non-frail individuals [mean 64.4 years (SD 11.1)]. Correction for gender or treatment modality did not alter this result. Frail participants had a (1 point) higher Charlson comorbidity index score {median 4 [interquartile range (IQR) 3–6]} compared with non-frail participants [median 3 (IQR 2–5); P = 0.02]. Treatment modality did not differ between frail and non-frail participants. Table 2 shows there was no difference observed in the prevalence of frailty between dialysis and pre-dialysis patients.

Table 1.

Patient characteristics: frailty status, according to the Frailty Index

Characteristic	Non-frail (n = 60)	Frail (n = 35)	P-value
Female, %	20 (33.3)	21 (60.0)	0.01
Age, years	64.4 (± 11.1)	66.6 (± 13.5)	0.39
Race, Caucasian, %	58 (96.7)	33 (94.3)	0.42
Treatment modality, %
HD	25 (41.7)	17 (48.6)	0.80
PD	9 (14.5)	5 (15.2)
Pre-dialysis	26 (43.3)	13 (37.1)
Body mass index	27.1 (24.0–29.8)	27.0 (25.0–29.4)	0.81
Laboratory
eGFR, mL/min/1.73 m2 (non-dialysis, n = 39)	14.0 (13.0–16.0)	16.0 (13.5–20.5)	0.10
Albumin, g/dL	3.49 (± 0.36)	3.33 (± 0.53)	0.10
Haemoglobin, g/dL	11.47 (± 1.21)	10.94 (± 1.55)	0.07
Urea nitrogen, mg/dL	61.64 (50.40–69.44)	52.23 (39.87–74.69)	0.12
Time on dialysis, months (n = 56)	14.0 (8.8–43.3)	8.0 (4.5–18.8)	0.06
No dialysis, %	26 (43.3)	13 (37.1)	0.21
<12 months	14 (23.3)	14 (40.0)
≥12 months	20 (33.3)	8 (22.9)
MMSE (n = 87)	28.5 (27.0–29.0)	27.0 (26.0–29.0)	0.15
Number of hospitalizations in previous year	1 (0–2)	1 (0–3)	0.45
Number of medications	9.0 (8.0–13.8)	12.0 (10.0–14.0)	0.03
Charlson comorbidity index score	3 (2–5)	4 (3–6)	0.02

Data are presented as: number (%), mean (± SD) or median (25–75% IQR).

Conversion factors for units: serum albumin in g/dL to g/L, ×10; serum haemoglobin in g/dL to mmol/L, ×0.6206; urea nitrogen in mg/dL to mmol/L, ×0.357.

MMSE, Mini Mental State Examination.

Table 2.

Comparison of frailty parameters in dialysis and pre-dialysis patients

Characteristic	Pre-dialysis (n = 39)	Dialysis (HD and PD) (n = 56)	P-value
Frail by Frailty Index	13 (33.3)	22 (39.3)	0.55
Frail by GFI	23 (59.0)	34 (60.7)	0.87
Frail by VMS	18 (46.2)	29 (51.8)	0.59
Frail by ISAR-HP	18 (46.2)	25 (44.6)	0.88

Data are presented as number (%).

Screening instruments

Table 3 shows the sensitivity and specificity of the three screening instruments, with the FI as the gold standard. Of the three instruments, the GFI had the highest sensitivity (89%) and the lowest specificity (57%). In addition, the GFI had the highest proportion of participants screened as frail (i.e. 57 participants), with the lowest positive predictive value of 54.4%, whereas it also showed the highest negative predictive value of 89.5%. The ISAR-HP had the highest positive predictive value of 67.4% in this population compared with the other two instruments.

Table 3.

Comparison of screening instruments using the Frailty Index

	Frail (n = 35)		Non-frail (n = 60)		Sensitivity, %	Specificity, %	AUC (95% CI)	Positive predictive value, %	Negative predictive value, %
	Frail	Non-frail	Frail	Non-frail
Frailty Index
GFI	31 (88.6)	4 (11.4)	26 (43.3)	34 (56.7)	89	57	0.83 (0.74–0.91)	54.4	89.5
VMS	27 (77.1)	8 (22.9)	20 (33.3)	40 (66.7)	77	67	0.76 (0.65–0.86)	57.4	83.3
ISAR-HP	29 (82.9)	6 (17.1)	14 (23.3)	46 (76.7)	83	77	0.89 (0.82–0.95)	67.4	88.5

The Frailty Index [14] is used as the gold standard.

Data are presented as number (%).

AUC, area under the curve.

Discussion

This study showed that different short questionnaire–based screening instruments, i.e. the GFI, ISAR-HP and VMS, can all be applied to the ESRD population to screen for frailty in both young and older ESRD patients. Of these three screening instruments, the GFI showed the highest sensitivity and negative predictive value for screening frailty in dialysis and pre-dialysis patients, with the FI as the gold standard. In addition, the ISAR-HP also showed comparable performance to that of the GFI, with slightly lower negative predictive value, and had the highest positive predictive value of all three instruments.

In this study, which included adults undergoing haemodialysis, peritoneal dialysis or pre-dialysis care, the prevalence of frailty exceeded one-third. The prevalence of frailty was high in young, as well as older, ESRD patients, and no association between age and frailty was found in this population. These findings suggest that screening for frailty should be performed not only in ESRD patients ≥65 years of age, but also in all younger ESRD patients, given that more than a quarter of patients <65 years of age in this study were identified as frail.

The GFI is an effective tool for the identification of those who are at high risk for frailty and who may benefit from a CGA. The test is easy to administer, and our study showed that it has strengths as a screening tool for frailty in an ESRD population, including high sensitivity. A limitation for its use is that the instrument is only available in Dutch. Also, the high sensitivity gave rise to a markedly high proportion of false-positive results (43%). A CGA would help identify any underlying medical and geriatric conditions in detail and provide valuable insight into a patient's ability to tolerate dialysis, thereby guiding treatment decisions. However, they feel ‘good in themselves’ and therefore do not consider any need to undergo a CGA or they fear a possible diagnosis of cognitive impairment that could have consequences on their daily living. It would be interesting to explore why a proportion of ESRD patients in our study declined participation in this straightforward frailty assessment and whether this has implications for the implementation of a CGA before the initiation of dialysis. The selection could have led to either an underestimation of the prevalence of frailty (patients feared additional diagnoses) or an overestimation (the healthiest subjects were not motivated for screening). Still, it is important not to overlook any underlying clinical pathologies that can be identified using a CGA, such as cognitive impairment, because of their possible interference with treatment for renal failure.

Our results should be interpreted with some caution. In two studies on community-dwelling elderly patients directly comparing the GFI with the FI, it was found that these two frailty screening instruments have a moderate overlap, but that each identifies distinct groups of individuals as frail [11, 17]. Findings from one of the two studies indicated that the GFI has a lower sensitivity for identifying frailty in community-dwelling elderly individuals [17] than in our dialysis population. This may be due to the lower prevalence of comorbidities in the general population.

Two other screening instruments—the ISAR-HP and the VMS—were originally developed to screen for the risk of functional decline in hospitalized patients. Since functional decline and disability are strongly associated with frailty, we explored whether both screening instruments were able to discriminate between frail and non-frail patients. Both instruments showed moderate sensitivity and specificity in screening for frailty, possibly due to a lack of their applicability in an outpatient setting. The ISAR-HP is the quickest to perform, which is useful when faced with limited time and resource availability [9]. The VMS is less able to identify ESRD patients who are at high risk for frailty, probably because it does not take into account the presence of any comorbidity [8].

Although research on frailty in ESRD patients is limited, screening and diagnostic instruments for frailty are increasingly used in daily clinical practice. Evaluation of the discriminative values of these instruments, as for the instruments assessed in this study, has been hampered by the lack of a gold standard in the scientific literature, which therefore emphasizes the need for a consensus on the definition of frailty. The strengths of this study included measurements of a validated construct of frailty and screening instruments in a diverse Dutch ESRD population. Additionally, the fact that data collection from assessments using multiple screening instruments took place at the same time by the same person, thus ensuring a high degree of reliable comparability within a population. Due to the single-centre study design including 95 participants, limitations in the statistical power to detect small subgroup effects and in the generalizability for the general dialysis population have to be taken into account. Also, information on residual renal function related to mortality, and possibly associated with frailty, was lacking.

Practical application

With a growing population of elderly dialysis and pre-dialysis patients, frailty will become an important aspect to consider in the clinical care of ESRD patients. This study showed that different short questionnaire–based screening instruments can be used to screen for frailty in both young and older ESRD patients. The instruments contribute to the identification of those who are frail and may benefit from a CGA. Of the three screening instruments assessed, the GFI demonstrated the best discriminative value for frailty in the ESRD population, followed by the ISAR-HP. Before implementation of any of the screening instruments on a large scale, they should be validated first in an independent population. Based on the small scale of this study, it is not yet possible to propose a firm choice for one of these instruments or specific items in the screening for frailty. Ideally, in the future, a choice for one specific screening instrument will be made by international consensus, based on multiple studies. Additionally, following a positive screening for frailty, the effectiveness of a CGA on the prognostication and prevention of development or deterioration of frailty should be investigated, e.g. by comparative effectiveness research.

Supplementary data

Supplementary data are available online at http://ndt.oxfordjournals.org.

Conflict of interest statement

None declared. The results presented in this paper have not been published previously in whole or in part, except in abstract form.