Abstract
Background: older people are at an increased risk of adverse outcomes following attendance at acute hospitals. Screening tools may help identify those most at risk. The objective of this study was to compare the predictive properties of five frailty-rating scales.
Method: this was a secondary analysis of a cohort study involving participants aged 70 years and above attending two acute medical units in the East Midlands, UK. Participants were classified at baseline as frail or non-frail using five different frailty-rating scales. The ability of each scale to predict outcomes at 90 days (mortality, readmissions, institutionalisation, functional decline and a composite adverse outcome) was assessed using area under a receiver-operating characteristic curve (AUC).
Results: six hundred and sixty-seven participants were studied. Frail participants according to all scales were associated with a significant increased risk of mortality [relative risk (RR) range 1.6–3.1], readmission (RR range 1.1–1.6), functional decline (RR range 1.2–2.1) and the composite adverse outcome (RR range 1.2–1.6). However, the predictive properties of the frailty-rating scales were poor, at best, for all outcomes assessed (AUC ranging from 0.44 to 0.69).
Conclusion: frailty-rating scales alone are of limited use in risk stratifying older people being discharged from acute medical units.
Keywords: frailty, risk stratification, acute hospital care, older people
Introduction
The proportion of the UK population aged 65 and older will increase from 17 to 23% by the year 2035 [1], which is expected to lead to an increase in acute hospital care use. Older people accessing acute care are at an increased risk of adverse outcomes [2]. Interventions that improve outcomes for older people will be more cost-effective in selected populations than if just aimed at all older people [3]. For older patients presenting with medical emergencies, risk stratification tools have been used to identify those people at the highest risk of adverse outcomes [4]; however, the predictive properties of such tools are insufficient for clinical practice [5, 6].
An alternative approach to using risk stratification tools is to describe a clinical phenotype of older people who are at a high risk of adverse outcomes. Frailty is a state of loss of resistance to external stressors leading to extreme vulnerability and subsequent decline [7]. In community populations, frailty is associated with an increased risk of functional decline, death, hospitalisation and worsening mobility [8] so it is logical to test frailty scales to assess their predictive properties in acute care to see if they can be used as a method of identifying those at risk.
The aim of this study was to assess the predictive value of frailty-rating scales in older people presenting as medical emergencies.
Method
Study design
This was a secondary analysis of the Acute Medicine Outcome Study (AMOS)—a two-centre, observational cohort study involving older people discharged home from acute medical units (AMUs) within 72 h. Participants were recruited from the AMUs at Queen's Medical Centre, Nottingham, and Leicester Royal Infirmary, Leicester between January 2009 and November 2010. Patients were eligible if they were resident in the hospital catchment area, were ≥70 years and were expected to be discharged home from the AMU within 72 hours. Patients were excluded if they lacked mental capacity, unless a family consultee or the responsible AMU physician agreed. The selection process for inclusion can be seen in Supplementary data available in Age and Ageing online, Appendix S1.
Baseline assessment
Data were collected either from patient records or by direct enquiry, including:
demographics including age, gender, height and weight;
Charlson comorbidity index [9] (a comorbidity score derived from a weighted list of medical conditions);
Geriatric syndromes (defined as the reported presence in the medical notes for that admission of any of the following: fall, reduced mobility, continence disorder, current pressure sore, memory problems or polypharmacy ≥4 prescribed medications);
list of prescribed medications;
Mini Nutritional Assessment [10] (a six item tool which is widely used to identify people at risk of malnutrition);
Folstein Mini-Mental State Examination [11] (MMSE—a standardised measure of global cognitive function);
Barthel ADL index [12] (a 10-item scale assessing basic activities of daily living, with possible scores ranging from 0 to 20);
EuroQoL-5D [13] (EQ5D—a standard measure of quality of life);
General Health Questionnaire-12 (GHQ-12—a 12-item screening tool for identifying minor psychiatric disorder ranging from 0 to 36).
Frailty-rating scales
Five frailty-rating scales were tested, including the models proposed by Fried (CHS model) [8], Ensrud (SOF model) [14], Rothman [15], Ávila-Funes [16] and a frailty index (FI) [17]. Frailty indices count the number of deficits present and describe these as a proportion of all deficits assessed; individuals with an FI of >0.25 are at an increased risk of adverse outcomes [18]. For the full list of variables used to construct the FI, Please see Supplementary data available in Age and Ageing online, Appendix S2. Table 1 shows the original items for each scale, with the adapted items used in this study, and the cut-off score used to categorise participants into frail or non-frail comparisons.
Table 1.
Frailty-rating scales cut-offs, domains, original items and adapted items
| Scale | Domain | Original Item | Adapted Item |
|---|---|---|---|
| CHS model frail = 3/5 | Nutritional status | Weight loss >10 lbs unintentionally in the preceding year | Weight loss >5 kg in preceding year |
| Strength | Grip strength, the lowest 20% scoring positive | Grip strength, the lowest 20% in this population scoring positive | |
| Energy | Self-reported exhaustion | Assessed from the question ‘Do you feel full of energy?’, where an answer of ‘no’ scored positive | |
| Mobility | Gait speed over 15 feet, the lowest 20% scoring positive | Gait speed 2.4 m walk, the lowest 20% scoring positive | |
| Physical activity | Kcals/week, lowest 20% scoring positive | Assessed by questions from EuroQol-5D, the lowest 20% scoring positive
|
|
| SOF model frail = 2/3 | Nutritional status | Weight loss ≥5% between third and fourth examination | Weight loss >5 kg in preceding year |
| Strength | Inability to rise from a chair five times without using his/her arms | Inability to rise from a chair five times without using his/her arms | |
| Energy | Assessed from the question ‘Do you feel full of energy?’ | Assessed from the question ‘Do you feel full of energy’?, where an answer of ‘no’ scored positive | |
| Ávila-Funes's instrument frail = 3/5 | CHS model | See above | See above |
| Cognition | MMSE and Isaacs set test, lowest 25% scoring positive | MMSE, lowest 25% scoring positive in this population | |
| Rothman's instrument frail = 2/4 | Mobility | Gait speed of >10 s to walk back and forth over a 10 foot course | Gait speed 2.4 m walk, the lowest 20% scoring positive |
| Physical activity | Score <64 for men, <52 for women on the physical activity scale for the elderly | Assessed by questions from EuroQol-5D, the lowest 20% scoring positive
|
|
| Nutritional status | Weight loss >10 lbs in past year (intentional and unintentional) | Weight loss >5 kg in preceding year | |
| Cognition | MMSE <24/30 indicating cognitive impairment | MMSE <24/30 indicating cognitive impairment |
Outcomes
The outcomes were assessed at 90 days using hospital records and a postal follow-up questionnaire. Outcomes were mortality, institutionalisation (defined as a new admission to a care home or a move to a new care home as a result of an increased care need), unplanned hospital readmission and decline in function (defined as a decrease of two or more points on Barthel ADL). In addition, a composite outcome was constructed and defined as any of the outcomes of mortality, institutionalisation, unplanned hospital readmission and decline in function, at 90 days.
Statistical analyses
Descriptive statistics were drawn to characterise the baseline population. Relative risks (RRs) were calculated to compare the risk of poor outcomes in frail and non-frail participants. To assess the predictive properties of each scale, a receiver-operating characteristic curve reflecting the full range of values in each scale was plotted with the area under the curve (AUC) calculated for each scale and outcome. The AUC analysis was carried out using the full range of values in each scale—so the AUC reported for the CHS model reflects all five levels of the scale. An area >0.70 was used to show that a scale has an acceptable discriminatory value, 0.60–0.69 as poor and 0.50–0.59 as very poor [1]. A score of 0.50 indicates that the scale has no predictive value. In addition, the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were also calculated for the groups classified as frail versus non-frail using each scale. 95% confidence intervals (CIs) were reported for AUC and RRs. Statistical analyses were performed using Stata version 9.
Results
In total, 669 participants were recruited from two centres (Nottingham and Leicester) between 2009 and 2010. Two participants subsequently withdrew consent and their data could not be used. Of those remaining (667) in the study, 494 (83%) returned at least some outcome questionnaires at 90 days, and service outcomes such as readmissions and length of stay were available for 644 participants (23 withdrew). Demographics and clinical characteristics of the participants are shown in Table 2.
Table 2.
Baseline characteristics of the older participants
| Variable | Patients |
|---|---|
| Age (median, inter-quartile range) | 80 (75, 85) |
| Gender (n, female %) | 386 (57.9) |
| Ethnicity (n, White %) | 647 (97.0) |
| Residence (n, alone %) | 309 (46.3) |
| Charlson rating (n, %) | |
| None (0) | 235 (35.2) |
| Mild (1–2) | 335 (50.2) |
| Moderate (3–4) | 79 (11.8) |
| Severe (≥5) | 18 (2.7) |
| Number of medications (median, IQR) | 6 (4, 9) |
| Number of geriatric syndromes (mean, SD) | 1.3 (±0.8) |
| Cognition (n, %) | |
| Normal (≥25/30) | 523 (78.4) |
| Mild (21–24/30) | 84 (12.6) |
| Moderate (10–20/30) | 50 (7.5) |
| Severe (≤9/30) | 10 (1.5) |
| EQ5D (median, IQR) | 0.69 (0.36, 0.80) |
| Barthel ADL (median, IQR) | 18 (17, 20) |
| General Health Questionnaire (median, IQR) | 11 (8, 15) |
| Prevalence of frailty (n/N frail, %) | |
| CHS model | 110/613 (17.9) |
| SOF model | 427/643 (66.4) |
| Ávila-Funes instrument | 142/571 (24.9) |
| Rothman instrument | 142/589 (24.1) |
| Frailty index | 173/559 (30.9) |
IQR, inter-quartile range; SD, standard deviation.
Outcomes
At 90 days following the index visit to AMU 36.7% (245/667) of participants experienced one or more adverse outcome. Overall, the risk of mortality was 5.1% (34/667), the risk of institutionalisation was 0.9% (6/667) and the risk of readmission was 26.7% (172/644). Of the participants who completed the follow-up questionnaire (494/667), the risk of functional decline was 20.2% (97/480). The predictive properties of the different frailty measures are shown in Table 3.
Table 3.
Predictive properties of frailty-rating scales for 90-day outcomes
| Scale | Cut-off | Sensitivity | Specificity | PPV | NPV | AUC (95% CI) (full scale) | RR (95% CI) |
|---|---|---|---|---|---|---|---|
| Mortality (n = 667) | |||||||
| CHS model | ≥3/5 | 25.8 | 82.5 | 7.3 | 95.4 | 0.61 (0.52, 0.70) | 1.6 (0.9, 3.0) |
| SOF model | ≥2/3 | 76.5 | 34.2 | 6.1 | 96.3 | 0.59 (0.50, 0.67) | 1.6 (0.9, 3.1) |
| Ávila-Funes | ≥3/6 | 50.0 | 76.5 | 10.6 | 96.5 | 0.68 (0.59, 0.76) | 3.0 (2.0, 4.5) |
| Rothman | ≥2/4 | 50.0 | 77.3 | 10.6 | 96.6 | 0.67 (0.57, 0.76) | 3.1 (2.1, 4.6) |
| FI | >0.25/1.00 | 48.3 | 70.0 | 8.1 | 96.1 | 0.69 (0.60, 0.78) | 2.1 (1.4, 3.1) |
| Readmissions (n = 644) | |||||||
| CHS model | ≥3/5 | 22.9 | 84.6 | 35.0 | 75.3 | 0.52 (0.47, 0.57) | 1.4 (1.0, 2.0) |
| SOF model | ≥2/3 | 68.5 | 35.1 | 28.1 | 75.0 | 0.53 (0.48, 0.58) | 1.3 (1.0, 1.7) |
| Ávila-Funes | ≥3/6 | 31.5 | 78.5 | 34.6 | 76.1 | 0.55 (0.50, 0.60) | 1.4 (1.1, 2.0) |
| Rothman | ≥2/4 | 33.6 | 80.0 | 37.3 | 77.2 | 0.53 (0.48, 0.59) | 1.6 (1.2, 2.2) |
| FI | >0.25/1.00 | 40.7 | 72.5 | 35.1 | 76.9 | 0.57 (0.52, 0.63) | 1.5 (1.2, 2.0) |
| Institutionalisation (n = 667) | |||||||
| CHS model | ≥3/5 | 0.0 | 81.9 | 0.0 | 99.0 | 0.57 (0.42, 0.72) | |
| SOF model | ≥2/3 | 60.0 | 33.5 | 0.7 | 99.1 | 0.44 (0.25, 0.63) | Null |
| Ávila-Funes | ≥3/6 | 0.0 | 74.9 | 0.0 | 98.8 | 0.50 (0.36, 0.64) | |
| Rothman | ≥2/4 | 0.0 | 75.7 | 0.0 | 98.9 | 0.45 (0.30, 0.60) | |
| FI | >0.25/1.00 | 40.0 | 69.1 | 1.2 | 99.2 | 0.55 (0.25, 0.85) | 1.5 (0.5, 4.4) |
| Functional decline (n = 480) | |||||||
| CHS model | ≥3/5 | 22.8 | 88.5 | 33.9 | 81.6 | 0.55 (0.49, 0.62) | 1.8 (1.1, 2.9) |
| SOF model | ≥2/3 | 69.1 | 36.4 | 21.5 | 82.4 | 0.56 (0.49, 0.62) | 1.2 (0.8, 1.7) |
| Ávila-Funes | ≥3/6 | 36.7 | 75.7 | 26.9 | 83.1 | 0.59 (0.52, 0.66) | 1.9 (1.2, 2.8) |
| Rothman | ≥2/4 | 29.3 | 84.6 | 32.0 | 82.8 | 0.59 (0.52, 0.66) | 2.1 (1.5, 3.2) |
| FI | >0.25/1.00 | 33.7 | 84.5 | 35.4 | 83.5 | 0.57 (0.50, 0.64) | 1.6 (1.1, 2.2) |
| Composite (n = 667) | |||||||
| CHS model | ≥3/5 | 22.1 | 84.5 | 45.5 | 65.0 | 0.53 (0.48, 0.57) | 1.3 (0.9, 1.8) |
| SOF model | ≥2/3 | 70.2 | 35.8 | 39.1 | 67.1 | 0.54 (0.50, 0.59) | 1.2 (0.9, 1.5) |
| Ávila-Funes | ≥3/6 | 32.2 | 79.3 | 47.2 | 67.1 | 0.57 (0.52, 0.62) | 1.4 (1.1, 1.9) |
| Rothman | ≥2/4 | 33.2 | 81.1 | 50.0 | 68.0 | 0.55 (0.50, 0.59) | 1.6 (1.2, 2.1) |
| FI | >0.25/1.00 | 39.5 | 74.0 | 46.8 | 67.9 | 0.59 (0.54, 0.64) | 1.5 (1.1, 1.9) |
PPV, positive predictive value; NPV, negative predictive value; AUC, area under the curve; RR, relative risk.
–, These are the scores for the dichotomised scale.
Four of the five frailty models significantly predicted death, but only poorly using AUC analysis. Three of the five models predicted readmissions, again only very poorly. The number of people who were institutionalised was too low for meaningful estimations. Four of the five significantly predicted functional decline, but again only very poorly. All five predicted the composite outcome, but again only very poorly.
Discussion
This study assessed the role of frailty-rating scales in predicting adverse health outcomes in AMUs in the UK. At 90 days, over a third of participants experienced one or more adverse outcome. The most common of these was readmission (26.7%) followed by functional decline (20.2%).
Although most of the scales performed better than chance in predicting a range of poor outcomes, none of them performed adequately, and most performed either poorly or very poorly. Those scales that included cognition (Ávila-Funes, Rothman and FI) had a better predictive accuracy for all outcomes bar institutionalisation.
Strengths of this study include a large sample size, with detailed individual health and service outcomes collated from two sites. Owing to initial constraints placed on the study by the ethical committee, we were unable to include a larger proportion of participants with cognitive impairment, so study population may not be representative of all older patients accessing AMUs. If we had included more higher-risk patients, this would have increased sensitivity and reduced specificity, but the effect on the AUC is uncertain. A sizeable proportion (17%) of those recruited declined ascertainment of clinical outcomes despite the protocol offering postal, telephone or face-to-face follow-up, and these patients tended to have more adverse characteristics than those in whom clinical outcomes were ascertained. Despite these two factors, the incidence of adverse outcomes was much higher than expected, which will have increased the power of the study. Thus, we believe that the estimates of the frailty scales to predict such adverse outcomes are broadly correct. It is unlikely that larger study would contribute significantly to our understanding of the role of these tools as the CI for each of the AUC scores are incompatible with good or excellent predictive ability.
A possible limitation of this study is that we did not use the exact published tool but adapted them as described so that data collected in this cohort study could be used. To minimise this problem, the scales were adapted as closely as possible to the published scales, reflecting the common domains used in frailty research. We believe that this process does not explain their poor performance. One final limitation is that the frailty scales might be better at predicting longer-term outcomes rather than these short-term ones.
The weak predictive properties of the frailty scales are consistent with the findings from other studies [6, 19], adding further doubt to the usefulness of risk stratification alone in the acute care context. The frailty level of a patient is one of many factors that contribute to a poor outcome, and currently, the presence of frailty as measured by these scales alone is not sufficient to predict outcomes.
Despite the poor predictive values, the scales are still better than chance, and should not be fully dismissed. All scales had a high NPV for mortality, institutionalisation and functional decline, indicating that those who are not classified as frail would not be likely to have the adverse outcome. The RR scores for nearly all of the frailty scales demonstrated that frailty was able to identify people at a higher risk of adverse events. This may provide an opportunity to use frailty-rating scales as a tool for exclusion considering the majority of the population was classified as non-frail and are unlikely to have an adverse outcome. An exception to this is the SOF model which classified the majority as frail, possibly related to the large proportion (78%) of people who could not rise from a chair unaided—none of the other frailty scales used this item. It is unlikely that available frailty scales alone will have adequate predictive ability to be used in standard day-to-day clinical management. Instead, further work is needed to identify a battery of indicators, including clinical judgement that might have a good predictive value when combined.
Key points.
Frail older people are at an increased risk of adverse outcomes compared with those who are non-frail
Frailty-rating scales have limited clinical application due to their poor predictive properties in identifying those at risk of adverse outcomes
Ethical approval
Ethical approval was obtained from Southampton REC (reference 08/H0502/139) with research governance approvals being obtained in each centre.
Funding
The study was funded by the National Institute for Health Research (NIHR) and the University of Leicester. This article presents independent research funded by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research funding scheme (RP-PG-0407-10147). The views expressed in this article are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.
Supplementary data
Supplementary data mentioned in the text is available to subscribers in Age and Ageing online.
Acknowledgements
The authors would like to thank the patients and carers who participated in this research, and the Trent and the Leicestershire, Northamptonshire and Rutland Comprehensive Local Research Networks for supporting this research.
References
- 1. Office For National Statistics. Older people's day 2011 Available at http://www.ons.gov.uk/ons/rel/mortality-ageing/focus-on-older-people/older-people-s-day-2011/stb-opd-2011.html#tab-Older-people-in-the-UK. (29 January 2013, date last accessed )
- 2.Aminzadeh F, Dalziel WB. Older adults in the emergency department: a systematic review of patterns of use, adverse outcomes, and effectiveness of interventions. Ann Emerg Med. 2002;39:238–47. doi: 10.1067/mem.2002.121523. [DOI] [PubMed] [Google Scholar]
- 3.Ellis G, Whitehead MA, Robinson D, O'Neill D, Langhorne P. Comprehensive geriatric assessment for older adults admitted to hospital: meta-analysis of randomised controlled trials. BMJ. 2011;343:6553. doi: 10.1136/bmj.d6553. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Dendukuri N, McCusker J, Belzile E. The identification of seniors at risk screening tool: further evidence of concurrent and predictive validity. J Am Geriatr Soc. 2004;52:290–6. doi: 10.1111/j.1532-5415.2004.52073.x. [DOI] [PubMed] [Google Scholar]
- 5.Buurman BM, van den Berg W, Korevaar JC, Milisen K, de Haan RJ, de Rooij SE. Risk for poor outcomes in older patients discharged from an emergency department: feasibility of four screening instruments. Eur J Emerg Med. 2011;18:215–20. doi: 10.1097/MEJ.0b013e328344597e. [DOI] [PubMed] [Google Scholar]
- 6.Graf CE, Giannelli SV, Herrmann FR, et al. Can we improve the detection of old patients at higher risk for readmission after an emergency department visit? J Am Geriatr Soc. 2012;60:1372–3. doi: 10.1111/j.1532-5415.2012.04026.x. [DOI] [PubMed] [Google Scholar]
- 7.Fried LP, Ferrucci L, Darer J, Williamson JD, Anderson G. Untangling the concepts of disability, frailty, and comorbidity: implications for improved targeting and care. J Gerontol A Biol Sci Med Sci. 2004;59:255–63. doi: 10.1093/gerona/59.3.m255. [DOI] [PubMed] [Google Scholar]
- 8.Fried LP, Tangen CM, Walston J, et al. Frailty in older adults: evidence for a phenotype. J Gerontol A Biol Sci Med Sci. 2001;56:M146–56. doi: 10.1093/gerona/56.3.m146. [DOI] [PubMed] [Google Scholar]
- 9.Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373–8. doi: 10.1016/0021-9681(87)90171-8. [DOI] [PubMed] [Google Scholar]
- 10.Rubenstein LZ, Harker JO, Salva A, Guigoz Y, Vellas B. Screening for undernutrition in geriatric practice: developing the short-form mini-nutritional assessment (MNA-SF) J Gerontol A Biol Sci Med Sci. 2001;56:M366–72. doi: 10.1093/gerona/56.6.m366. [DOI] [PubMed] [Google Scholar]
- 11.Folstein MF, Folstein SE, McHugh PR. ‘Mini-mental state’. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12:189–98. doi: 10.1016/0022-3956(75)90026-6. [DOI] [PubMed] [Google Scholar]
- 12.Wade DT, Collin C. The barthel ADL index: a standard measure of physical disability? Int Disabil Stud. 1988;10:64–7. doi: 10.3109/09638288809164105. [DOI] [PubMed] [Google Scholar]
- 13.Brazier JE, Walters SJ, Nicholl JP, Kohler B. Using the SF-36 and euroqol on an elderly population. Qual Life Res. 1996;5:195–204. doi: 10.1007/BF00434741. [DOI] [PubMed] [Google Scholar]
- 14.Ensrud KE, Ewing SK, Taylor BC, et al. Comparison of 2 frailty indexes for prediction of falls, disability, fractures, and death in older women. Arch Intern Med. 2008;168:382–9. doi: 10.1001/archinternmed.2007.113. [DOI] [PubMed] [Google Scholar]
- 15.Rothman MD, Leo-Summers L, Gill TM. Prognostic significance of potential frailty criteria. J Am Geriatr Soc. 2008;56:2211–16. doi: 10.1111/j.1532-5415.2008.02008.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Avila-Funes JA, Amieva H, Barberger-Gateau P, et al. Cognitive impairment improves the predictive validity of the phenotype of frailty for adverse health outcomes: the three-city study. J Am Geriatr Soc. 2009;57:453–61. doi: 10.1111/j.1532-5415.2008.02136.x. [DOI] [PubMed] [Google Scholar]
- 17.Searle SD, Mitnitski A, Gahbauer EA, Gill TM, Rockwood K. A standard procedure for creating a frailty index. BMC Geriatr. 2008;8:24. doi: 10.1186/1471-2318-8-24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Rockwood K, Andrew M, Mitnitski A. A comparison of two approaches to measuring frailty in elderly people. J Gerontol A Biol Sci Med Sci. 2007;62:738–43. doi: 10.1093/gerona/62.7.738. [DOI] [PubMed] [Google Scholar]
- 19.Kansagara D, Englander H, Salanitro A, et al. Risk prediction models for hospital readmission: a systematic review. JAMA. 2011;306:1688–9. doi: 10.1001/jama.2011.1515. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.