Development of a Claims-Based Frailty Indicator Anchored to a Well-Established Frailty Phenotype

Abstract

Background

Fried et al described a frailty phenotype measured in the Cardiovascular Health Study (CHS). This phenotype is manifest when 3 or more of the following are present: low grip strength, low energy, slowed waking speed, low physical activity, or unintentional weight loss. We sought to approximate frailty phenotype using only administrative claims data in order to enable frailty to be assessed without physical performance measures.

Study Design

We used the CHS cohort data linked to participants Medicare claims. The reference standard was the frailty phenotype measured at visits 5 and 9. With penalized logistic regression, we developed a parsimonious index for predicting the frailty phenotype using a linear combination of diagnoses, operationalized with claims data. We assessed the predictive validity of frailty index by examining how well it predicted common aging-related outcomes including hospitalization, disability and death.

Results

There were 4,454 CHS participants from 4 clinical sites. 84% were white, 58% were women and their mean age was 72 years at enrollment. Approximately 11% of the cohort was frail. The model had an area under the ROC curve of 0.75 to concurrently predict a frailty phenotype. This Claims-based Frailty Index (CFI) significantly predicted death [Odds-Ratio 1.84], time to death [Hazard-Ratio 1.71], number of hospital admissions [Incidence Rate-Ratio 1.74], and nursing home admission [OR 1.47] in models adjusted for age and sex.

Conclusions

Claims data alone can be used to classify individuals as frail and non-frail. The CFI might be used in research with large datasets for confounding adjustment or risk prediction. The indicator might also be used for emergency preparedness for identification of regions enriched with frail individuals.

INTRODUCTION

As demonstrated by Fried and colleagues more than a decade ago, there is a measurable frailty phenotype. (1) Affected individuals can be reproducibly identified with standardized measures that can be used in clinical and research settings. The five measures that define the phenotype are unintentional weight loss of 10 pounds in the past year, self-reported exhaustion, weakness as measured with grip strength, slow walking speed, and low physical activity. This phenotype has been validated as predictive of mortality and disability.(2–4) Individuals with this phenotype, however, cannot be easily identified in data that do not have these specialized, although simple, performance measures. While these measures may be frequently generated in research and in geriatric practice settings, they are not routinely measured in most clinical encounters.

It would be broadly useful to identify individuals with a frailty phenotype using only administrative data. Given the ubiquity of claims data, an indicator of frailty with good positive and negative predictive values may be widely used as an exposure variable to understand the health outcomes of individuals with frailty. (5) Additionally, it may be used as an outcome variable for evaluating the impact of interventions either designed to prevent or delay frailty, or to assess frailty as an unwanted outcome of interventions such as hospitalization or surgery. A claims-based index may also be used as an effect modifier in investigation of treatment effectiveness where there is expected to be differences in treatment-response due to frailty. A claims-based index may also be valuable for health services delivery planning or emergency disaster preparedness, where identification of high concentrations of frail individuals may be essential for public safety. With these as motivators, we aimed to develop a tool, the Claims-based Frailty Indicator (CFI), that can be generated easily with claims data alone, and that has good operating characteristics. We considered the Fried frailty phenotype to be the reference standard and developed a parsimonious model to approximate the frailty phenotype and thus identify frail individuals using claims data alone.

METHODS

The Johns Hopkins Institutional Review Board approved the conduct of this study, which also received approval from the Cardiovascular Health Study investigators and from the Center for Medicare and Medicaid Services.

Data

The data were from the Cardiovascular Health Study (CHS) that had been previously linked to Medicare claims. (6) The CHS is a population-based longitudinal cohort that recruited 5201 older people, from four U.S. communities, beginning in 1989. An additional 687 African Americans were recruited later. Those eligible included all persons living in the household of each individual sampled from the Health Care Financing Administration sampling frame, who were 65 years or older at the time of examination, were noninstitutionalized, were expected to remain in the area for the next three years, and were able to give informed consent and did not require a proxy respondent at baseline. Potentially eligible individuals who were wheelchair-bound in the home at baseline, receiving hospice treatment, radiation therapy or chemotherapy for cancer were excluded. Cohort participants were seen for annual examinations through 1999, and then followed with phone calls every six months subsequently. This dataset includes Medicare claims data for the enrolled participants from 1991 through 2009. For inclusion in our study population, the participants needed to have continuous enrollment in Medicare Parts A and B during the six-month windows from which we used their claims data for construction of the CFI.

Frailty Phenotype

In the Cardiovascular Health study, Fried et al defined a phenotypic measure of frailty consisting of 5 criteria: slow walking, weak grip strength, low physical activity, exhaustion, and weight loss.(1) An individual is said to be frail if 3 or more of this criteria are satisfied, prefrail if 1 or 2 criteria are met, and nonfrail if none of the criteria are met. This frailty phenotype was rigorously validated by Bandeen-Roche et al,(7) and it is the most widely used instrument for assessing frailty.(8) We considered both the pre-frail and non-frail cohort members as not frail for this study.

Candidate Variable Selection

To identify relevant candidate variables for inclusion in a model, we searched Pubmed and Web of Science for relevant published literature. A combination of subject headings and key words were used including: “claims based frailty measurement” “frailty” “frailty variables” “frailty index” “phenotype frailty indicators” “definition of frailty.” In addition, references cited in the articles were scanned to identify other relevant articles not found by the initial search.

We extracted lists of clinical conditions that had been used in the identified articles for classifying individuals as frail or disabled, either with claims data or with electronic medical records. We reviewed the list to assess the feasibility of operationalizing each condition with claims data alone. The list was reviewed by clinical experts in geriatrics who made recommendations about categorizing the variables by their proximity to the hypothesized underlying feature of frailty (i.e. inflammation), and then by the consequences of frailty (e.g. falls), and then other conditions of aging that are probably not frailty-related (e.g. cataracts). To these, we added all of the variables that have been classified by the Agency for Healthcare Research and Quality with their Clinical Classifications Software (CCS) for use with the International Classification of Disease (ICD-9-CM) codes that had not already identified by the process above. (9) We used the 268 single level diagnosis labels from the CCS. For the clinical conditions identified in the literature that did not have a CCS diagnosis level, a clinician (JBS) identified the relevant ICD-9 codes using Flash Code™ medical coding software (Flash Code Solutions, LLC).

The independent variables that were considered as possible correlates with frailty were derived from the individuals’ billed claims from up to six months before the fifth and ninth year study visits. Therefore, each individual could contribute data up to two times to the cohort. We restricted the claims to those from inpatient and outpatient encounters, excluding claims from nursing home care or skilled nursing care. Individuals needed to be enrolled in both Medicare Part A and Part B continuously during the six month windows of interest.

Modeling Frailty

We aimed to identify the best linear combination of variables, obtainable from claims data alone, which can predict whether an individual is frail. We approached this as a predictive learning problem. Our goal was to have a claims-based index that is: a) both sensitive and specific for frailty, and b) reasonably parsimonious so that it can be operationalized easily across different data. We considered 3 techniques widely used in the machine-learning field: logistic regression with lasso penalty, gradient boosting machine and random forests. In logistic regression with lasso, a logistic regression model is fit to the data with a penalty on the sum of the absolute magnitudes of the regression coefficients.(10) This has the consequence of eliminating variables with tiny effects as well as shrinking the magnitude of those with large, but uncertain effects. Gradient boosting and random forests are similar in that they produce collections (ensembles) of tree learners. The difference is in how they arrive at the final aggregate tree learner. Boosting combines simple prediction models (e.g., decision trees with one or two branches), each of which has large bias and small variance, to produce a complex aggregate model,(11) whereas the random forest starts by generating multiple, uncorrelated decision trees and combines them using a technique called “bagging” to produce an aggregate (“forest”) of trees.(12) The degree of complexity in all the methods is determined by cross-validation on leave-out samples.

Our method of choice was the logistic regression with lasso penalty for model building, for reasons to be explained later. The optimal Lasso penalty was chosen via 10-fold cross-validation. Our goal was to maximize the area under the receiver operator curve in the cross-validation samples to yield a prediction model that, depending on the chosen cutoff, can identify frail individuals with good sensitivity and specificity. The optimal algorithm, based on this cross-validation, was the algorithm with no interaction terms. Hence, age by sex interactions are not included in the final model. We also tested the alternative modeling techniques including the random forest approach, the gradient boosting model, logistic regression with forward and backward selection of variables for retention in the model, and an approach which uses a count of comorbid diagnoses.(13) In our tree learning algorithms, we tried different interaction depths, 1 (no interactions), 2 (two-way interactions), and 3 (three-way). The models were estimated using the glmnet package in R. (14) To test whether the addition of these claims-based indicators provide added value beyond age and sex alone for predicting the frailty phenotype, we evaluated the statistical significance of the incremental AUC using DeLong’s test.(15)

Assessment of Validity

The adaptive lasso approach included a cross-validation procedure so that the area under the receiver operating curve (ROC) reflects that performance in the validation set. We further assessed the predictive validity of the CFI by examining the rates of events among individual classified as frail or non-frail with the CFI by choosing a cutoff that yielded a high positive predictive value. We explored several cutoffs including one chosen to maximize specificity and one to simultaneously maximize sensitivity and specificity. We used the predicted frailty, with our chosen cutoff, at the fifth year visit of cohort participation and quantified the frequency of hospitalizations, fractures, nursing home admission, disability [impairment in at least one activity of daily living (ADL)] and death for individuals classified as frail and non-frail over the subsequent 5-year time window. Death and hospitalization were obtained from the Medicare claims data. Fractures, nursing home admission entry, and impairment in ADLs were taken from the CHS interview data.

We calculated hazards ratios, odds ratios, and incidence rate ratios for these events, as appropriate, both without and with adjustment for age and sex. We compared these measures of association to those ones generated using the frailty phenotype for prediction of outcomes. We also compared these measures of association to those from using the Charlson Comorbidity Index alone for risk prediction. (16,17)

RESULTS

We reviewed 9 articles in which investigators aimed to identify frail or disabled individuals using knowledge of their clinical conditions. (5,18–25) Ninety-nine unique clinical or functional variables were extracted from primarily three articles.(5,21,25) Of these, 44 variables were deemed as feasible to operationalize with claims data alone. [Appendix Table 1] Most variables were discarded as they required measured clinical data (such as body mass index) or results from physical examination (such as bradykinesia, or diminished vibratory sense). Others were discarded because they would only be known from interview (problems cooking, difficulty with stairs). Others could not be retained because they required demographic information that is typically unavailable in claims data (marital status, level of education). As stated, we also included 268 CCS variables operationalized by the Agency for Healthcare Research and Quality in the modeling.

Our CHS study cohort included the 5,888 participants from 4 clinical sites. As we required continuous fee-for-service enrollment in the two 6-month windows of interest, our final analytic cohort included 4454 people. The individuals who were not continuously enrolled differed modestly from those continuously enrolled.[Appendix Table 2] Our cohort participants were 84% white; 59% were women and their mean age was 75 years at enrollment in CHS. Approximately 11% of the cohort was frail at visits 5 and 9. [Table 1] This population has been described in detail. (1)

Table 1.

Characteristics of Participants in Cardiovascular Health Study, with Continuous Enrollment in a Fee-for-Service Medicare Plan in the 6 months Preceding Visit 5^* (N=4454)

Variable	#	%
Race*
White	3747	84.1
Black	685	15.4
American Indian/Alaskan Native	10	0.22
Asian/Pacific Islander	2	0.04
Other	10	0.22
Hispanic Origin (# Missing: 21)
Yes	46	1.04
Gender**
Female	2621	58.9
Male	1833	41.1
Marital Status (# Missing: 6)**
Married	2947	66.2
Widowed	1092	24.5
Divorced	173	3.89
Separated	40	0.90
Never Married	199	4.47
Occupation (# Missing: 9)**
Professional/technical/managerial/admin	1582	35.6
Sales/clerical services	629	14.1
Craftsman/machine operator/laborer	693	15.6
Farming/forestry	98	2.20
Housewife	976	21.9
Other	470	10.6
Income (# Missing: 380)**
Under $5,000	203	4.87
$5,000 TO $7,999	411	9.85
$8,000 TO $11,999	510	12.2
$12,000 TO $15,999	632	15.1
$16,000 TO $24,999	824	19.8
$25,000 TO $34,999	621	14.9
$35,000 TO $49,999	398	9.54
Over $50,000	572	13.7
Age (# Missing: 0)**, years	75.2 (SD 5.5)
Education (# Missing: 9)*, years	13.8 (SD 4.7)

^*indicates where the continuously enrolled population differs from those who were not continuously enrolled (with a p-value of <0.05 or less)

^**p<0.01

The adaptive lasso regression yielded a model that identified individuals as frail, using their claims data from the six months preceding their assignment of a frailty phenotype, with an area under the ROC curve of 0.75. [Figure 1] From the initial large candidate set of variables, the lasso approach selected 21 variables, including age and sex.[Table 2] A model including only age and sex had an area under the ROC curve of 0.69. Thus the incremental AUC for our final model was .06, which was highly significantly better (p<1e-15) for prediction of the phenotype than a model with age and sex alone. A model that was unconstrained by the lasso technique (included many variables), yielded an AUC of 0.74 for out of sample prediction.

Figure 1.

Receiver Operator Curve for Claims-based Frailty Index

Table 2.

Variables in Claims-Based Frailty Indicator

B- coefficient	Variable
1.24	Impaired mobility
0.54	Depression
0.50	Congestive Heart Failure
0.50	Parkinson’s disease
−0.49	White race
0.43	Arthritis (any type)
0.33	Cognitive impairment
0.31	Charlson comorbidity index (>0, 0)
0.28	Stroke
0.24	Paranoia
0.23	Chronic skin ulcer
0.21	Pneumonia
−0.19	Male sex
0.18	Skin and soft tissue infection
0.14	Mycoses
0.09	Age (in 5 year categories)
0.09	Admission in past 6 months
0.08	Gout or other crystal-induced arthropathy
0.08	Falls
0.05	Musculoskeletal problems
0.05	Urinary tract infection

Using the model with 21 variables and a predicted probability cutoff of 0.12 to classify individuals as frail, the sensitivity and specificity were both maximized with a sensitivity of 66% and a specificity of 73%. At a cutoff of 0.20, the sensitivity and specificity were 35% and 91%.

The alternative modeling techniques yielded similar results but were thought to be less interpretable. The random forest model yielded an area under the ROC curve of 0.73. The gradient boosted model had an area under the ROC curve of 0.76. The model which included a simple count of any 40 comorbidities yielded an area under the ROC of 0.72. The logistic regression models with forward and backward stepwise selection selected more than 40 covariates, and had areas under the ROC curve of 0.73. (Appendix Table 3)

The CFI, using a probability cutoff of 0.20, significantly predicted death in five years [OR 3.8 (95% C.I. 3.2–4.6)], time to death [HR 3.2 (95% C.I. 2.7–3.7)], hospital admission in five years [OR 2.2 (95% C.I. 1.7–2.9)], and nursing home admission [OR 3.8 (95% C.I. 3.0–4.9)] in unadjusted models. [Table 3] Adjustment for sex and age attenuated the strength of the association considerably since these variables are already included in the CFI.

Table 3.

Tests of Predictive Validity

Outcome	Measure	Claims-based Frailty Index Unadjusted [95% C.I.]*	Claims-based Frailty Index Adjusted [95% C.I.]*
Death within 5 years	OR	3.81 [3.15 – 4.62]	1.81 [1.41 – 2.31]
Time to death	HR	3.18 [2.72 – 3.71]	1.61 [1.30 – 2.00]
Hospital admission within 5 years	OR	2.18 [1.67 – 2.86]	1.46 [1.07 – 1.99]
Time to first admission	HR	1.71 [1.46 – 1.99]	1.30 [1.06 – 1.58]
Fracture within 5 years**	OR	1.18 [0.95 – 1.45]	0.97 [0.76 – 1.25]
Nursing home admission	OR	3.80 [2.96 – 4.88]	1.45 [1.04 – 2.01]
Disability within 5 years		3.56 [2.90 – 4.37]	2.15 [1.69 – 2.74]
Number of admissions in 5 years/Zero-Inflated Negative Binomial	IRR	1.71 [1.40 – 2.09]	1.49 [1.23 – 1.80]
Number of admissions in 5 years/Zero-Inflated Poisson	IRR	1.37 [1.24 – 1.51]	1.42 [1.26 – 1.60]
Number of fractures in 5 years***/Zero-Inflated Negative Binomial	IRR	1.10 [0.81 – 1.49]	1.00 [0.78 – 1.29]
Number of fractures in 5 years***/Zero-Inflated Poisson	IRR	1.09 [0.86 – 1.37]	1.15 [0.86 – 1.56]

^*adjusted for sex and age (in years), used a cutoff of 0.20

^**counting first fracture per year of each eligible body part

^***two stage model –estimate reported is impact of frailty among people with any fractures

OR=odds ratio, HR=hazard ratio, IRR=incidence rate ratio

More patients classified as frail had 2 or more hospital admissions relative to those not classified as frail. [Figure 2] The CFI’s ability to predict outcomes was slightly less good that the frailty phenotype, but with markedly overlapping confidence intervals surrounding the measures of association. [Figure 3a and 3b]

Figure 2.

Percentage of Individuals with Inpatient Visits as Classified by Claims-based Frailty Index with a Cutoff of 0.20

Light boxes represent percentages of frail individuals. Dark boxes represents percentages of non-frail individuals. *p-value <0.05

Figure 3.

Figure 3a. Comparison of Outcomes Prediction by Claims-based Frailty Index with a Cutoff of 0.20 and Measured Frailty Phenotype, Unadjusted Models

Light diamond represents odds ratio or hazards ratio with Claims-based Frailty Index.

Dark circle represents odds ratio or hazards ratio with Frailty Phenotype. Confidence bars are 95% confidence intervals around estimates of odds ratio or hazards ratio.

Figure 3b. Comparison of Outcomes Prediction by Claims-based Frailty Index with a Cutoff of 0.20 and Measured Frailty Phenotype, Models Adjusted for Age and Sex

Light diamond represents odds ratio or hazards ratio with Claims-based Frailty Index.

Dark circle represents odds ratio or hazards ratio with Frailty Phenotype. Confidence bars are 95% confidence intervals around estimates of odds ratio or hazards ratio.

DISCUSSION

Our model, the CFI, classifies individuals as frail or not frail using only administrative claims. The area under the ROC curve was 0.75. The cutoff for classifying individuals as frail or not frail can be chosen based on the intended use. If the desire is to specifically identify frail individuals, perhaps for enrollment in a care management program, a higher cutoff would be selected, e.g., 0.25. If the desire is to more inclusively identify frail individuals, such as for planning evacuation services for a natural disaster, a lower cutoff may be more appropriate, e.g. 0.12. With this unique linked data, we are the first to build a claims-based index which is validated against an accepted reference standard for assessing the frailty syndrome.(8,26)

The field of frailty assessment is complex due to the rapid proliferation of numerous instruments to assess frailty. Buta et al. (8) identified 67 frailty instruments of which the frailty phenotype developed by Fried et al in 2001 (1) was by far the most widely used one. The second most common frailty instrument is the deficit accumulation index of Rockwood and colleagues, which takes a different approach to assessing frailty.(13,25) Here we chose to use the Fried phenotype for 2 major reasons: the phenotype instrument was developed in the CHS study, which we used for this study, and secondly, CHS study is linked to Medicare claims making it feasible for us to develop claims-only frailty index.

We found that the CFI more accurately classifies individuals as frail in comparison to a count of accumulated age-related deficits, referred to as the Frailty Index. (13,25,27) However, we recognize the utility of the Frailty Index as there have been many published comparisons of the frailty phenotype and the Frailty Index, which largely suggest reasonable concordance between these measures,(28,29) as individuals with a frailty phenotype have accumulated more aging-related deficits. However, the Frailty Index is often constructed using measures that can only be known from clinical encounters; no one has previously demonstrated that frailty can be predicted with reasonable accuracy solely with information from administrative data. Our model had an area under the ROC curve of 0.75 in comparison to 0.72 with the Frailty Index approach applied to claims data alone.

Our attention to using alternative methods of deriving a prediction model suggests that there may be models with just slightly better predictive accuracy (i.e., with higher areas under the curve), but at the cost of parsimony and transparency. For example, with random forests and gradient boosting, two of the most popular and powerful prediction approaches, we do not obtain a parsimonious model and in addition, the prediction model is totally opaque and cannot be easily communicated to other stakeholders. In fact, the AUC of ROC for random forest was the same as that of lasso, and for gradient boosting it was only larger by .01. We expect that the utility of having a transparent regression model, which can be implemented with access to 6 months of claims data, is more broadly useful than the black-box machine learning algorithms that would require distribution of specialized software to reproduce the model. Similarly, our choice to use lasso regression to identify a parsimonious model is driven by our interest in having a model that will generalize well to new data sets.

For this first construction of the CFI, we collapsed the pre-frail and non-frail cohort members into a not-frail category. We intentionally opted to categorize pre-frail individuals with non-frail individuals for this exercise because we were interested in maximizing the positive predictive value of the CFI; if the CFI categorizes an individual as frail (using a chosen cutoff), we expect them to be truly frail. If we had been most interested in identifying a robust population, we would have categorized the pre-frail with the frail individuals to assure that the model could identify a population free of frailty.

Our validation processes were three fold: 1) the adaptive lasso technique includes a cross validation process so that the reported prediction accuracy, area-under-the ROC, is from out-of-sample validation sets; 2) the face validity of the included co-variates is good – many of the included covariates have been previously identified as being correlated, cross-sectionally, with frailty, including congestive heart failure, depression, and orthopedic issues that contribute to slow gait (30,31); and 3) the predictive validity of the CFI for aging-related clinical outcomes including hospitalization, disability and mortality that are known to occur at higher rates among individuals who have been classified as frail with the phenotype.

We modeled frailty with claims data using an accepted measure of frailty – the phenotype developed in the CHS. The challenge of this study was that it required access to a data set that has both measured frailty (such as the phenotype) and claims. Increasingly, there are appropriate datasets available including the National Health and Aging Trends Study (NHATS), which is linked to Medicare data, (32) and the Atherosclerosis Risk in Communities (ARIC) cohort that is linked to Medicare claims and whose investigators have recently begun measuring frailty. (33) These provide opportunities for testing the reproducibility of this work – that is, will the same variables be selected in these data.

As described briefly above, the CFI is expected to have many uses for research, for population health management, and for emergency preparedness. A claims-based index may also be used as an effect modifier in investigation of treatment effectiveness where there is expected to be differences in treatment-response due to frailty. This may be valuable for analyses of trial or cohort study data where the risk-benefit balance of treatments may differ or not differ from that which is seen in non-frail individuals. For example, clinicians are often reluctant to offer chemotherapy to frail individuals, as they think that the risks exceed the potential benefits. If we were to analyze claims data (perhaps registry-linked claims data), and find that individuals with lymphoma and frailty as identified with the CFI have survival outcomes following chemotherapy comparable to non-frail individuals, this may encourage oncologists to offer therapy and payers to provide coverage. Presently, such investigations require the measurement of frailty in a clinical setting.

The use of frailty as an effect modifier is likely more appropriate in many situations than using age as an effect modifier, although this remains to be tested. A claims-based index may be valuable for population health planning – the identification of a population of frail individuals may help in distributing services to those most likely to benefit, such as care management services, transition of care services, or other high-intensity management services aimed at high value care delivery and best use of resources.

A claims-based index may also be valuable for emergency disaster preparedness where identification of high concentrations of frail individuals may be essential for public safety. It was recognized most poignantly during Hurricane Katrina, but also during more recent storms, that the most vulnerable individuals are likely to suffer severely during natural disasters.(34,35) There is increasing attention to planning for nursing home evacuations,(36,37) but less planning for service delivery to frail individuals living in the community. Although voluntary registration of frail or disabled individuals may be helpful, a method to identify these individuals, through their Medicare claims, may be valuable in an emergency situation. More likely, identification of communities that are enriched with frail individuals will allow for better preparedness in these communities.

Conclusion

Claims data alone can be used to classify individuals as frail and non-frail. Although other models may exist or be developed that have higher predictive validity for clinical outcomes, they have not been designed to specifically identify a frail population, validated against a well-established assessment of frailty. Many investigators believe that frailty is a distinct clinical condition and that interventions for primary prevention of frailty may be developed. (38) Similarly, the interventions required for prevention of adverse outcomes among those with frailty may differ from those needed to prevent adverse outcomes among other populations who are known to be at high risk (such as a multimorbid population). Therefore, this CFI was developed to identify specifically a frail population.