Assessment of coding-based frailty algorithms for long-term outcome prediction among older people in community settings: a cohort study from the Shizuoka Kokuho Database

Abstract

Objectives

To assess the applicability of Electronic Frailty Index (eFI) and Hospital Frailty Risk Score (HFRS) algorithms to Japanese administrative claims data and to evaluate their association with long-term outcomes.

Study Design and Setting

A cohort study using a regional government administrative healthcare and long-term care (LTC) claims database in Japan 2014–18.

Participants

Plan enrollees aged ≥50 years.

Methods

We applied the two algorithms to the cohort and assessed the scores’ distributions alongside enrollees’ 4-year mortality and initiation of government-supported LTC. Using Cox regression and Fine–Gray models, we evaluated the association between frailty scores and outcomes as well as the models’ discriminatory ability.

Results

Among 827,744 enrollees, 42.8% were categorised by eFI as fit, 31.2% mild, 17.5% moderate and 8.5% severe. For HFRS, 73.0% were low, 24.3% intermediate and 2.7% high risk; 35 of 36 predictors for eFI, and 92 of 109 codes originally used for HFRS were available in the Japanese system. Relative to the lowest frailty group, the highest frailty group had hazard ratios [95% confidence interval (CI)] of 2.09 (1.98–2.21) for mortality and 2.45 (2.28–2.63) for LTC for eFI; those for HFRS were 3.79 (3.56–4.03) and 3.31 (2.87–3.82), respectively. The area under the receiver operating characteristics curves for the unadjusted model at 48 months was 0.68 for death and 0.68 for LTC for eFI, and 0.73 and 0.70, respectively, for HFRS.

Conclusions

The frailty algorithms were applicable to the Japanese system and could contribute to the identifications of enrollees at risk of long-term mortality or LTC use.

Key Points

• Evaluation of the transportability of Electronic Frailty Index (eFI) and Hospital Frailty Risk Score (HFRS) to health systems outside the UK is needed.

• The usefulness of eFI and HFRS in predicting long-term clinical outcomes is understudied.

• In our study, we were able to apply eFI and HFRS algorithms to the Japanese administrative claims database.

• eFI and HFRS were able to discriminate those with or without incidences of death or long-term care (LTC) initiation within four years.

• eFI and HFRS could contribute to identify older people at increased risk of long-term mortality or in need of LTC.

Introduction

Frailty is an age-related accumulation of physiological debilities, resulting in vulnerability, especially when triggered by clinical events. Previous studies have demonstrated that frail subjects have an increased risk of poor clinical outcomes, such as the requirement of long-term care (LTC) or mortality [1, 2]. With increasing life expectancy, the number of frail older people has increased substantially [3, 4] and has become a major issue [5]. The guideline published by the International Conference of Frailty and Sarcopenia Research task force strongly recommends that older people (aged ≥65 years) be screened with suitable frailty instruments [6]. Identification of frailty can be used to direct older persons towards appropriate interventions and assist them in the maintenance of a healthy and independent lifestyle.

To facilitate the early identification of frail individuals in community settings and subsequent intervention, a validated screening tool, ideally associated with long-term outcomes and the use of healthcare resources, is required. Large-scale healthcare data (e.g. administrative claims data and electronic health records) currently provide information on patients’ clinical encounters, diagnoses and resource use. These data are increasingly used in clinical research, and provide evidence to facilitate clinical and policy decisions, complementing the evidence from clinical trials. To date, several algorithms have been developed to identify and measure frailty using these data, including the Electronic Frailty Index (eFI) [7] and the Hospital Frailty Risk Score (HFRS) [8].

HFRS has been used and largely validated in acute care settings [9–11], whereas eFI has been widely used in general practice in England [12] and converted to other diagnostic codes [13, 14]. However, the international validity of these instruments and their association with long-term outcomes have been insufficiently studied. Therefore, we aimed to assess whether the eFI and HFRS, originally developed in the UK, can be implemented in the Japanese healthcare system. We also evaluated the association between frailty measured using these algorithms with long-term outcomes (i.e. mortality and the initiation for government-supported LTC).

Methods

Study design and data source

We conducted a cohort study using the Shizuoka Kokuho Database (SKDB), an administrative claims database of enrollees in the municipal government health insurance programme in Shizuoka Prefecture, Japan [15]. The SKDB contains health insurance claim data (e.g. monthly claims for patients’ diagnoses, procedures and medications) and LTC insurance claim data. Among residents aged <75 years, 22.3% are enrolled in the National Health Insurance (NHI) and all residents ≥75 are enrolled in the late-stage medical care system for the elderly. All residents aged ≥65 years or those aged 40–64 years with severe illness defined by the government (e.g. advanced cancer, rheumatic disease, amyotrophic lateral sclerosis, cerebrovascular disease) are eligible for LTC insurance. Details of the LTC system are explained in Appendix 1.

We accessed all data on procedures and diagnoses involved in health insurance (NHI and late-stage medical care system for the elderly) claims and the use of LTC insurance for all beneficiaries during April 2012–September 2018. The study was approved by the Institutional Review Board of Shizuoka General Hospital (Shizuoka, Japan) [SGHIRB #2020004].

Participants

We selected insurance enrollees aged ≥50 years on 1 April 2014, from the database. We included enrollees aged ≥50 years to compare the frailty score distributions among those aged 50–64 years with those who were older. We designated April 2014 as the ‘index month’, and the preceding 12-month period as the ‘look-back period’. We excluded any subjects who were ineligible for the health plan during the 24-month period preceding the index month to limit the target cohort to subjects with complete diagnostic records during the look-back period. We also excluded participants who had no recorded diagnosis during the look-back period. The assessment of frailty score distribution and its relationship with age was conducted using this full cohort. To reduce the required computation time, we conducted a time-to-event analysis in a randomly sampled 10% subset of the full analytical cohort. We sampled the enrollees using a simple random sampling method without replacement in SAS (PROC SURVEYSELECT). A longitudinal assessment of LTC service use was conducted among those from the subset of enrollees with no record of LTC service use during the look-back period and who were aged ≥65 years in April 2014 (LTC assessment cohort) (Figure 1). We deleted those aged <65 years because the requirement for the receipt of LTC services differed between participants aged 50–64 and ≥65 years.

Figure 1.

A flow diagram of the enrolment of the study cohort.

Member characteristics

Sex and age data were extracted from the subscriber list, with data on each patient’s diagnosis, medications and LTC service use during the 12-month look-back period obtained from the claims data. Diagnosis codes were extracted with the Japanese electronic claim codes, which are linked to the corresponding International Classification of Diseases 10th version (ICD-10) codes [16].

Frailty measurements

The eFI was developed by Clegg and colleagues based on the cumulative deficit model of frailty. The index is measured as the proportion of 36 equally weighted deficits present in an individual, captured with the Clinical Terms Version 3 Read codes in routinely collected primary care data in the United Kingdom [7, 17]. We used the converted ICD-10 list of deficits based on previous studies (Appendix 2) [13, 14]. We defined ‘polypharmacy’ as the prescription of ≥5 drugs over ≥6 months during the look-back period [18].

The HFRS is a score developed by Gilbert and colleagues from 109 ICD-10 codes. This score was developed to identify patients’ high-risk conditions using routinely collected secondary care hospital administrative data in the United Kingdom [8, 17]. Each ICD-10 code is given a weight of 0.1–7.1, corresponding to the strength of its association with being in the frail subgroup.

For each patient, we calculated eFI and HFRS based on the ICD-10 codes that were recorded in the inpatient and outpatient claims during the 12-month look-back period. We then grouped the members into categories, using the cut-off points described in the original studies [7, 8]: fit (0–0.12), mild frailty (>0.12–0.24), moderate frailty (>0.24–0.36) and severe frailty (>0.36) for eFI; low (<5), intermediate (5–15) and high (>15) risk for HFRS.

Outcomes

The main outcomes of interest were all-cause mortality and the use of government-supported LTC, which—accounting for death as a competing event—was assessed among those beneficiaries with no LTC service use records from the look-back period.

Statistical analysis

We tabulated the baseline characteristics of the plan enrollees. We assessed the distributions of the two frailty scores and assessed their categorisation. Each diagnosis code and polypharmacy’s prevalence was evaluated. Spearman’s correlation coefficient was used to assess the relationship between age and the continuous frailty scores.

We censored participants at the time when their insurance plan enrolment terminated or on 30 September 2018, whichever came first (maximum follow-up period of 53 months). We estimated overall survival for each frailty category using the Kaplan–Meier survival curves and compared them using the log-rank test. We constructed Cox proportional hazard models (with/without adjustment for age and sex) to determine the association between frailty score categories and mortality. We evaluated LTC initiation using cumulative incidence functions with Gray’s test and Fine–Gray’s subdistribution hazard models, with/without adjustment for age and sex. The discrimination was assessed from the time-dependent area under the receiver operating characteristic curves (area under the curve, AUC) using inverse probability weighting [19] for eFI and HFRS for outcomes at 12, 24, 36 and 48 months. Statistical analyses were performed using SAS version 9.4 (SAS Institute, Cary, NC, USA). R software version 3.5 (R Foundation for Statistical Computing, Vienna, Austria) was used to estimate the AUCs of the models with competing risks [20, 21].

Results

Participant characteristics

Among the 827,744 insurance enrollees, the mean (standard deviation, SD) age was 74.2 (10.4) years and 59.4% were female. Participants were followed up for a median (interquartile ranges, IQRs) of 4.4 (4.4–4.4) years. The median (IQRs) frailty measure was 0.14 (0.09–0.26) for eFI and 2.2 (0–5.3) for HFRS (Table 1). The distribution of each score was heavily right skewed, with maximum scores of 0.86 and 51.4, respectively (Appendix 3). Of the study population, 354,023 (42.8%) were categorised by eFI as fit, 258,549 (31.2%) as mildly frail, 145,091 (17.5%) as moderately frail and 70,081 (8.5%) as severely frail. Most participants were categorised by HFRS as low risk (604,211 [73.0%]), followed by intermediate (201,359 [24.3%]) and high (22,174 [2.7%]) risk. Among those aged <65 years (n = 140,774), frail patients were less frequently observed compared to the older patients, with the 2,610 (1.9%) patients categorised as severely frail using eFI and the 707 (0.5%) patients categorised as high risk using HFRS. Upon limiting the enrollees to those included in the original eFI study (i.e. aged ≥65 years), 255,668 (37.2%) were categorised as fit, 227,520 (33.1%) as mildly frail, 136,311 (19.8%) as moderately frail and 67,471 (9.8%) as severely frail. Among the participants aged ≥75 years with ≥1 admission record, the proportions of patients in the intermediate and high-risk groups based on HFRS were 54.1 and 16.7%, respectively (Appendix 5).

Table 1.

Baseline characteristics

Characteristics		N = 827,744
Age (years), mean (SD)		74.2	(10.4)
Age group, N (%)
	50–64	140,774	(17)
	65–74	309,771	(37.4)
	75–84	228,628	(27.6)
	≥85	148,571	(17.9)
Female, N (%)		491,497	(59.4)
Previous LTC service use, N (%)		109,511	(13.2)
LTC level, N (%)
	Support level	21,465	(2.6)
	Care level 1–3	63,597	(7.7)
	Care level 4–5	24,449	(3)
Number of medications, median (IQR)		13	(6–22)
eFI score, mean (SD)		0.17	(0.12)
HFRS score, mean (SD)		3.6	(4.3)
eFI frailty category, N (%)
	Fit	354,023	(42.8)
	Mild	258,549	(31.2)
	Moderate	145,091	(17.5)
	Severe	70,081	(8.5)
HFRS frailty category, N (%)
	Low	604,211	(73.0)
	Intermediate	201,359	(24.3)
	High	22,174	(2.7)

Prevalence of predictors and relationships with age

We derived both HFRS and eFI using Japanese electronic claim codes linked to the corresponding ICD-10 codes. Appendix 6A and B list the variables and corresponding codes used for these measurements, together with their prevalence in the cohort. For HFRS, codes corresponding to 92 of the 109 ICD-10 diagnostic codes were available. The prevalence of each variable in the HFRS increased with age. Among the highly weighted diagnostic codes, Alzheimer’s disease and the sequelae of cerebrovascular disease were markedly more prevalent in people aged ≥85 years (15.2 and 12.9%, respectively) than in those aged 50–64 years (0.2 and 1.9%, respectively) (Appendix 6A). For eFI, which consists of 36 variables, 35 were identified in the Japanese coding system, the exception being the code for ‘falls’. Similar to the HFRS variables, many of the deficits had a much higher prevalence in people aged ≥85 years than in those aged 50–64 years (Appendix 6B). The prevalence of polypharmacy, cardiovascular diseases and osteoporosis was highest among people aged ≥85 years. The proportion of subjects classified as severely frail increased with age: from 5.6% among 50–64 years olds to 37.9% among those aged ≥85 years for eFI, and from 8.9 to 49.4%, respectively, for HFRS (Figure 1). Spearman’s correlation coefficients between two frailty scores and age were 0.43 [95% confidence interval (CI), 0.427–0.439] for eFI and 0.42 (95% CI, 0.415–0.427) for HFRS.

Risk of mortality and use of LTC service

Among a 10% random sample of participants (n = 82,775), all-cause mortality per 100 person year was 3.9. The survival rate at 48 months decreased with increasing severity of the frailty category, from 92.8% (95% CI, 92.5–93.0%) in the fit group to 66.6% (95% CI, 65.5–67.7%) in the severely frail group according to eFI categories, and from 91.4% (95% CI, 91.2–91.6%) in the low-risk group to 45.6% (95% CI, 43.5–47.6%) in the high-risk group according to HFRS category (Figure 2). The hazard of death increased alongside the severity of frailty, with/without adjustment for age/sex: adjusted hazard ratio (HR) 2.09 (95% CI, 1.98–2.21) for severely frail subjects relative to fit subjects according to eFI, and HR 3.79 (95% CI, 3.56–4.03) for high-risk subjects relative to low-risk subjects according to HFRS (Table 2).

Figure 2.

Kaplan–Meier survival curves for mortality and cumulative incidence functions of government-supported LTC service use for up to 53 months according to frailty measures derived from healthcare claims data.

Table 2.

Results of Cox proportional hazard model regression for mortality and Fine–Gray proportional subdistribution hazard model regression for use of government-supported long-term care services among participants who used no long-term care services during the look-back period

		eFI									HFRS
		No. of events(*)		Unadjusted			Adjusted†				No. of events(*)		Unadjusted			Adjusted†
Outcome				HR (95% CI)¶			HR (95% CI)¶						HR (95% CI)¶			HR (95% CI)¶
Mortality‡		12,580	(3.9)								12,580	(3.9)
	Fit	2,714	(1.9)	1	-		1	-		Low	5,573	(2.3)	1	–		1	–
	Mild	3,920	(3.8)	2.05	(1.95–2.15)		1.22	(1.16–1.28)		Intermediate	5,695	(7.8)	3.45	(3.33–3.58)		1.87	(1.80–1.95)
	Moderate	3,460	(6.3)	3.35	(3.19–3.53)		1.52	(1.45–1.60)		High	1,312	(20.1)	8.99	(8.47–9.55)		3.79	(3.56–4.03)
	Severe	2,486	(10.1)	5.43	(5.14–5.73)		2.09	(1.98–2.21)
Use of LTC services§		9,264	(3.9)								9,264	(3.9)
	Fit	2,073	(2.1)	1	–		1	–		Low	5,284	(2.8)	1	–		1	–
	Mild	3,100	(3.9)	1.9	(1.80–2.01)		1.3	(1.23–1.38)		Intermediate	3,623	(7.4)	2.74	(2.63–2.86)		1.75	(1.67–1.83)
	Moderate	2,584	(6.2)	3.14	(2.97–3.33)		1.7	(1.60–1.80)		High	357	(15.2)	5.99	(5.32–6.74)		3.31	(2.87–3.82)
	Severe	1,507	(9.5)	5.03	(4.71–5.38)		2.45	(2.28–2.63)

^*Event rate per 100 person years.

^†Adjusted for age and sex.

^‡ n = 82,775.

^§ n = 57,899; 24,876 participants aged ≤65 years or those with the records of the use of LTC service during the look-back period were excluded.

^¶HRs for the use of LTC service were estimated using Fine–Gray’s subdistribution hazard models.

In participants aged ≥65 years with no previous LTC service use (n = 57,899), the event rate of the LTC service initiation was 3.9 per 100 person years. We observed a similar relationship between frailty severity and the 48-month cumulative incidence of LTC service use: from 7.9% (95% CI, 7.5–8.2%) in the fit group to 34.5% (95% CI, 33.0–35.9%) in the severely frail group for eFI, and from 10.8% (95% CI, 10.5–11.0%) in the low-risk group to 49.4% (95% CI, 45.6–53.0%) in the high-risk group for HFRS (Figure 2). We observed an increase in the incidence of LTC service initiation with increasing frailty severity; even after adjusting for age and sex. The subdistribution hazard ratio (sHR) for the severely frail compared with fit was 2.45 (95% CI, 2.28–2.63) on eFI, and the adjusted sHR for high risk compared with low-risk subjects was 3.31 (95% CI, 2.87–3.82) on HFRS (Table 2). Appendices 7 and 8 show the Kaplan–Meier survival curves and cumulative incidence functions stratified by age group, respectively. Mortality and incidence of LTC service use increased with age. Separation of the curves was most pronounced in the 75–84 years age group. Depicted separation was also more pronounced across the HFRS than eFI categories.

Model discrimination

The two algorithms discriminated the participants well at each time point. The AUC for mortality on HFRS (base model: 0.77 at 12 months, 0.73 at 48 months) was slightly superior to that on eFI (0.70 and 0.68, respectively). Among people aged ≥65 years with no history of LTC service use during the look-back period, the AUCs for LTC service use outcome at 12 and 48 months were 0.71 and 0.68, respectively, for eFI, and 0.75 and 0.70, respectively, for HFRS (Table 3). After adjustment for age and sex, the 48-month AUC for LTC service use was similar for eFI (48-month AUCs: mortality = 0.82, LTC service use = 0.84) and HFRS (0.84 and 0.85, respectively).

Table 3.

Time-dependent AUC estimates for mortality and government-supported long-term care service use

Outcome		AUC
		12 months		24 months		36 months		48 months
		Base model	+ Age, sex	Base model	+ Age, sex	Base model	+ Age, sex	Base model	+ Age, sex
Mortality*
	eFI	0.70	0.81	0.69	0.82	0.68	0.82	0.68	0.82
	HFRS	0.77	0.83	0.75	0.83	0.73	0.83	0.73	0.84
Use of LTC services†
	eFI	0.71	0.82	0.68	0.83	0.68	0.84	0.68	0.84
	HFRS	0.75	0.84	0.72	0.84	0.70	0.84	0.70	0.85

^* n = 82,775.

^† n = 57,899; 24,876 participants aged ≤65 years or those with the records of the use of LTC service during the look-back period were excluded.

Discussion

We applied two frailty scores, eFI and HFRS, to a Japanese health systems database. Although several diagnostic codes in the algorithms were unavailable in the Japanese coding system, the application of these algorithms resulted in frailty score distributions with good variation, similar to those reported in the original cohorts in the UK [7]. Frailty scores were strongly associated with the incidences of death and LTC service use, and showed acceptable discriminatory power (AUC > 0.68), supporting their transferability to the Japanese systems. The HFRS better discriminated the outcomes than the eFI at each time point.

The observed right-skewed distributions were similar to those reported in the original and other externally validated studies [7, 8, 22]. eFI was developed within an ambulatory population. eFI scores in our cohort were higher (fit, 43%; mild frailty, 31%; moderate frailty, 18%; severe frailty, 9%) than in the original study (fit, 50%; mild frailty, 35%; moderate frailty, 12%; severe frailty, 3%). Several deficits (e.g. cerebrovascular disease, diabetes mellitus, heart failure and peptic ulcer) were more prevalent in our cohort than in the original. The HFRSs in our cohort were lower (low risk, 73%; intermediate risk, 24%; high risk, 3%) than those in a previous study constructed with a 1-year look-back window (low risk, 41%; intermediate risk, 37%; high risk, 22%) [23]. This is probably attributable to the inclusion in our cohort of health plan enrollees without admission records. Upon restricting the participants to those aged ≥75 years with a history of one or more than one admissions, the prevalence of frailty became more similar to the previous study, and most patients were in the intermediate-risk category. Fewer participants were classified as low or high risk than in the previous study. This could be partly explained by our use of diagnostic records from both inpatient and outpatient claims.

Among the 36 deficits used in the algorithm to capture the original eFI using ICD-10 codes, one deficit, ‘fall’, was missing from the Japanese claim codes. Among the 109 codes used to define HFRS in the original study, 17 were missing. ‘Fall’, which is included in both eFI and HFRS, is one of the key factors used to measure frailty, given its high prevalence and strong association with frailty [24, 25]. Missing such a component from the algorithm to capture frailty possibly distorted the assessment of frailty relative to the originally intended concept. However, several diagnostic codes included in score measurements are strongly associated with falls [26]. In our study, codes for fragility fractures within the HFRS and eFI, such as fractures of the proximal humerus (included as part of ICD 10 code S42; fracture of shoulder and upper arm), distal radius (included as part of S52; fracture of forearm), and femoral neck and trochanteric fracture (included as part of S72; fracture of femur), were observed. These codes may have acted as proxies for ‘falls’ in our study, reducing the impact of missingness.

The long-term prognoses of frail patients have been evaluated with the eFI [7, 27]. HFRS is commonly used in studies evaluating clinical outcomes during hospitalisation or after discharge, [9, 11, 28] but the association with long-term outcomes is less studied. In our assessment, the discrimination of the HFRS model was better than that of the eFI model at each time point, supporting its potential utility in studies of long-term outcomes. At the same time, both frailty scores were associated with both outcomes in our study, even after stratification by age group. This supports the notion that these frailty scores identify frailty-specific conditions, independent of age.

The two frailty scores were clearly associated with mortality and LTC initiation, which is consistent with past prospective studies using clinically measured frailty scores [29–31]. The relationship between frailty and LTC resource utilisation is especially important in the context of LTC insurance system sustainability in ageing societies. In prior analyses, the need for help from others was considered as a proxy for frailty [32–34]. Another study developed a frailty score from healthcare claim data using activities of daily living dependency as an outcome [34]. Our study adds support to the longitudinal association between frailty and LTC requirement. The assessment of how frailty among older people increases the need for LTC services in the target population may help us to estimate resource needs in the near future.

Limitations

This study had several noteworthy limitations. First, the frailty scores of the participants admitted to geriatric health service facilities may have been underestimated because the payment for medical services (e.g. drugs prescribed for long-term disease management by the attached medical institution) is not covered by health insurance but by LTC insurance [35]. Second, the applicability of the algorithms to administrative claim data in this study does not ensure its transferability to other databases. Further studies are required to evaluate how generalisable these findings are to other health systems in epidemiological studies or for use in systematic frailty screening.

Conclusions

Our study shows that adapting two frailty algorithms with globally used diagnostic codes was useful for evaluating individuals’ frailty. The frailty scores can be measured in the Japanese claims database, and individuals at greater risk of death or more likely to use LTC services can be identified.

Declaration of Conflicts of Interest

Dr Kohsaka reports the receipt of investigator-initiated grant funding from Bayer and Daiichi Sankyo. Dr Kumamaru has received consultation fees from Mitsubishi Tanabe Pharma and speaker fees from Pfizer Japan Inc and Johnson & Johnson K.K. Dr Yamamoto has received consultation fees from Mitsubishi Tanabe Pharma, speaker fees from Chugai Pharmaceutical Co., Ltd and Ono Pharmaceutical Co., Ltd, and payment for a manuscript from Astellas Pharma Inc. Dr Miyata has received a research grant from AstraZeneca K.K. for an independent research project through the PeoPLe Consortium at Keio University. Drs Kohsaka, Kumamaru, Nishimura, Ichihara, Yamamoto and Miyata are affiliated with the Department of Healthcare Quality Assessment at The University of Tokyo. This department is a social collaboration department supported by the National Clinical Database, Johnson & Johnson K.K. and Nipro Corporation. Dr Sandhu receives research support from the National Heart, Lung and Blood Institute (1K23HL151672-01).

Declaration of Conflict of Interest

The other authors report no potentially conflicting interests.

Declaration of Sources of Funding

The Research Support Center at Shizuoka General Hospital and Shizuoka Graduate University of Public Health conducts contract research projects on public health in Shizuoka Prefecture, including this study.