Comparing the Clinical Frailty Scale and an International Classification of Diseases-10 Modified Frailty Index in Predicting Long-Term Survival in Critically Ill Patients

IMPORTANCE:

The Clinical Frailty Scale (CFS) is the most used frailty measure in intensive care unit (ICU) patients. Recently, the modified frailty index (mFI), derived from 11 comorbidities has also been used. It is unclear to what degree the mFI is a true measure of frailty rather than comorbidity. Furthermore, the mFI cannot be freely obtained outside of specific proprietary databases.

OBJECTIVE:

To compare the performance of CFS and a recently developed International Classification of Diseases-10 (ICD-10) mFI (ICD-10mFI) as frailty-based predictors of long-term survival for up to 1 year.

DESIGN:

A retrospective multicentric observational study.

SETTING AND PARTICIPANTS:

All adult (≥16 yr) critically ill patients with documented CFS scores admitted to sixteen Australian ICUs in the state of Victoria between April 1, 2017 to June 30, 2018 were included. We used probabilistic methods to match de-identified ICU admission episodes listed in the Australia and New Zealand Intensive Care Society Adult Patient Database with the Victorian Admission Episode Dataset and the Victorian Death Index via the Victorian Data Linkage Centre.

MAIN OUTCOMES AND MEASURES:

The primary outcome was the longest available survival following ICU admission. We compared CFS and ICD-10mFI as primary outcome predictors, after adjusting for key confounders.

RESULTS:

The CFS and ICD-10mFI were compared in 7,001 ICU patients. The proportion of patients categorized as frail was greater with the CFS than with the ICD-10mFI (18.9% [n = 1,323] vs. 8.8% [n = 616]; p < 0.001). The median (IQR) follow-up time was 165 (82–276) days. The CFS predicted long-term survival up to 6 months after adjusting for confounders (hazard ratio [HR] = 1.26, 95% CI, 1.21–1.31), whereas ICD-10mFI did not (HR = 1.04, 95% CI, 0.98–1.10). The ICD-10mFI weakly correlated with the CFS (Spearman’s rho = 0.22) but had a poor agreement (kappa = 0.06). The ICD-10mFI more strongly correlated with the Charlson comorbidity index (Spearman’s rho 0.30) than CFS (Spearman’s rho = 0.25) (p < 0.001).

CONCLUSIONS:

CFS, but not ICD-10mFI, predicted long-term survival in ICU patients. ICD-10mFI correlated with co-morbidities more than CFS. These findings suggest that CFS and ICD-10mFI are not equivalent.

RELEVANCE:

CFS and ICD-10mFI are not equivalent in screening for frailty in critically ill patients and therefore ICD-10mFI in its current form should not be used.

KEY POINTS

• Question: We aimed to test the primary hypothesis that the Clinical Frailty Scale (CFS) would be a stronger predictor of longest available survival than the International Statistical Classification of Diseases and Related Health Problems, 10th revision (ICD-10) coding–derived modified frailty index.

• Finding: This multicenter cohort study found that CFS, but not ICD-10 coding–derived modified frailty index (ICD-10mFI), predicted long-term survival in ICU patients. ICD-10mFI correlated with comorbidities more than CFS. The findings also suggest that CFS had greater validity in discriminating long-term survivors from those who die.

• Meaning: CFS and ICD-10mFI are not equivalent in screening for frailty in critically ill patients.

Frailty is a clinically recognizable state of increased vulnerability due to an aging-associated decline in physical, physiologic, and cognitive reserve (1). People with frailty have a reduced ability to cope with acute stressors (2) and have poorer outcomes (3, 4). Frailty is common in patients admitted to ICUs (5, 6). Such patients generally have worse outcomes (3), including quality of life, functional dependence (3, 4), disability, and mortality at hospital discharge and 1-year post admission (4, 6–8).

In ICU, the CFS (9) is the most used frailty measure in ICU patients (3, 10–12) and is validated to stratify older adults according to the level of vulnerability (9) and to reliably predict poor short- and longer term outcomes in critically ill patients (6, 8, 12–14).

As an alternative to the CFS and because of conveniences and electronic data availability, administrative data have been used as surrogates, to assess the epidemiology of frailty retrospectively (15). There is also increasing interest in researching the validity of frailty assessment using administrative data such as ICD-10 because of their global reach (15, 16). A recent study found that an ICD-10-derived score, the Hospital Frailty Risk Score (HFRS), and CFS independently predicted up to 1-year survival following an ICU admission with moderate discrimination (14). Similarly, the modified frailty index (mFI) may be one such administrative data-based surrogate. The mFI, however, is derived from mapping 11 variables contained within the National Surgical Quality Improvement Program database to variables contained within the original 70-item Canadian Study of Health and Aging Frailty Index (17). Regrettably, nine of the 11 mFI items do not conform to the 2011 expert consensus statement on frailty (18, 19). Thus, it is unclear to what degree the mFI is a true measure of frailty rather than comorbidity. Furthermore, the mFI cannot be freely obtained outside of specific proprietary databases. To address this challenge, experienced intensivists and geriatricians achieved consensus for appropriate ICD-10 codes that could be used as surrogates for the mFI (20), potentially enabling the use of ICD-10 global datasets to study the epidemiology of frailty.

In this study, using such ICD-10 codes, we aimed to test the primary hypothesis that the CFS would be a stronger predictor of survival at longest available follow-up than the ICD-10mFI.

METHODS

Ethics Approval

The Peninsula Health Ethics Committee (reference number HREC/47502/PH-2019-177333, DHHS/RQ907) approved the study on June 24, 2019, with a waiver of informed consent. The study was conducted in accordance with the Helsinki Declaration of 1975.

Study Design, Setting, and Patients

We conducted a retrospective multicenter cohort study from April 1, 2017, to June 30, 2018, including consecutive critically ill patients admitted to 16 public ICUs in the state of Victoria, Australia, with a documented CFS score. The date for censoring survival follow-up was July 31, 2018, to ensure that there was at least a 1-month follow-up for all patients. We only included the first hospital admission during the study period.

Data Sources and Measurement

The Australian and New Zealand Intensive Care Society Adult Patient Database

This bi-national clinical quality registry dataset collects de-identified information on admissions to contributing adult ICUs in Australia and New Zealand. All ICUs within public hospitals in the study state (Victoria) contributed throughout the study period. Patient demographic details, diagnostic, biochemical, physiologic, chronic health parameters, illness severity scores from the first 24 hours of ICU admission and in-hospital outcomes are captured by the Charlson comorbidity index (CCI) derived from this admission data. Definitions of each condition are described in the Australian and New Zealand Intensive Care Society-Adult Patient Database (ANZICS-APD) data dictionary (21).

Victorian Admitted Episodes Dataset

All Victorian public hospitals submit data to the Victorian Department of Health and Human Services. This administrative dataset contains the ICD-10-coded diagnostic information, demographic data, and outcomes for all hospitalizations in the study state of Victoria.

Victorian Death Index

This administrative dataset records the date and cause of all deaths that occur in the study state (Victoria), based on the issued death certificates. Information was available up to July 31, 2018.

Probabilistic methods were used to match deidentified APD, Victorian admitted episodes dataset (VAED), and Victorian Death Index (VDI) data.

The VAED and VDI databases are summarized in Supplementary Tables 1 and 2 (http://links.lww.com/CCX/B72).

Frailty Definitions

CFS.

Frailty was measured with a modified version of the Canadian Study of Health and Aging CFS that categorizes patients as nonfrail (1 = very fit, 2 = well, 3 = managing well, 4 = vulnerable) or frail (9) (5 = mild, 6 = moderate, 7 = severe, 8 = very severe). Terminally ill patients (CFS = 9) were excluded from analysis. The CFS is collected as part of the ANZICS-APD, at ICU admission in 16 of the 23 hospitals in Victoria, Australia. The CFS was assigned by trained data collectors working in ICU, including junior doctors, nurses, and administrative staff and based on the patient’s level of physical function in the 2 months preceding ICU admission (21).

ICD-10mFI.

The ICD-10mFI (range 0–9) was calculated as an approximation using all the pertinent ICD-10 codes, from prior and new conditions recorded and accrued from indexed hospitalization, as required to identify the 11 variables using the Delphi consensus investigation (Supplementary Table 3, http://links.lww.com/CCX/B72) (20). The ICD-10mFI categorizes patients as frail if the score is greater than or equal to 3. Frailty categorized by the ICD-10mFI was compared with the CFS-categorized frailty.

Study Aims and Outcomes

The primary aim was to assess the use of the Delphi-consensus estimated ICD-10mFI as a frailty screening tool in ICU patients by comparing its performance with the CFS as a predictor of longest available survival following ICU admission. The secondary aims were to compare the performance of the CFS and the ICD-10mFI obtained at hospital admission as predictors of ICU, hospital, 28-day, 90-day, 6-month, and 1-year mortality, and, where applicable, home discharge. Finally, ICU and hospital mortality rates were obtained from the APD records, with after discharge mortality determined from the VDI.

Statistical Analysis

Categorical comparisons between nonfrail (CFS = 1–4, ICD-10mFI = 0–2) and frail (CFS ≥ 5, ICD-10mFI ≥ 3) patients were performed using chi-square tests for categorical data, two-sample t tests for normally distributed data, and Wilcoxon rank-sum test otherwise with the results reported as proportion (number), mean (sd ), or median (interquartile range [IQR]), respectively. The relationship between the CFS, ICD-10mFI, and CCI was determined using Spearman correlation coefficients. Kappa coefficients as binary (nonfrail vs frail) were used to determine the agreement between the two frailty measures. Patient survival was compared using Cox proportional hazards regression adjusting for patient illness severity and sex, with results reported as hazard ratios (HRs, 95%CI) with results reported using Kaplan-Meier survival. The performance of the CFS and the ICD-10mFI in predicting time-specific mortality rates and discharge home was determined using logistic regression models with results reported as area under the receiver operating characteristic (AUROC) plots with comparison using chi-square tests (22). Multivariable logistic regression analysis, adjusted for sex and illness severity, was used to compare the performance of the CFS and the ICD-10mFI as predictors of mortality rates. Illness severity was determined using the Australian and New Zealand risk of death (ANZROD) which is a highly predictive mortality prediction model used for benchmarking ICU performance in Australia and New Zealand (23, 24). To determine if frailty measures differed according to age, subgroup analysis was performed considering three age groups (< 65, 65–75, > 75 yr).

Although hypertensive elements were integral to the Delphi development of ICD-10mFI (20), the most common coding for hypertension (ICD-10 U82.3) did not meet consensus criteria. However, over one third of our population experienced this form of hypertension. Thus, to explore the robustness of the Delphi consensus, we conducted a sensitivity analysis additionally including ICD-10 U82.3 as a potential marker for the hypertension component of the ICD-10 based frailty assessment. The ICD-10mFI was quantified with the ICD-10 codes from the linked dataset using R software, Version 3.5.0 (The R Foundation, Vienna, Austria). Analysis was performed using SPSS Version 27 (IBM Corp, Armonk, NY), and a two-sided p value of 0.05 was used to indicate statistical significance.

RESULTS

We studied 20,457 admissions from 16 hospitals (9 rural, 3 metropolitan, and 4 tertiary). Of these, 14,943 (73%) were linked with the VAED and VDI datasets. Among linked admissions, 9,890 consecutive patients had a documented CFS. However, 2,439 without relevant ICD-10 codes were excluded. A further 450 patients who were readmitted to the ICU during the same hospital stay were also excluded (Supplementary Fig. 1, http://links.lww.com/CCX/B72). The final study dataset comprised 7,001 patients from whom both the CFS and ICD-10mFI measures were available. Supplementary Table 4 (http://links.lww.com/CCX/B72) illustrates the comparison between included (n = 7,001) versus excluded (n = 13,457) patients. The included cohort appearing to be representative of the larger population with similar mortality, length of stay in ICU, and patient illness severity in both groups. There were significant variations in the level of CFS documentation in the 16 hospitals that reported on the CFS, but documentation of the CFS improved over time (17% at commencement, 85% at completion) (Supplementary Table 5, http://links.lww.com/CCX/B72).

Of the 7,001 patients, the overall mean (sd) age was 60.5 (17.7) years (median [IQR], 63.7 yr [49.1–74.0 yr]); 59.5% were male. Patients with frailty had poorer health outcomes, longer ICU, and hospital lengths of stay, and a lower proportion of these patients were discharged home compared with nonfrail patients. The median (IQR) follow-up time was 165 days (82–276 d). The demographic characteristics, illness severity scores, and proportion requiring mechanical ventilation are presented in Table 1.

Table 1.

Demographic Table Describing the Dichotomized Comparison Between Frail and Nonfrail Between the CFS and the ICD-10mFI

Variables	Overall, % (n)	Clinical Frailty Scale			ICD-10-Modified Frailty Index
		Nonfrail	Frail	p	Nonfrail	Frail	p
Number of patients, % (n)	7,001	81.1 (5,678)	18.9 (1,323)	—	91.2 (6,385)	8.8 (616)	—
Age (yr), mean (sd)	60.5 (17.7)	58.8 (17.7)	68.1 (15.5)	< 0.001	59.9 (18.0)	67.3 (11.4)	< 0.001
Male,% (n)	59.5 (4,166)	61.6 (3,495)	50.7 (671)	< 0.001	58.3 (3,722)	72.1 (444)	< 0.001
Hospital admission source,% (n)
Private residence	77.1 (5,397)	76.9 (4,367)	77.9 (1,030)	< 0.001	78.4 (5,004)	63.8 (393)	< 0.001
Transfer from nursing home	0.3 (22)	0.1 (8)	1.1 (14)		0.3 (17)	0.8 (5)
Transfer from rehabilitation	22.1 (1,545)	22.6 (1,283)	19.8 (262)		20.8 (1,330)	34.9 (215)
From transitional care program	0.0 (2)	0.0 (0)	0.0 (2)		0.0 (1)	0.1 (1)
No information	0.5 (35)	0.4 (20)	1.1 (15)		0.5 (33)	0.3 (2)
Hospital classification,% (n)
Tertiary	61.8 (4,326)	63.7 (3,616)	53.7 (710)	< 0.001	60.9 (3,886)	71.4 (440)	< 0.001
Metropolitan	11.6 (812)	10.4 (588)	16.9 (224)		11.3 (723)	14.4 (89)
Rural/regional	26.6 (1,863)	26.0 (1,474)	29.4 (389)		27.8 (1,776)	14.1 (87)
ICU admission type,% (n)
Elective surgery	24.9 (1,740)	27.0 (1,533)	15.6 (207)	< 0.001	24.3 (1,554)	30.2 (186)	< 0.001
Emergency surgery	20.2 (1,415)	19.9 (1,129)	21.6 (286)		20.8 (1,326)	14.4 (89)
Medical	54.9 (3,846)	53.1 (3,016)	62.7 (830)		54.9 (3,505)	55.4 (341)
Comorbidities,% (n)
Chronic respiratory disorder	7.7 (539)	5.0 (282)	19.4 (257)	< 0.001	7.2 (462)	12.5 (77)	< 0.001
Cardiovascular disorder	6.7 (470)	5.1 (288)	13.8 (182)	< 0.001	6.2 (399)	11.5 (71)	< 0.001
Chronic renal failure	3.7 (260)	2.9 (162)	7.4 (98)	< 0.001	3.5 (225)	5.7 (35)	0.007
Hepatic failure	0.7 (48)	0.6 (33)	1.1 (15)	0.040	2.6 (169)	1.9 (12)	0.30
Cirrhosis	2.6 (181)	2.3 (129)	3.9 (52)	0.001	5.8 (373)	3.6 (22)	0.020
Immunodeficiency	5.6 (395)	4.6 (261)	10.1 (134)	< 0.001	2.8 (176)	1.1 (7)	0.016
Metastatic cancer	2.6 (183)	2.1 (121)	4.7 (62)	< 0.001	0.8 (48)	0.6 (4)	0.78
Lymphoma	0.7 (52)	0.7 (37)	1.1 (15)	0.08	1.6 (104)	1.1 (7)	0.35
Leukemia	1.6 (111)	1.4 (81)	2.3 (30)	0.035	0 (0–1)	2 (1–3)	< 0.001
Charlson comorbidity index, median (interquartile range)	0 (0–2)	0 (0–1)	0 (0–2)	< 0.001	0 (0–1)	2 (1–3)	< 0.001
Pre-ICU (hr), median (interquartile range)	7.4 (3.8–20.1)	7.2 (3.8–18.2)	9.0 (4.3–27.5)	< 0.001	7.3 (3.8–17.4)	10.5 (4.4–80.3)	< 0.001
ICU admission post MET call	12.6 (885)	11.0 (625)	19.7 (260)	< 0.001	12.2 (780)	17.0 (105)	< 0.001
Treatment limitations	8.7 (607)	5.2 (296)	23.5 (311)	< 0.001	9.2 (584)	13.4 (83)	< 0.001
Scoring and risk of death scores, mean (sd)
Acute Physiology And Chronic Health Evaluation-III score	51.9 (24.7)	49.6 (24.3)	61.7 (24.0)	< 0.001	50.9 (24.6)	62.1 (23.7)	< 0.001
Australia New Zealand risk of death (%)	9.0 (17.0)	8.0 (16.0)	15.0 (20.0)	< 0.001	9.0 (16.6)	14.1 (19.0)	< 0.001
Organ failure and supports,% (n)
Invasive ventilation	46.7 (3,266)	47.9 (2,722)	41.1 (544)	< 0.001	45.1 (2,877)	63.1 (389)	< 0.001
Renal replacement therapy	6.8 (474)	6.1 (348)	9.5 (126)	< 0.001	6.0 (386)	14.3 (88)	< 0.001

MET = medical emergency team review.

Respiratory disorders: chronic restrictive, obstructive disease resulting in severe exercise restriction (unable to climb stairs or perform household duties), or documented chronic hypoxia, hypercapnia, secondary polycythemia, severe pulmonary hypertension (mean > 40 mm Hg); or ventilator dependency.

Cardiovascular: New York Heart Association Class IV: angina or symptoms at rest or on minimal exertion.

Liver: “Biopsy proven” cirrhosis and documented portal hypertension, or episodes of past upper GI bleeding attributed to portal hypertension.

Renal: Must be receiving chronic hemodialysis or peritoneal dialysis.

Immune suppressive disease (immune disease): Condition that is sufficiently advanced to suppress resistance to infection: leukemia, AIDS, lymphoma, severe autoimmune disease, or documented diffuse metastatic cancer.

Immunosuppressive therapy (immunosuppressed): The patient has received therapy that has suppressed resistance to infection: e.g., immunosuppression, chemotherapy within 4 wk of admission, radiation, high-dose steroid treatment (e.g., > 1.5 mg/kg methylprednisolone or equivalent for ≥ 5 d), and long-term treatment with > 20 mg/d steroid.

Numbers in bold imply statistical significance.

Comparison Between CFS and mFI

The number of patients classified as frail differed between the two frailty measures. The proportion of patients with frailty was higher with the CFS compared with the ICD-10mFI (18.9% vs 8.8%; p < 0.001) (Table 1) (Supplementary Fig. 2, http://links.lww.com/CCX/B72). There were significant variations in the proportion of each of the nine comorbidity measures between the CFS and the ICD-10mFI (Supplementary Table 6, http://links.lww.com/CCX/B72). Both functional status (dependent) and impaired sensorium (Supplementary eTable 6, http://links.lww.com/CCX/B72) were significantly lower with CFS. Hospital mortality was 9.2% (n = 642; CFS: nonfrail vs frail 7.6% vs 15.8%; p < 0.001 and ICD-10mFI: nonfrail vs frail 8.7% vs 14.3%; p < 0.001). The ICD-10mFI weakly correlated with the CFS (Spearman rho = 0.22) (Table 2) but had a poor agreement (kappa = 0.06). The ICD-10mFI was more strongly correlated with the CCI (rho = 0.30) than the CFS (rho = 0.25).

Table 2.

Association Between the Clinical Frailty Scale and the International Statistical Classification of Diseases and Related Health Problems, 10th revision Code-Derived Modified Frailty Index for All Patients

ICD-10mFI	Spearman Correlationa	Agreementb
	Correlation Coefficient (95% CI)	Kappa (95% CI)
All patients (n = 7,001)
ICD-10mFI with CFS	0.22 (0.19–0.24)	0.06 (0.04–0.08)
ICD-10mFI with CCI	0.30 (0.28–0.32)	0.08 (0.07–0.09)
CFS with CCI	0.25 (0.23–0.27)	0.06 (0.02–0.09)

CCI = Charlson comorbidity index, CFS = Clinical Frailty Scale, ICD-10mFI = International Classification of Diseases and Related Health Problems, 10th revision code-derived modified frailty index.

^aSpearman correlation based on continuous variables.

^bKappa agreement based on dichotomous variables.

Primary Outcome: Long-Term Survival

The categorical (nonfrail vs frail) unadjusted longest available mortality for the two frailty measures is summarized in Supplementary Table 7 (http://links.lww.com/CCX/B72). Cox proportional hazards regression (both CFS and ICD-10mFI as ordinal variables), adjusted for ANZROD and sex, demonstrated that the CFS (HR = 1.26; 95% CI, 1.21–1.31) was associated with long-term survival up to 6 months, but not the ICD-10mFI (HR = 1.04; 95% CI, 0.98–1.10) (Table 3). Kaplan-Meier curves for the two frailty measures demonstrated greater survival separation between nonfrail and frail patients for the CFS compared with the ICD-10mFI (Fig. 1).

Table 3.

Cox Proportional Hazards Regression Analysis, for 6-Month Survival, Adjusted for Australian and New Zealand Risk of Death and Sex, for All Patients

Covariates	Unadjusted	Adjusted
	HR (95% CI)	HR (95% CI)
Clinical Frailty Scale	1.43 (1.38–1.49)	1.25 (1.20–1.30)
Male sex	—	0.89 (0.78–1.01)
ANZROD	—	1.05 (1.04–1.05)
International Classification of Diseases, 10th revision modified frailty index	1.25 (1.18–1.32)	1.03 (0.97–1.10)
Male sex	—	0.97 (0.85–1.10)
ANZROD	—	1.05 (1.05–1.05)

ANZROD = Australian and New Zealand risk of death, HR = hazard ratio.

Numbers in bold imply statistical significance.

Figure 1.

Kaplan-Meier curves between frail and nonfrail patients (treated as a dichotomous variable) for Clinical Frailty Scale (CFS) and International Classification of Diseases, 10th revision code-derived modified frailty index (ICD-10mFI) for all patients and those stratified based on age.

Secondary Outcomes

The AUROC for the prediction of short-term (ICU, hospital, 28-d, and 90-d) mortalities and long-term (6 and 12 mo) mortalities demonstrated that the CFS was a consistently better predictor than the ICD-10mFI (Table 4) (Supplementary Fig. 3, http://links.lww.com/CCX/B72). However, after multivariable adjustment, neither frailty measurement was able to improve the discrimination provided by patient illness severity assessed by ANZROD. The multivariable logistic regression, adjusted for ANZROD and sex, demonstrated that the CFS independently predicted both the short-term and long-term mortalities, where applicable, but the ICD-10mFI only predicted long-term mortalities (Fig. 2) (Supplementary Table 8, http://links.lww.com/CCX/B72), but the magnitude of prediction was lower than the CFS prediction.

Table 4.

Comparison of Clinical Frailty Scale and Modified Frailty Index for Predicting Clinical Outcomes, for All Patients

Outcomes	Raw Mortality (n/N)a	Discrimination, C-Statistic (95% CI)b			Adjusted Discrimination, C-Statistic (95% CI)c
		CFS	ICD-10mFI	p	CFS	ICD-10mFI	p
All Patients (N = 7,001)
ICU mortality	472/6,994	0.60 (0.57–0.63)	0.54 (0.52–0.57)	0.001	0.92 (0.91–0.93)	0.92 (0.90–0.93)	0.07
Hospital mortality	642/7,000	0.62 (0.60–0.65)	0.56 (0.54–0.58)	< 0.001	0.91 (0.89–0.92)	0.90 (0.89–0.92)	0.80
28-d mortality	630/7,001	0.63 (0.61–0.65)	0.56 (0.54–0.59)	< 0.001	0.91 (0.89–0.92)	0.91 (0.90–0.92)	0.61
90-d mortality	828/7,001	0.65 (0.63–0.67)	0.57 (0.55–0.59)	< 0.001	0.88 (0.87–0.89)	0.89 (0.87–0.90)	0.12
6-mo mortality	944/7,001	0.66 (0.64–0.68)	0.58 (0.56–0.60)	< 0.001	0.88 (0.87–0.89)	0.88 (0.86–0.89)	0.76
1-yr mortality	1,005/7,001	0.66 (0.65–0.68)	0.58 (0.56–0.60)	< 0.001	0.87 (0.86–0.88)	0.87 (0.86–0.88)	0.72

CFS = Clinical Frailty Scale, ICD-10mFI = International Classification of Diseases, 10th revision code-derived modified frailty index.

^aOutcome based on CFS.

^bUnadjusted.

^cAdjusted for Australian and New Zealand risk of death and male sex using logistic regression models.

Numbers in bold imply statistical significance.

Figure 2.

Multivariable logistic regression, adjusted for Australia and New Zealand risk of death and sex, for short- and long-term mortality between Clinical Frailty Scale (CFS) and International Classification of Diseases, 10th revision code-derived modified frailty index (mFI) treated as a continuous variable for all patients and those stratified based on age.

There were 3,642 patients less than 65 years, 1,735 between 65 and 75 years and 1,624 greater than 75 years, respectively. The ICD-10mFI correlated weakly with the CFS and had poor agreements between the two frailty measures for all age groups (Supplementary Table 9, http://links.lww.com/CCX/B72). Cox proportion hazards regression, after adjustment for ANZROD and sex, demonstrated that although CFS was independently predictive of 6-month survival for all age groups, ICD-10mFI was not (Supplementary Table 10, http://links.lww.com/CCX/B72). The CFS demonstrated discrimination in differentiating survivors from those who died for short-term mortality (ICU, hospital, 28-d, and 90-d). However, this discrimination was greatest for long-term mortality for the less than 65 years age group. For the other age groups, the CFS demonstrated greater discrimination for both short- and long-term (6 and 12 mo) mortality and consistently better than the ICD-10mFI (Supplementary Table 11, http://links.lww.com/CCX/B72; Supplementary Fig. 3, http://links.lww.com/CCX/B72). However, after multivariable adjustment, neither frailty measurement was able to improve the discrimination provided by patient illness severity assessed by ANZROD for all age groups. In the multivariable logistic regression, adjusted for ANZROD and sex, the CFS independently predicted mortality at all study time points for patients in less than 65 and 65–75 years age groups respectively, and 90-day, 6-month, and 1-year mortality for the greater than 75 years age group. In contrast, the ICD-10mFI was not predictive of long-term mortality in any group (Supplementary Fig 5, http://links.lww.com/CCX/B72) (Supplementary Table 8, http://links.lww.com/CCX/B72). Kaplan-Meier survival curves for the two frailty measures demonstrated greater survival separation between nonfrail and frail patients for the CFS compared with the ICD-10mFI for all age groups (Supplementary Fig 6, http://links.lww.com/CCX/B72).

Sensitivity Analysis

By including the ICD-10-U82.3 (hypertension), the number of patients with hypertension diagnosis increased from 850 to 3,235. Consequently, the total number of patients with frailty by ICD-10mFI increased to 1,091 (15.6%). The comparison between ICD-10mFI with and without the ICD-10-U82.3, including unadjusted 1-year mortality and discharge destinations, is summarized in Supplementary Table 12 (http://links.lww.com/CCX/B72). The Spearman correlation and Kappa agreement were similar between the CFS and ICD-10mFI with and without the ICD-10-U82.3 (Supplementary Table 13, http://links.lww.com/CCX/B72). Cox proportional hazards regression for ICD-10mFI with ICD-10-82.3, adjusted for ANZROD and sex, was not associated with the longest available survival (6 mo) (HR = 1.03; 95% CI, 0.98–1.09) (Supplementary Table 14, http://links.lww.com/CCX/B72). A comparison of the ICD-10mFI with and without ICD-10-82.3 demonstrated identical AUROC for predicting outcomes (Supplementary Table 15, http://links.lww.com/CCX/B72). Compared with the ICD-10mFI with the ICD-10-U82.3, the Kaplan-Meier survival curves for the ICD-10mFI without ICD-10-82.3 demonstrated a greater survival separation between nonfrail and frail patients (Supplementary Fig. 4, http://links.lww.com/CCX/B72).

DISCUSSION

This large multicenter retrospective cohort study demonstrated that the proportion of patients with frailty was significantly higher with the CFS compared with the ICD-10mFI as quantified by the Delphi consensus. There was poor agreement between the two frailty measures. The CFS predicted long-term survival, independent of comorbidities and baseline illness severity on ICU admission, whereas the ICD-10mFI did not. The Kaplan-Meier survival curves for the two frailty measures demonstrated greater survival separation between nonfrail and frail patients for the CFS, compared with the ICD-10mFI. The unadjusted AUROC demonstrated that the CFS was a consistently better long-term mortality predictor than the ICD-10mFI. However, neither frailty measure was able to clinically improve upon the predictability provided by baseline patient illness severity as assessed by ANZROD. Finally, the long-term survival prediction of CFS was marginally lower for older patients. There were no differences for the very old patients when compared with their younger counterparts.

The prevalence of frailty as estimated by the ICD-10mFI was lower when compared with the CFS and considerably lower than in a recent study (25). This is probably because the authors mapped the mFI variables from their large ICU Frailty database. In contrast, the ICD-10mFI-categorized frailty prevalence in our study was comparable with the frailty prevalence observed in a large Brazilian cohort (26) that used a commercial ICU database (Epimed-Monitor) with a specific structured library of diagnoses and comorbidities (27, 28).

A recent post hoc analysis of a multicenter study (6) demonstrated good discrimination and predictive validity between the CFS and mFI despite low concordance between the two measures (25). Contrary to recently published studies (5, 25, 26), we found poor discrimination and agreement between the CFS and the ICD-10mFI. Some factors may explain these differences. In particular, we used ICD-10 codes to estimate mFI. The prevalence of frail categorization by both the CFS and the ICD-10mFI was lower, and the Kappa agreement was significantly lower. Furthermore, we allocated a point for all 11 variables as in the original study (26), whereas other investigators did not allocate points for previous myocardial infarction (MI) or coronary intervention in their coding model. Given the higher combined frequency of MI or coronary intervention in our study, we believe that it was appropriate to allocate 1 point for each to minimize concordance bias. Finally, the 1-year mortality of frail patients in our study was significantly lower than previously reported for both frailty measures.

Higher age, comorbidity burden, and primary diagnosis have been associated with decreased survival in all patient groups following ICU admission (29, 30). The CFS has been validated to predict short-term and long-term mortality in critically ill patients (6, 12, 31-35). The CFS demonstrated moderate discrimination in identifying survivors, but the ICD-10mFI did not.

The comorbidity composite measure of the mFI does not assess other essential frailty domains such as cognitive impairment, communication, mood, continence, nutrition, and medications (36) and its ability to truly capture frailty has been questioned (36, 37). Seven of the 11 ICD-10mFI variables feature in the CCI. Frailty as a concept is independent of comorbidities and acute illness severity. However, the ICD-10mFI integrates the measurement of performance status, a core domain in assessing patients at risk of frailty. Although mFI is inexpensive and construed as oversimplistic (36), its core performance status domain measure may be beneficial in screening patients at risk for frailty (25).

Our group recently published a study using the same dataset comparing another administrative frailty measure, the HFRS, with CFS. We found both HFRS and CFS independently predicted up to 1-year survival following ICU admission with moderate discrimination (14). When compared with ICD-10mFI, HFRS better discriminated patients who died than those who survived. Although both frailty measures independently predicted long-term mortality, their magnitude of prediction was considerably lower than for the CFS. Furthermore, the HFRS was a better measure in very old patients in predicting 90-day, 6-month, and 1-year mortalities, whereas ICD-10mFI was not (Supplementary Table 16, http://links.lww.com/CCX/B72). This suggests that, if administrative scores were to be used to determine frailty, HFRS has greater validity in predicting long-term mortality, than ICD-10mFI.

The ICD-10mFI underestimated the prevalence of frailty, compared with the CFS. The ICD-10mFI correlated weakly and had a poor agreement with the CFS but had a relatively stronger correlation with the CCI. The CFS independently and better predicts long-term survival, whereas the ICD-10mFI does not. However, both frailty measures were unable to clinically improve upon the predictability provided by baseline acute illness severity. Therefore, based on our findings, caution must be used in using the ICD-10mFI to screen frailty in critically ill patients. In contrast, despite the requirement of resource intensive training, the clinically assessed CFS has some validity for differentiating long-term survivors from those who die, even before the calculation of acute illness severity. Recently published CFS scoring classification tree could further reliably assist for relatively inexperienced raters (38). The mFI is relatively newer frailty measure with some validity but is not widely available. In contrast, the ICD-10mFI is a new concept with great potential because of the global availability of ICD-10 coding. However, our findings suggest that ICD-10mFI in its current form should not be used.

The strengths of the study included a direct comparison of CFS and ICD-10mFI in a large patient dataset with long-term outcomes. This was the first study to use Delphi consensus-derived ICD-10mFI. Furthermore, it adjusted for key confounders, compared the frailty measures with CCI, and performed sensitivity analyses.

A few limitations need to be acknowledged. First, the ICD-10mFI scores were approximated from the Delphi consensus (20), where a panel of intensivists and geriatricians reached a consensus on the ICD-10 codes required to estimate the mFI, in contrast to direct scoring. Second, our ICD-10mFI may not be related to the proprietary mFI (27, 28) Third, the CFS measures frailty 2 months before ICU admission, whereas the mFI was measured it using ICD-10-based codes upon the index admission. Fourth, there is a possibility of the wrong categorization due to incorrect ICD-10 coding. This could have affected the frailty status of patients (39). Fifthly, we were unable to determine interrater reliability for CFS in our study. However previous studies showed that the interrater reliability was strong for CFS (38, 40, 41). Sixthly, we expect, but cannot prove, that ICD-10 coding practices are generalizable across Victoria. Finally, the lower ICU mortality rates and shorter ICU length of stay are real-world outcomes for ANZ ICUs and may not apply to other countries.

CONCLUSIONS

This retrospective multicenter cohort study found that the CFS, but not the ICD-10mFI, predicted long-term survival in ICU patients. The findings suggest that the CFS had greater validity in discriminating 6-month survivors from those who die. However, neither frailty measure could improve upon the prediction performance of baseline patient illness severity.

ACKNOWLEDGMENT

We acknowledge the contribution of Mr. Cameron Green, Mr. Brendan McAlister, and Dr Scott May of the “Frailtrail” team from the Bendigo Datathon 2018. The authors and the Australian and New Zealand Intensive Care Society CORE management committee would like to thank clinicians, data collectors, and researchers at the following contributing sites: Alfred Hospital, Austin Hospital, Bendigo Healthcare Group, Box Hill Hospital, Central Gippsland Health Service, Dandenong Hospital, Frankston Hospital, Goulburn Valley Health, Latrobe Regional Hospital, Monash Medical Centre-Clayton Campus, Northeast Health Wangaratta, South West Healthcare (Warrnambool), University Hospital Geelong, Western District Health Service (Hamilton), and Wimmera Healthcare Group (Horsham).