Abstract
Background
The LACE index—length of stay (L), acuity (A), Charlson co‐morbidities (C), and emergent visits (E)—predicts 30‐day outcomes following heart failure (HF) hospitalization but is complex to score. A simpler LE index (length of stay and emergent visits) could offer a practical advantage in point‐of‐care risk prediction.
Methods and results
This was a sub‐study of the patient‐centred care transitions in HF (PACT‐HF) multicentre trial. The derivation cohort comprised patients hospitalized for HF, enrolled in the trial, and followed prospectively. External validation was performed retrospectively in a cohort of patients hospitalized for HF. We used log‐binomial regression models with LACE or LE as the predictor and either 30‐day composite all‐cause readmission or death or 30‐day all‐cause readmission as the outcomes, adjusting only for post‐discharge services. There were 1985 patients (mean [SD] age 78.1 [12.1] years) in the derivation cohort and 378 (mean [SD] age 73.1 [13.2] years) in the validation cohort. Increments in the LACE and LE indices were associated with 17% (RR 1.17; 95% CI 1.12, 1.21; C‐statistic 0.64) and 21% (RR 1.21; 95% CI 1.15, 1.26; C‐statistic 0.63) increases, respectively, in 30‐day composite all‐cause readmission or death; and 16% (RR 1.16; 95% CI 1.11, 1.20; C‐statistic 0.64) and 18% (RR 1.18; 95% CI 1.13, 1.24; C‐statistic 0.62) increases, respectively, in 30‐day all‐cause readmission. The LE index provided better risk discrimination for the 30‐day outcomes than did the LACE index in the external validation cohort.
Conclusions
The LE index predicts 30‐day outcomes following HF hospitalization with similar or better performance than the more complex LACE index.
Keywords: Heart failure, Risk prediction, Readmission
Introduction
Heart failure (HF) is a common cause of hospitalization in older adults. The period after hospitalization is one of high risk, when patients commonly experience decompensation, readmission, and death. 1 Risk prediction can help identify patients who are at high risk of readmission or death and who may benefit from close follow‐up. Whereas several clinical prognostic models have been developed to predict the risk of readmission or death following hospitalization for HF, most rely on retrospective administrative data and are difficult to compute at the point of care.
The LACE index (length of stay, acuity, Charlson co‐morbidity index, and number of emergency department [ED] visits within 6 months) was derived using administrative data from patients who were discharged to the community after a medical or surgical admission. 2 In this population, the LACE index, with a threshold score of 10, predicted the risk of 30‐day composite all‐cause readmission or death with reasonable discrimination (C‐statistic 0.68). The LACE index has also been validated for use among patients hospitalized for HF, with modest risk discrimination for 30‐day composite all‐cause readmission or death (C‐statistic 0.57) 3 and an optimal threshold of 13. The LACE index is currently used to identify at‐risk patients in Ontario and other provinces to plan services following hospital discharge. 4
Although the LACE index is widely used, its application at the point of care is limited by its complexity. The Charlson (C) co‐morbidity index includes 19 variables and requires a chart review and computational aide. It is possible that components such as acuity and Charlson co‐morbidity do not provide discriminative information among patients hospitalized for HF. These patients typically present for hospitalization via the ED, receiving a point for acuity, and are multi‐morbid, receiving several points for co‐morbidities including HF. A simplified score that omits acuity and co‐morbidities would represent a practical step forward in risk prediction.
In this sub‐study of the patient‐centred care transitions in HF (PACT‐HF) trial, 5 we assessed whether the LE index, a sum of length of hospital stay (in days) and number of ED visits in the preceding 6 months, can predict 30‐day composite all‐cause readmission or death and 30‐day all‐cause readmission with comparable performance to the LACE index. We externally validated the LE index and determined an optimal threshold to stratify risk.
Methods
Study design
This was a prospective cohort sub‐study of a multicentre clinical trial.
Study population
The derivation cohort was a subset of patients hospitalized for HF and enrolled in the multicentre PACT‐HF stepped wedge cluster randomized trial from February 2015 to March 2016. 5 , 6 The study was approved by the research ethics boards of all participating institutions with a waiver for written consent. Eligible patients provided informed verbal consent to participate in the study. This study complies with the Declaration of Helsinki. This study follows the STROBE guidelines for reporting observational data (Supplementary file). The patients in the sub‐study comprised a group who received transitional care services in hospital and following discharge. 5 Transitional care services included nurse‐led patient education and structured home visits. We included patients with a diagnosis of HF on admission, confirmed by the nursing case manager with either the Boston criteria 7 or natriuretic peptide (BNP/NT‐pro‐BNP) levels who were discharged between March 2015 and March 2016. We excluded patients who did not meet the diagnostic criteria for HF, who died during hospitalization, or who were discharged to another hospital.
The external validation was performed retrospectively in a separate cohort of patients hospitalized for HF and enrolled in the PACT‐HF pilot study from March 2012 to July 2013. 3 We excluded patients who did not meet the diagnostic criteria for HF, who died during hospitalization, or who were discharged to another hospital.
Data collection
We obtained clinical characteristics for analysis from hospital charts and administrative databases held at ICES. ICES is an independent, non‐profit research institute whose legal status under Ontario’s health information privacy law allows it to collect and analyze health care and demographic data, without consent, for health system evaluation and improvement. We used a 5‐year look‐back period to identify baseline co‐morbidities. We identified inpatient services using the Canadian Institute for Health Information Discharge Abstract Database (CIHI‐DAD) and deaths using the Ontario Registered Persons Database (ORPD). These datasets were linked using unique encoded identifiers and analysed at ICES.
Outcomes
Outcomes included 30‐day composite all‐cause readmission or death and 30‐day all‐cause readmission.
Statistical analysis
We reported continuous variables using means (SD) and medians (IQR) and categorical variables using numbers and percentages.
We used log‐binomial regression models with either the LACE or LE index as the predictor, adjusted for receipt of post‐discharge home care or HF clinic services. We reported relative risk (RR) and 95% confidence intervals (CI). We compared the ability of the LACE and LE indices to discriminate between patients at risk of 30‐day outcomes using the C‐statistic. 8 We assessed the model goodness of fit using the Brier score. 8 , 9 , 10 The Brier score is a measure of the accuracy of predictive models for binary outcomes, with a score of 0 indicating perfect accuracy and a score of 1 indicating the model is completely inaccurate. 8
Additional analyses
We fitted the same model for both 30‐day composite all‐cause readmission or death and 30‐day all‐cause readmission using dichotomized LE index values above and below various cut‐points. We computed the RR (95% CI), C‐statistic (95% CI), sensitivity, specificity, positive likelihood ratio (PLR), and negative likelihood ratio (NLR) for each model. We used the closest to (0,1) criterion—defined as the minimum Euclidean distance to (0,1), the point at which sensitivity and specificity are maximized on the receiver operating characteristic (ROC) curve for a given outcome 10 —to determine the optimal threshold for the LE index. 11
We internally validated each model with 100 bootstrapping resamples and computed the optimistic C‐statistic. We externally validated the LE index by applying the model that was developed in the derivation cohort in a separate cohort of patients and assessed model performance in predicting each of the 30‐day outcomes with the C‐statistic and the Brier score. 8
We conducted analyses using SAS Version 9.4 for UNIX (SAS Institute Inc., Cary, NC) and set the nominal significant level for testing at 5%.
Results
Baseline characteristics
There were 1985 patients in the derivation cohort, and their baseline characteristics are presented in Table 1 . The mean (SD) age of patients was 78.1 (12.1) years, and 49.4% were female. The majority of patients (99.0%) presented for hospitalization with high acuity (via the ED). Among patients in the derivation cohort, 50.8% had diabetes, 75.9% had hypertension, and 29.2% had prior PCI or CABG. All patients received in‐hospital transitional care services, and 47.7% of patients were referred for post‐discharge home care or HF clinic services.
Table 1.
Clinical characteristics of the derivation and external validation cohorts
| Characteristics | Derivation cohort (n = 1985) | Validation cohort (n = 378) |
|---|---|---|
| Demographics | ||
| Age (year), mean (SD) | 78.1 (12.1) | 73.1 (13.2) |
| Sex, n (%) | ||
| Male | 1004 (50.6) | 214 (56.6) |
| Women | 981 (49.4) | 164 (43.4) |
| Resides in long‐term care, n (%) | 160 (8.1) | 30 (7.9) |
| Co‐morbidities | ||
| Left ventricular ejection fraction, n (%) | 48.0 (14.5) | 40.9 (14.5) |
| Hypertension, n (%) | 1506 (75.9) | 288 (76.2) |
| Atrial fibrillation, n (%) | 1118 (56.3) | 199 (52.6) |
| Diabetes, n (%) | 1008 (50.8) | 197 (52.1) |
| Chronic kidney disease, n (%) | 498 (25.1) | 122 (32.3) |
| Prior PCI or CABG, n (%) | 580 (29.2) | 96 (25.4) |
| Chronic pulmonary disease, n (%) | 488 (24.6) | 85 (22.5) |
| Peripheral vascular disease, n (%) | 214 (10.8) | 62 (16.4) |
| Cerebrovascular disease, n (%) | 208 (10.5) | 76 (20.1) |
| Dementia, n (%) | 194 (9.8) | 42 (11.1) |
| Liver disease, n (%) | 65 (3.3) | 9 (2.4) |
| Cancer (any), n (%) | 205 (10.3) | 53 (14.0) |
| Resource utilization | ||
| High acuity admission (via ED), n (%) | 1966 (99.0) | 378 (100.0) |
| Number of ED visits in preceding 6 months, mean (SD) | 2.5 (2.0) | 1.5 (2.1) |
| Length of stay, mean (SD) | 9.9 (11.0) | 8.2 (6.8) |
| Charlson co‐morbidity index, mean (SD) | 4.4 (2.5) | 4.1 (1.9) |
| Estimated risk | ||
| LE index, mean (SD) | 6.8 (2.0) | 5.7 (1.9) |
| LACE index, mean (SD) | 13.6 (2.7) | 12.7 (2.4) |
CABG, coronary artery bypass grafting; ED, emergency department; LACE, length of stay, acuity, Charlson co‐morbidity index, and number of emergency department visits in the prior 6 months; LE, length of stay and number of emergency department visits in the prior 6 months; PCI, percutaneous coronary intervention.
Outcomes
Of the 1985 patients in the derivation cohort, 458 patients (23.1%) were readmitted or died, and 416 (21%) were readmitted within 30 days of discharge. The mean (SD) LACE index was 14.6 (2.7) among those who were readmitted or died and 13.3 (2.7) among those who were neither readmitted nor died within 30 days. The mean (SD) LE index was 7.5 (2.1) for those who were readmitted or died and 6.6 (1.9) for those who were neither readmitted nor died within 30 days. The scoring of the LE index is described in Figure 1 .
Figure 1.
Scoring and performance of the LE index to predict 30‐day outcomes in patients hospitalized for HF. In a sub‐study of the Patient‐centred Care Transitions in Heart Failure multicentre trial, the LE index, comprising length of stay (L) and number of ED visits in preceding 6 months (E), predicted 30‐day outcomes in patients hospitalized for HF with similar performance as the more complex LACE index. The index was externally validated in a separate cohort.
30‐day composite all‐cause readmission or death
Each increment in the LACE index was associated with a 17% increase in 30‐day composite all‐cause readmission or death (RR 1.17; 95% CI 1.12, 1.21 per unit; C‐statistic 0.64; 95% CI 0.61, 0.67) (Table 2 ). Each increment in the LE index was associated with a 21% increased risk of 30‐day composite all‐cause readmission or death (RR 1.21; 95% CI 1.15, 1.26 per unit; C‐statistic 0.63; 95% CI 0.59, 0.66). The LACE (Brier score 0.17) and LE (Brier score 0.17) indices were well calibrated for 30‐day composite all‐cause readmission or death.
Table 2.
Performance of the LACE and LE indices in predicting 30‐day all‐cause readmission and 30‐day composite all‐cause readmission or death in the derivation (n = 1985) and external validation (n = 387) cohorts
| Risk index | N with events (% of total) | Relative risk (95% CI) a | C‐statistic (95% CI) b | Brier score c | Optimistic C‐statistic (95% CI) d |
|---|---|---|---|---|---|
| 30‐day composite all‐cause readmission or death | |||||
| LACE | 458 (23%) | 1.17 (1.12, 1.21) | 0.64 (0.61, 0.67) | 0.17 | 0.64 (0.61, 0.67) |
| LE | 458 (23%) | 1.21 (1.15, 1.26) | 0.63 (0.59, 0.66) | 0.17 | 0.63 (0.59, 0.66) |
| LE e | 105 (28%) | 1.18 (1.11, 1.27) | 0.64 (0.58, 0.70) | 0.19 | NA |
| 30‐day all‐cause readmission | |||||
| LACE | 416 (21%) | 1.16 (1.11, 1.20) | 0.64 (0.60, 0.68) | 0.16 | 0.64 (0.60, 0.68) |
| LE | 416 (21%) | 1.18 (1.13, 1.24) | 0.62 (0.60, 0.66) | 0.16 | 0.62 (0.59, 0.66) |
| LE e | 97 (26%) | 1.21 (1.13, 1.31) | 0.66 (0.60, 0.72) | 0.18 | NA |
CI, confidence interval; LACE, length of stay, acuity, Charlson co‐morbidity index, and number of emergency department visits in the prior 6 months; LE, length of stay and number of emergency department visits in the prior 6 months.
Adjusted for post‐discharge services.
The C‐statistic is a measure of discrimination from 0 to 1 with higher scores indicating stronger discrimination.
The Brier score is a measure of model accuracy from 0 to 1, with lower scores indicating higher accuracy.
The optimism adjusted C‐statistic is the result of internal validation by bootstrapping with 100 samples.
External validation cohort.
30‐day all‐cause readmission
Each increment in the LACE index was associated with a 16% increase in 30‐day all‐cause readmission (RR 1.16; 95% CI 1.11, 1.20 per unit; C‐statistic 0.64; 95% CI 0.60, 0.68). Increments in the LE index were associated with an 18% increased risk for 30‐day all‐cause readmission (RR 1.18; 95% CI 1.13, 1.24 per unit; C‐statistic 0.62; 95% CI 0.60, 0.66). The LACE (Brier score 0.16) and LE (Brier score 0.16) indices were well calibrated for 30‐day all‐cause readmission.
Optimal threshold for the LE index
A LE index of ≥7 (vs. <7) was associated with an 86% increased risk of 30‐day composite all‐cause readmission or death (RR 1.86; 95% CI 1.55, 2.22; C‐statistic 0.60; 95% CI 0.57, 0.62) and a 73% increased risk of 30‐day all‐cause readmission (RR 1.73; 95% CI 1.43, 2.10; C‐statistic 0.60; 95% CI 0.57, 0.63). Using the closest to (0,1) criterion, we suggest a LE index threshold of ≥7 based on the minimum distance to (0,1) on the ROC curve (Figure 2 ). A score ≥7 had a sensitivity of 0.67, a specificity of 0.52, a PLR of 1.40, and a NLR of 0.63 for 30‐day composite all‐cause readmission or death. A score ≥7 yielded a sensitivity of 0.66 and a specificity of 0.52, a PLR of 1.37, and a NLR of 0.65 for 30‐day all‐cause readmission. The performance of the LE index at various cut‐points is depicted in Table 3 .
Figure 2.
Receiver operating characteristic curve analysis of LE index thresholds in predicting 30‐day composite all‐cause readmission or death in 1985 patients hospitalized for HF. A threshold LE index of ≥7 has optimal sensitivity and specificity for predicting 30‐day composite all‐cause readmission or death in patients hospitalized for HF.
Table 3.
Performance of the LE index at various cut‐points among 1985 patients hospitalized for heart failure
| LE cut‐point | Event rate (#events/n) (%) | Relative risk (95% CI) a | C‐statistic (95% CI) b | Sensitivity | Specificity | LR+ | LR− | Minimum distance c |
|---|---|---|---|---|---|---|---|---|
| 30‐day composite all‐cause readmission or death (458 events) | ||||||||
| ≥6 | 388/1478 (26%) | 1.88 (1.48, 2.39) | 0.57 (0.55, 0.60) | 0.85 | 0.29 | 1.19 | 0.53 | 0.73 |
| vs. < 6 | 70/507 (14%) | |||||||
| ≥7 | 306/1034 (30%) | 1.86 (1.55, 2.22) | 0.60 (0.57, 0.62) | 0.67 | 0.52 | 1.40 | 0.63 | 0.58 |
| vs. < 7 | 152/951 (16%) | |||||||
| ≥8 | 235/737 (32%) | 1.78 (1.51, 2.10) | 0.59 (0.57, 0.62) | 0.51 | 0.67 | 1.56 | 0.73 | 0.59 |
| vs. < 8 | 223/1248 (18%) | |||||||
| ≥9 | 146/390 (37%) | 1.89 (1.60, 2.24) | 0.59 (0.56, 0.62) | 0.32 | 0.84 | 1.99 | 0.81 | 0.70 |
| vs. < 9 | 312/1595 (20%) | |||||||
| 30‐day all‐cause readmission (416 events) | ||||||||
| ≥6 | 351/1478 (24%) | 1.73 (1.35, 2.23) | 0.59 (0.56, 0.62) | 0.84 | 0.28 | 1.17 | 0.55 | 0.74 |
| vs. < 6 | 65/507 (13%) | |||||||
| ≥7 | 276/1034 (27%) | 1.73 (1.43, 2.09) | 0.60 (0.57, 0.63) | 0.66 | 0.52 | 1.37 | 0.65 | 0.59 |
| vs. < 7 | 140/951 (15%) | |||||||
| ≥8 | 211/737 (29%) | 1.66 (1.39, 1.98) | 0.60 (0.57, 0.63) | 0.51 | 0.66 | 1.51 | 0.74 | 0.60 |
| vs. < 8 | 205/1248 (16%) | |||||||
| ≥9 | 133/390 (34%) | 1.83 (1.53, 2.19) | 0.60 (0.57, 0.63) | 0.32 | 0.84 | 1.95 | 0.81 | 0.70 |
| vs. < 9 | 283/1595 (18%) | |||||||
CI, confidence interval; LE, length of stay and number of emergency department visits in the prior 6 months; LR+, positive likelihood ratio; LR−, negative likelihood ratio.
Adjusted for post‐discharge care services.
The C‐statistic is a measure of discrimination from 0 to 1 with higher scores indicating stronger discrimination.
Minimum distance refers to the minimum distance to the (0,1) point on the receiver operating characteristic curve.
Internal validation
In the internal validation with 100 bootstrap samples, the LACE index had moderate discrimination for 30‐day composite all‐cause readmission or death (optimistic C‐statistic 0.64; 95% CI 0.61, 0.67) and 30‐day all‐cause readmission (optimistic C‐statistic 0.64; 95% CI 0.60, 0.68). The LE index had moderate discrimination for both 30‐day composite all‐cause readmission or death (C‐statistic 0.63; 95% CI 0.59, 0.66) and 30‐day all‐cause readmission (optimistic C‐statistic 0.62; 95% CI 0.59, 0.66).
External validation
There were 378 patients in the external validation cohort, and their baseline characteristics are presented in Table 1 . The mean (SD) age was 73.1 (13.2) years, and 43.4% of all patients were female. Of the 378 patients, 76.2% had hypertension, 52.1% had diabetes, and 25.4% had prior PCI/CABG. In this cohort, 97 (26%) were readmitted, 17 (4.5%) died, and 105 (28%) were either readmitted or died within 30 days of discharge.
The LACE index had modest discrimination for 30‐day composite all‐cause readmission or death (C‐statistic 0.57; 95% CI 0.51, 0.64) and 30‐day all‐cause readmission (C‐statistic 0.59; 95% CI 0.52, 0.65). The LE index had greater discrimination than the LACE index for 30‐day composite all‐cause readmission or death (C‐statistic 0.64; 95% CI 0.58, 0.70) and 30‐day all‐cause readmission (C‐statistic 0.66; 95% CI 0.60, 0.72). Each increment in the LE index predicted an 18% increased risk of 30‐day composite all‐cause readmission or death (RR 1.18; 95% CI 1.11, 1.27) (Table 2) and a 21% increased risk of 30‐day all‐cause readmission (RR 1.21; 95% CI 1.13, 1.31). The LE index was well calibrated for 30‐day all‐cause readmission or death (Brier score 0.19) and 30‐day all‐cause readmission (Brier score 0.18).
Discussion
In this sub‐study of the multicentre PACT‐HF trial of patients hospitalized for HF, we found that the LE index—a sum of the length of hospital stay and number of ED visits in the preceding 6 months—predicted 30‐day composite all‐cause readmission or death and 30‐day all‐cause readmission following hospital discharge. Risk discrimination using the LE index was as good as the LACE index, with good model fit in the derivation cohort. Compared to the derivation cohort, the external validation cohort of patients hospitalized for HF 2 years prior to the PACT‐HF trial were younger and had a slightly lower burden of co‐morbidities. Risk discrimination of the LE index, as measured by the C‐statistic, was better than that of the LACE index in the validation cohort for both 30‐day composite all‐cause readmission or death and 30‐day all‐cause readmission.
The simplicity of the LE index—without loss of model performance—is a major advantage over other risk prediction models following hospitalization for HF. The LACE index requires computation of the Charlson co‐morbidity index, which is difficult to estimate at the bedside and is not a robust predictor of outcome in patients admitted for HF. 13 Other risk prediction models in patients hospitalized for HF include the Enhanced Feedback for Effective Cardiac Treatment (EFFECT) risk score, 12 which includes 11 variables, and the Medicare‐endorsed Readmission Risk Score (RRS), 13 which includes 37 variables. The LE index can be used at the point of care on the day of hospital discharge to identify high‐risk patients who may benefit from closer follow‐up. The ease of use of the LE index may increase the number of referrals of eligible, vulnerable patients to additional HF services post‐discharge.
The discrimination of the LE index was modest but comparable or better than previously established risk prediction models tested in patients hospitalized for HF. The LE index had similar performance as the LACE index in the derivation cohort and better discrimination than the LACE index in the validation cohort. The LE index performs comparably to other risk scores such as the RRS (C‐statistic 0.62 for 30‐day composite all‐cause readmission or death; 0.61 for 30‐day all‐cause readmission) 13 as well as a readmission risk model derived from the Get With The Guidelines (GWTG) dataset (C‐statistic 0.59 for 30‐day composite all‐cause readmission or death). 14 Among patients hospitalized for HF in the GWTG HF registry, machine learning approaches demonstrated modest C‐statistics of 0.59 to 0.62 for 30‐day all‐cause readmission. 15 Thus, the LE index represents an important advantage over existing readmission risk prediction models, offering ease of use without loss of discrimination. The modest risk discrimination offered by risk prediction models in HF for early readmissions—with C‐statistics typically <0.65 even for the most complex models—remains a challenge.
We derived and validated the LE index in hospitalized patients who have a higher baseline risk than ambulatory patients. The performance of this index cannot be compared to risk models derived in ambulatory patients, who have a wider spectrum of risk and therefore generate greater risk discrimination when included in models. 16 To date, a majority of HF risk models were derived and validated in the ambulatory setting, most are derived from administrative data, and most predict 30‐day HF‐specific rather than all‐cause readmission. 17 Although all‐cause readmission is more challenging to predict, it is a more relevant endpoint as up to two‐thirds of readmissions following hospitalization for HF are for causes other than HF. 18 , 19
Risk prediction models that include readmission as an endpoint have lower risk discrimination than those with mortality alone, 22 as readmission may be related to unmeasured variables such as clinical judgement, availability of hospital beds, 20 socio‐economic status, and quality of post‐discharge care. 14 A more complex risk prediction model derived from administrative data offered the best discrimination for 30‐day mortality (C‐statistic 0.75), modest discrimination for 30‐day composite readmission or death (C‐statistic 0.62), and low discrimination for 30‐day readmission (C‐statistic 0.59). 21
Although the optimal threshold for the LE index to establish risk of the 30‐day outcomes was 7, risk discrimination was modest (C‐statistic 0.60). A threshold of ≥7 met by 52% of patients in our derivation cohort identified 67% of patients who were readmitted or died within 30 days. The performance of the threshold of ≥7 is better than the established threshold for the LACE index in patients admitted for HF, ≥13, which identified 62% of patients who were readmitted or died within 30 days. 3 A threshold of ≥7 could be used as a more reliable and practical criterion for referral to additional HF services than the LACE index. 4
Our study has several strengths. First, the LE risk index was derived from a reasonably large, multicentre trial in which variations in post‐discharge care were accounted for in the model. Second, due to the limited exclusion criteria, the study population—elderly and with multiple medical co‐morbidities—was representative of clinical practice. Third, the diagnosis of HF was confirmed by research nurses using validated criteria, avoiding limitations associated with administrative data where up to 20% of patients may be incorrectly classified as having or not having HF. 22 Fourth, our predictor index can be computed at the point of care without extensive chart review, improving its relevance for clinical use. Lastly, the results were externally validated in a separate population of adults hospitalized for HF.
Limitations
Though our model was adjusted for the post‐discharge transitional care services received and HF clinic follow‐up, we did not adjust for other clinical characteristics. This is because our goal was to develop and validate a simple, practical risk index without the need for data extraction and computational aids that can limit uptake at the point of care. We did not externally validate the optimal threshold for the LE index as the cut‐point analysis was exploratory. The performance of the LE index at each cut‐point should be interpreted with caution in light of this limitation.
Conclusions
The simplified LE index is preferable to the LACE index for predicting 30‐day composite all‐cause readmission or death and 30‐day all‐cause readmission in patients admitted for HF. It is easy to compute on the day of discharge, does not require knowledge of clinical variables, has as good or better discrimination than the LACE index, and retains the performance of more complex risk prediction models. Although there is a continuum of risk, a score of ≥7 may be used as an optimal threshold for predicting 30‐day composite all‐cause readmission or death and 30‐day all‐cause readmission.
Conflict of interest
Dr Van Spall has received research salary support from Ontario's Ministry of Health, Hamilton Health Sciences Career Award, Women As One Escalator Award, and McMaster University Department of Medicine. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
Funding
This work was funded by the Ontario Ministry of Health and Long‐Term Care (MOHLTC) grant 6686 and Canadian Institutes of Health Research grant 135917 held by Dr Van Spall. The work was supported by ICES, which is funded by an annual grant from the MOHLTC. Parts of this material are based on data and information compiled and provided by the MOHLTC and the Canadian Institute for Health Information (CIHI). The analyses, conclusions, opinions, and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred.
Supporting information
Data S1. Supporting information
Acknowledgements
HGCV conceived the study idea, obtained funding, led the conduct of the study, informed the analytic plan, interpreted the results, and drafted and edited the manuscript. TA interpreted the results and drafted and edited the manuscript. SL, UEO, and RP performed the statistical analysis. SL and MAM interpreted the results and edited the manuscript. SJC and DTK reviewed the manuscript. HGCV assumes responsibility for the scientific integrity of this work. All involved authors approved the final article. The support of Anastasia Gayowsky in statistical programming is acknowledged. Parts of this report are based on Ontario Registrar General (ORG) information on deaths, the original source of which is ServiceOntario. The views expressed therein are those of the author and do not necessarily reflect those of ORG or the Ministry of Government Services.
Averbuch, T. , Lee, S. F. , Mamas, M. A. , Oz, U. E. , Perez, R. , Connolly, S. J. , Ko, D. T.‐W. , and Van Spall, H. G. C. (2021) Derivation and validation of a two‐variable index to predict 30‐day outcomes following heart failure hospitalization. ESC Heart Failure, 8: 2690–2697. 10.1002/ehf2.13324.
References
- 1. Tran DT, Ohinmaa A, Thanh NX, Howlett JG, Ezekowitz JA, McAlister FA, Kaul P. The current and future financial burden of hospital admissions for heart failure in Canada: a cost analysis. C Open 2016; 4: E365–E370. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Van Walraven C, Dhalla IA, Bell C, Etchells E, Stiell IG, Zarnke K, Austin PC, Forster AJ. Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. CMAJ 2010; 182: 551–557. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Yazdan‐Ashoori P, Lee SF, Ibrahim Q, Van Spall HGC. Utility of the LACE index at the bedside in predicting 30‐day readmission or death in patients hospitalized with heart failure. Am Heart J 2016; 179: 51–58. [DOI] [PubMed] [Google Scholar]
- 4. Health Quality Ontario . Adopting a common approach to transitional care planning: helping health links improve transitions and coordination of care [Internet]. 2013. http://www.hqontario.ca/Portals/0/documents/qi/health-links/bp-improve-package-traditional-care-planning-en.pdf (1 September 2020).
- 5. Van Spall HGC, Lee SF, Xie F, Erbas Oz U, Perez R, Mitoff PR, Maingi M, Tjandrawidjaja MC, Heffernan M, Zia MI, Porepa L. Effect of patient‐centered transitional care services on clinical outcomes in patients hospitalized for heart failure: the PACT‐HF randomized clinical trial. JAMA 2019; 321: 753–761. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Van Spall HGC, Lee SF, Xie F, Ko DT, Thabane L, Ibrahim Q, Mitoff PR, Heffernan M, Maingi M, Tjandrawidjaja MC, Zia MI. Knowledge to action: rationale and design of the patient‐centered care transitions in heart failure (PACT‐HF) stepped wedge cluster randomized trial. Am Heart J 2018; 199: 75–82. [DOI] [PubMed] [Google Scholar]
- 7. Carlson KJ, Lee DCS, Goroll AH, Leahy M, Johnson RA. An analysis of physicians' reasons for prescribing long‐term digitalis therapy in outpatients. J Chronic Dis 1985; 38: 733–739. [DOI] [PubMed] [Google Scholar]
- 8. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for some traditional and novel measures. Epidemiology 2010; 21: 128–138. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Teres D. A review of goodness of fit statistics for use in the development of logistic regression models 1. Am J Epidemiol 1982; 115: 92–106. [DOI] [PubMed] [Google Scholar]
- 10. Florkowski CM. Sensitivity, specificity, receiver‐operating characteristic (ROC) curves and likelihood ratios: communicating the performance of diagnostic tests. Clin Biochem Rev 2008; 29: S83–S87. [PMC free article] [PubMed] [Google Scholar]
- 11. Liu X. Classification accuracy and cut point selection. Stat Med 2012; 31: 2676–2686. [DOI] [PubMed] [Google Scholar]
- 12. Lee DS, Austin PC, Rouleau JL, Liu PP, Naimark D. Predicting mortality among patients hospitalized for heart failure derivation and validation of a clinical model. JAMA 2003; 290: 2581–2587. [DOI] [PubMed] [Google Scholar]
- 13. Keenan PS, Normand ST, Lin Z, Drye EE, Bhat KR, Ross JS, Schuur JD, Stauffer BD, Bernheim SM, Epstein AJ, Wang Y. An administrative claims measure suitable for profiling hospital performance on the basis of 30‐day all‐cause readmission rates among patients with heart failure. Circ Cardiovasc Qual Outcomes 2008; 1: 29–37. [DOI] [PubMed] [Google Scholar]
- 14. Hammill BG, Curtis LH, Fonarow GC, Heidenreich PA, Yancy CW, Peterson ED, Hernandez AF. Incremental value of clinical data beyond claims data in predicting 30‐day outcomes after heart failure hospitalization. Circ Cardiovasc Qual Outcomes 2011; 4: 60–67. [DOI] [PubMed] [Google Scholar]
- 15. Frizzell JD, Liang L, Schulte PJ, Yancy CW, Heidenreich PA, Hernandez AF, Bhatt DL, Fonarow GC, Laskey WK. Prediction of 30‐day all‐cause readmissions in patients hospitalized for heart failure: comparison of machine learning and other statistical approaches. JAMA Cardiol 2017; 2: 204–209. [DOI] [PubMed] [Google Scholar]
- 16. Canepa M, Fonseca C, Chioncel O, Laroche C, Crespo‐Leiro M, Coats A, Mebazaa A, Piepoli MF, Tavazzi L, Maggioni AP. Performance of prognostic risk scores in chronic heart failure patients enrolled in the European Society of Cardiology Heart Failure Long‐Term Registry. JACC Hear Fail 2018; 6: 452–462. [DOI] [PubMed] [Google Scholar]
- 17. Luca G, Tanna D, Wirtz H, Burrows KL, Globe G. Evaluating risk prediction models for adults with heart failure: a systematic literature review. PLoS One 2020; 15: 1–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Dharmarajan K, Lin Z, Ross JS, Horwitz LI, Barreto‐filho A, Kim N, Bernheim SM, Suter LG, Drye EE. Diagnoses and timing of 30‐day readmissions after hospitalization for heart failure, acute myocardial infarction, or pneumonia. JAMA 2015; 309: 355–363. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Yeh J, Lin C, Kao C. Relationship between pneumonia and cardiovascular diseases: a retrospective cohort study of the general population. Eur J Intern Med 2019; 59: 39–45. [DOI] [PubMed] [Google Scholar]
- 20. Wennberg J. Tracking Medicine: A Researcher's Quest to Understand Health Care. New York: Oxford University Press; 2010. [Google Scholar]
- 21. Eapen ZJ, Liang L, Fonarow GC, Heidenreich PA, Curtis LH, Peterson ED. Validated, electronic health record deployable prediction models for assessing patient risk of 30‐day rehospitalization and mortality in older heart failure patients. JACC Hear Fail 2013; 1: 245–251. [DOI] [PubMed] [Google Scholar]
- 22. McCormick N, Lacaille D, Bhole V, Avina‐Zubieta JA. Validity of heart failure diagnoses in administrative databases: a systematic review and meta‐analysis. PLoS One 2014; 9: e104519. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data S1. Supporting information