Abstract
Introduction
Accurate prognostication is central to decision-making in heart failure (HF). The recently-developed LIFE-HF models offer promise, but their performance has not been independently and externally validated. The aim of this study was to assess the external validity of the LIFE-HF models in a contemporary North American population.
Methods
We externally validated the LIFE-HF models for 1-year all-cause death and the composite of death or HF hospitalization using patient-level data from the GUIDE-IT trial. All LIFE-HF predictors were available; missing data were handled using multiple imputation. Predicted risks were calculated using original LIFE-HF model equations. We assessed model discrimination using time-dependent area under the receiver operating characteristic curve (AUROC), calibration [observed-to-expected (O/E) ratios, calibration slopes and curves], overall prediction error, and net benefit using decision curve analysis.
Results
The validation cohort included 801 participants (median age 64 years, 69% male). The mortality model demonstrated good discrimination [AUROC 0.77, 95% confidence interval (CI): 0.72–0.82], but underprediction (O/E 1.28, 95% CI: 1.01–1.54) and underfitting (calibration slope 1.58, 95% CI: 1.20–1.95). The composite model showed moderate discrimination (AUROC 0.69, 95% CI: 0.64–0.73), underprediction (O/E 1.40, 95% CI: 1.25–1.54), and overfitting (calibration slope 0.82, 95% CI: 0.63–1.00). Decision curve analysis showed net clinical benefit for the mortality model, but not the composite model, over a broad range of risk thresholds.
Conclusion
In a contemporary, high-risk North American HF with reduced ejection fraction cohort, the LIFE-HF models showed good discrimination but systematically underpredicted 1-year risk. The mortality model may support risk-informed decision-making, whereas the composite model requires further validation.
Keywords: Clinical prediction model, External validation, HFrEF, Prognosis, Risk communication, Shared decision-making
Graphical Abstract
Graphical Abstract.
Introduction
Heart failure (HF) affects over 64 million people globally and reduces patient quality of life and life expectancy.1 An accurate estimate of prognosis is considered central to decision-making, and HF guidelines recommend communication of prognosis to patients as a standard of care.2 Yet, prognosis is infrequently discussed with HF patients in practice, despite patients’ desire to receive this information, and is often conveyed using qualitative terms such as ‘high risk’, which are open to variable interpretation, or population-level values that do not reflect an individual’s risk.3,4 As a result, patients with HF often have a poor understanding of their prognosis and trajectory—usually underestimating their risk—which can lead to decisions misaligned with their preferences and values.4–6 Conversely, clinician gestalt tends to overestimate risk, whereas multivariable risk prediction models outperform gestalt alone.7,8
Several risk prediction models have been developed to estimate the risk of death and HF hospitalization in ambulatory patients with HF with reduced ejection fraction (HFrEF); yet few have been externally validated, among which even fewer have acceptable predictive performance.9–12 The recently-developed LIFE-HF models hold significant promise based on the inclusion of a comprehensive list of strong predictors of HF events, such as N-terminal pro B-type natriuretic peptide (NT-proBNP), and robust performance in external validation in multiple cohorts over an extended duration.9 These validation cohorts included national European registries (Swede-HF), international registries (ASIAN-HF), and international randomized controlled trials (DAPA-HF). Additionally, the LIFE-HF equations offer flexibility of prognosis communication using estimates of both death and HF hospitalizations at 1–5 years, as well as life expectancy, and can readily be incorporated into a web-based risk calculator or patient decision aid. However, North American patients account for only 5% of the model development cohort and 14% of the DAPA-HF validation cohort. Given the large global variations in HF treatment and outcomes, external validation in additional cohorts is warranted to ensure generalizability and transportability of risk estimates to broad populations.13,14
We aimed to validate the transportability of the LIFE-HF equations to a contemporary North American population using the dataset of the GUIDE-IT trial, which was conducted in Canada and the USA.15
Methods
We used data from the GUIDE-IT trial to externally validate the LIFE-HF models to predict death and the composite of death or HF hospitalization at 1 year in ambulatory patients with HFrEF. We reported this validation study according to the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) checklist.16 This study complies with the Declaration of Helsinki. Ethics approval was obtained by the University of British Columbia Clinical Research Ethics Board. Informed consent was not required for the present analysis, as this was secondary use of de-identified data.
Data source
We used the full anonymized database of the GUIDE-IT trial obtained from the National Heart, Lung, and Blood Institute Biologic Specimen and Data Repository Information Coordinating Center. In brief, GUIDE-IT randomized 894 patients with HFrEF from the USA and Canada between 2013 and 2016 to NT-proBNP-guided management or usual care.15 At a median follow-up of 15 months, there was no difference between intervention groups in the primary outcome of time-to-first HF hospitalization or cardiovascular mortality, or any secondary outcome. As such, we did not model the randomized treatment as a predictor. Data collection included all predictor variables included in the LIFE-HF model, HFrEF medication use, as well as all outcomes of interest with a median follow-up of 15 months, making it ideal to determine the performance of the LIFE-HF model in North American patients.
Participants
For this analysis, we included participants from the GUIDE-IT trial, restricted to patients between the ages of 40 and 85 years, as in the original LIFE-HF model development.9 In addition to having HFrEF defined by left ventricular ejection fraction (LVEF) ≤40%, inclusion criteria for GUIDE-IT required a HF event (HF hospitalization, emergency department visit for HF, or intravenous diuretics for HF) in the 12 months prior to randomization, and NT-proBNP >2000 pg/ml or BNP >400 pg/ml within 30 days prior to randomization.
Outcomes
The outcomes of interest included all-cause death (death outcome) and the composite of death from any cause or adjudicated HF hospitalization (composite outcome) at 1 year.
Predictors and LIFE-HF model calculation
We included all predictors for the LIFE-HF model: age, sex, geographic region (North America for all), New York Heart Association class (NYHA; I/II vs III/IV), prior HF hospitalization, diabetes, extracardiac vascular disease (prior stroke or peripheral artery disease), systolic blood pressure (SBP), estimated glomerular filtration rate (eGFR; calculated using the CKD-EPI 2021 race-free, creatinine-based equation),17 NT-proBNP, and LVEF.
For continuous predictors (eGFR, LVEF, NT-proBNP, and SBP), we winsorized values at the 1st and 99th percentiles of the LIFE-HF model development cohort.9 For missing predictor values, we used multiple imputation by chained equations with 20 imputations (using predictive mean matching for continuous variables and logistic regression for binary variables), pooled using Rubin’s rules.
Analysis
We described baseline characteristics of the full trial population and validation population of GUIDE-IT using medians and interquartile range (IQR) for continuous variables and frequencies (%) for categorical variables.
For validation, we used the original LIFE-HF model equations to calculate 1-year predicted risk of death and composite outcome for each patient and compared them to observed risk at 1 year using the Kaplan–Meier method.9 Development of the LIFE-HF models did not include HFrEF medications as predictors due to the universal baseline use of receiving an angiotensin-converting enzyme inhibitor (ACEI) or angiotensin-receptor blocker (ARB) and high use of beta-blockers (∼92%) in the development cohort.9 However, the LIFE-HF calculator combines predicted risk with hazard ratios from trials to provide individualized estimates. In the present study, we adjusted predictions of patients not receiving an ACEI/ARB or beta-blocker by subtracting the respective log of the hazard ratios from randomized controlled trials [0.84 and 0.76 for death and 0.74 and 0.77 for the composite outcome, respectively, for ACEI/ARB/angiotensin receptor-neprilysin inhibitor (ARNI) and beta-blocker].9
We assessed the predictive performance of the LIFE-HF models at 1 year in multiple domains: discrimination (how well the model differentiates between different risk levels), calibration (agreement between observed and predicted risk), overall performance, and clinical utility.18,19 First, we evaluated discrimination using the time-dependent area under the receiver operating characteristic curve (AUROC), which accounts for right-censoring, estimated using the inverse probability of censoring weighting approach.18,20 Second, we evaluated mean calibration using the observed-to-expected (O/E) ratio and calibration-in-the-large [logit(O)-logit(E)], with observed risks estimated using Kaplan–Meier analysis, and weak calibration using the calibration slope. Third, we evaluated moderate calibration using the smoothed calibration curve and the integrated calibration index (ICI); the latter represents the weighted mean of the absolute difference between observed and predicted probabilities.18 Fourth, we calculated the Brier score and index of prediction accuracy (IPA) for explained variance and overall model prediction error, reflecting the balance of discrimination and calibration.21 Finally, to evaluate the clinical utility of these models, we performed decision curve analysis, a widely-adopted method for quantifying the net benefit of clinical prediction models.16,22,23 In decision curve analysis, net benefit (y axis) is calculated as the difference between the proportion of true positives and the weighted proportion of false positives, with the weight determined by the treatment probability threshold (x axis). Net benefit calculation requires a treatment threshold placed on outcome risk for classifying individuals as low vs high risk (e.g. risk threshold to consider statins in primary prevention of cardiovascular disease). The choice of a treatment threshold for an intervention depends on the benefit-harm trade-off of that intervention. For example, a 20% treatment threshold indicates that we consider the harms (cost, side-effects, and other treatment burdens) to be 1/4 (the value of a true positive divided by false positive) of its clinical benefit.23 Such thresholds are not established for HF device or drug therapy for either risk of death or HF hospitalizations. Therefore, we performed these analyses across the range of 0%–50% thresholds on 1-year risk of death and the composite outcome, a broad range that includes the plausible range for guiding various HF treatment decisions pending established optimal treatment thresholds. Comparisons are made to ‘treat-none’ (i.e. treatment not offered regardless of risk) and ‘treat-all’ (i.e. treatment given regardless of risk) approaches at each treatment threshold and graphed across the range.
We conducted several post hoc sensitivity analyses to assess the robustness of the results to key modelling decisions: First, we repeated the analyses without winsorizing values to the LIFE-HF development cohort 1st–99th percentile (full predictor distribution). Second, we repeated the analyses without adjusting for baseline use of ACEI/ARB/ARNI or beta-blocker. Third, we repeated the analyses with use of the Modification of Diet in Renal Disease (MDRD) equation for eGFR,24 as calculated in the two trials in the LIFE-HF model development cohort. Fourth, we assessed model performance at 6 months. We conducted all analyses using R 4.5.1 (R Foundation). R code for model validation is available on https://github.com/resplab/papercode/tree/main/LIFE-HF.
Results
From 894 participants in the GUIDE-IT trial dataset, we included 801 participants after applying age restrictions. Values were missing for NT-proBNP, eGFR, and NYHA in 4.6%, 3.5%, and 1.45%, respectively, with other predictors having <0.6% missingness (Table 1). The median age was 64 years (IQR, 55–71), 69% were male, 43% had NYHA Class III/IV symptoms, 77% had a prior history of HF hospitalization, and 41% were receiving triple therapy for HFrEF (Table 1). Median values for continuous predictors were 25% for LVEF, 2642 pg/ml for NT-proBNP, 56 ml/min/1.73 m2 for eGFR (CKD-EPI 2021 race-free, creatinine-based equation), and 114 mmHg for SBP. Supplementary Figure S1 illustrates differences between the two eGFR equations, stratified by race. In addition to geographical differences to the LIFE-HF development cohort, patients in the GUIDE-IT trial validation cohort were more likely to be female and had a greater overall burden of all LIFE-HF predictors (Table 1). A table comparing the subset of patients used for the present validation study and the overall GUIDE-IT trial population is provided in Supplementary Table S1. Incidence rates over the trial follow-up for cardiovascular death, non-cardiovascular death, and the composite of cardiovascular death or HF hospitalization were notably higher than rates observed in the LIFE-HF validation cohort, including the North American subset (Table 1).
Table 1.
Baseline characteristics of patients in the GUIDE-IT trial validation cohort
| GUIDE-IT trial validation cohorta,b | LIFE-HF model development cohort9 | |
|---|---|---|
| n | 801 | 15 415 |
| Age, median (IQR) | 64 (55–71) | 64 (56–72) |
| Male, n (%) | 551 (68.8) | 12 058 (78) |
| Black race | 287 (35.8) | 537 (4) |
| Region, n (%) | ||
| North America | 801 (100) | 779 (5) |
| Canada | 125 (16) | — |
| USA | 676 (84) | — |
| Clinical variables, median (IQR) | ||
| Body mass index, kg/m2 | 28.6 (24.5–33.4) | 27.1 (24.0–30.8) |
| SBP, mmHg | 114 (102–128) | 127 (115–140) |
| eGFR, ml/min/1.73 m2 | ||
| CKD-EPI 2021 equations | 56 (41–73) | — |
| MDRD | 55 (39–71) | 70 (57–83) |
| LVEF, % | 25 (20–30) | 30 (25–34) |
| NT-proBNP, pg/ml | 2642 (1462–5372.5) | 1418 (770–2774) |
| Medical history, n (%) | ||
| Prior HF hospitalization | 613 (77) | 9462 (61) |
| NYHA III/IV | 343 (43) | 4656 (30) |
| Diabetes | 389 (49) | 4832 (31) |
| Extracardiac vascular disease | 160 (20) | 2411 (16) |
| Medications, n (%) | ||
| Loop diuretic | 750 (94) | 12 336 (80) |
| ACEI/ARB/ARNI | 641 (80) | — |
| ARNI | 10 (1) | 4187 (27) |
| Beta-blocker | 762 (95) | 14 243 (92) |
| MRA | 395 (49) | 7273 (47) |
| Triple therapy | 331 (41) | — |
| Outcome incidence rates, per 1000 person-years (North American subgroup of LIFE-HF model development cohort) | ||
| Cardiovascular death | 106c | 72 (61) |
| Non-cardiovascular death | 28c | 14 (20) |
| Composite of CV death or HF hospitalization | 385d | 119 (138) |
ACEI, angiotensin-converting enzyme inhibitor; ARB, angiotensin-receptor blocker; ARNI, angiotensin receptor-neprilysin inhibitor; CV, cardiovascular; eGFR, estimated glomerular filtration rate; IQR, interquartile range; LVEF, left ventricular ejection fraction; MRA, mineralocorticoid receptor antagonist; NT-proBNP, N-terminal pro B-type natriuretic peptide; NYHA, New York Heart Association; SBP, systolic blood pressure.
aVariables with missing data in full GUIDE-IT trial population (n = 894), in descending order: NT-proBNP (4.6%), eGFR (3.5%), body mass index (2.35%), NYHA class (1.45%), SBP (0.6%), beta-blocker (0.5%), ACEI/ARB/ARNI (0.3%), extracardiac vascular disease (0.1%), COPD (0.1%).
bMean five imputed datasets created after restricting to patients aged 40–85 years and winsorizing continuous variables.
cMedian follow-up of 1.2 years.
dMedian follow-up of 0.8 years.
Table 2 and the Graphical Abstract summarize the predictive performance of the LIFE-HF models. For death, the mean observed and predicted 1-year event rates were 12.6% and 9.9%, respectively [O/E ratio 1.28; 95% confidence interval (CI): 1.01–1.54]. For the composite outcome, the mean observed and predicted 1-year event rates were 35.5% and 25.4%, respectively (O/E ratio 1.40; 95% CI: 1.25–1.54). In both cases, models underpredicted observed risk. On the other hand, the models for the death outcome had good discrimination (time-dependent AUROC 0.77; 95% CI: 0.72–0.82), whereas discrimination was moderate for the composite outcome (0.69; 95% CI: 0.64–0.73), respectively. Supplementary Figure S2 provides illustrative Kaplan–Meier curves from a single imputed dataset for both outcomes stratified by predicted risk tertile, which demonstrated separation of curves without overlap between predicted risk groups for both outcomes.
Table 2.
Summary of LIFE-HF model performance in GUIDE-IT trial validation at 1 year
| All-cause death | All-cause death or HF hospitalization | |
|---|---|---|
| Discrimination: time-dependent AUROC at 1 year, (95% CI) | 0.77 (0.72–0.82) | 0.69 (0.64–0.73) |
| Mean calibration | ||
| O/E ratio (95% CI) | 1.28 (1.01–1.54) | 1.40 (1.25–1.54) |
| Observed 1-year risk | 12.6% | 35.5% |
| Predicted 1-year risk | 9.9% | 25.4% |
| Calibration-in-the-large (95% CI) | 0.27 (0.04–0.51) | 0.48 (0.32–0.64) |
| Weak calibration: slope (95% CI) | 1.58 (1.20–1.95) | 0.82 (0.63–1.00) |
| Moderate calibration: ICI | 4.1% | 10.4% |
| Overall performance | ||
| Brier | 0.090 | 0.177 |
| Index of prediction accuracy (95% CI) | 0.064 (0.027–0.101) | −0.200 (−0.259 to −0.141) |
AUROC, area under the receiver operating characteristic curve; CI, confidence interval; ICI, integrated calibration index; O/E, observed-to-expected.
Assessment of calibration-in-the-large (mean calibration) and visualization of the calibration curves (moderate calibration) confirmed systematic underprediction for both models (Figure 1). Calibration curves, along with their linear slopes, suggested underfitting of the model predicting death, whereas the composite outcome model was overfitted. Underprediction for death was small on the absolute scale (ICI 4.1%) but large for the composite outcome (ICI 10.4%). The Brier score and IPA indicated modest overall performance with the mortality model (IPA 0.064; 95% CI: 0.027–0.101), whereas the composite model performed worse than use of the null model (IPA −0.200; 95% CI: −0.259 to −0.141) (i.e. assigning the overall population estimate for all patients). Applying decision curve analysis, the mortality model was estimated to provide net benefit over default treat-all and treat-none strategies at treatment thresholds of 1-year risk between 2% and 33% (Figure 2). Conversely, the composite model did not demonstrate net benefit until thresholds ≥30%, consistent with miscalibration and overfitting within this predicted probability range.
Figure 1.
Smoothed calibration curves for (A) death and (B) the composite of death or heart failure hospitalization at 1 year. (A) Death. (B) Death or heart failure hospitalization. *As the true event rate is unobservable, the y axis is a proxy for observed risk estimated using a secondary Cox model developed from the validation dataset [with log(−log) of the estimated 1-year event rate as the predictor, modelled non-linearly using restricted cubic splines (three knots at default positions)]. The grey density curves at the top of each figure represent the distribution of predicted risks
Figure 2.
Decision curves analysis for (A) death and (B) the composite of death or heart failure hospitalization at 1 year. (A) Death. (B) Death or heart failure hospitalization
Discrimination was generally consistent across subgroups (Supplementary Table S2). Calibration differed by subgroup, with the mortality model being generally better calibrated in females than males, and the composite outcome model better calibrated in Canada than the USA; however, these were based on small subgroups with few events, leading to imprecision. Sensitivity analyses demonstrated similar model performance without winsorizing of predictors, without adjustment for baseline use of ACEI/ARB/ARNI and beta-blockers, with eGFR based on the MDRD equation, and with prediction at a shorter 6-month timeframe (Supplementary Table S3).
Discussion
Principal findings
In this external validation in patients within the GUIDE-IT trial, the LIFE-HF models had good discrimination. However, LIFE-HF systematically underpredicted 1-year risk of death and, to a greater extent, the composite of death or HF hospitalization. Despite this miscalibration, the IPA and net benefit from the decision curve analysis indicated that using the LIFE-HF mortality model to guide treatment decisions provided actionable information to inform decisions across a range of clinically relevant risk thresholds. Specifically, the mortality model outperformed default strategies of treating all patients or treating none when patients and clinicians had 1-year mortality risk tolerance thresholds 2% and 33%, indicating that use of the LIFE-HF mortality model to individualize care is superior to uniform decisions for all patients with HFrEF. This suggests that, even with some degree of underprediction, the LIFE-HF mortality model can support individualized decision-making compared with a uniform ‘treat all’ approach, which is important in the context of ever-present treatment-related harms.
Comparison to other studies
The LIFE-HF model was developed in a pooled cohort from the PARADIGM-HF and ATMOSPHERE trials and externally validated in the DAPA-HF trial, ASIAN-HF registry, and SWEDE-HF registry.9 In external validation described in the initial publication, the LIFE-HF model had good discrimination for both all-cause death and the composite of death or HF hospitalization, with C-statistics of 0.65–0.73 and 0.66–0.70, respectively, at timepoints ranging from 1 to 10 years. However, the LIFE-HF model required recalibration in the SWEDE-HF registry due to overprediction of deaths, and calibration curves suggested overprediction in higher-risk patients in the ASIAN-HF registry, whereas the model was well-calibrated in the overall DAPA-HF cohort. The present validation in the GUIDE-IT trial population demonstrated discriminative performance of both LIFE-HF death and composite models that was similar or better than previously reported. In contrast to the overprediction observed in SWEDE-HF and ASIAN-HF, the LIFE-HF models systematically underpredicted risk of death and the composite outcomes in GUIDE-IT trial patients. This may reflect a higher-risk population enrolled into GUIDE-IT, which employed enrichment criteria including recent HF event and at least moderately-elevated natriuretic peptides. Compared with the LIFE-HF model development cohort, GUIDE-IT had a lower median LVEF, blood pressure, and eGFR, along with higher NYHA class and prevalence of HF hospitalization and baseline use of loop diuretics, whereas use of HFrEF medications was similar, except for lower use of renin-angiotensin system inhibitors. This is corroborated by higher incidence rates for outcomes of interest, particularly HF hospitalizations, compared with the cohorts described in the original LIFE-HF publication. Arguably, the GUIDE-IT trial captures the large-volume clinical centres where prediction models such as LIFE-HF are most likely to be used to enable shared decision-making with patients at higher risk post-HF events. These results support the ability of the LIFE-HF model to discriminate between patients at higher vs lower risk of death, with the need for further validation of the composite model and likely recalibration of both models for this setting. This is consistent with the approach undertaken to evaluate calibration within a new setting or jurisdiction prior to routine implementation of other cardiovascular risk prediction models for primary and secondary disease prevention.25,26 The present study extends these results further by assessing clinical utility using decision curve analysis and overall performance using the Brier score and IPA, which were not reported in the original development and validation of the LIFE-HF model.
Implications of study results
Despite imperfect calibration, the present study demonstrates that use of the LIFE-HF mortality model in decision-making would have clinical utility over uniform treatment of all patients with HFrEF regardless of individual prognosis. This aligns with guideline recommendations that promote shared decision-making by routinely discussing prognosis with patients.2,6,27,28 In particular, this may help to inform decision-making regarding treatments that may have limited availability or may come with trade-offs in terms of out-of-pocket costs, side-effects, or added treatment-related burden.29 Whereas physicians tend to overestimate the risk of death and other consequences from HF, patients with HFrEF often underestimate their risk.5–7 Inaccurate prognosis expectations may lead to decisions that contradict patient values, leading to mismatch between treatment and preferences.5,30 This may in part explain the persistence in suboptimal use of guideline-directed medical therapy in patients with HFrEF, which has been attributed to clinical inertia.31–34 Given the good-to-excellent performance of the LIFE-HF mortality model, future studies could explore the integration of this model into point-of-care decision aids to support shared understanding of prognosis and shared decision-making, and its impact on clinical outcomes.35 It could also be used to invigorate use of diagnostic and therapeutic interventions in HF patients, where a clear gradient of reduced access to these processes of care have been demonstrated according to socioeconomic status.36 In contrast, the LIFE-HF composite model requires further validation and likely recalibration before it can reliably support decision-making in sicker populations such as those enrolled in GUIDE-IT.
Limitations
The present validation study has several limitations. First, the GUIDE-IT trial had a modest sample size and number of events; however, 95% CIs were reasonably narrow for AUROCs, and precision was sufficient to identify significant miscalibration. Second, the GUIDE-IT trial was restricted to patients with a HF event within 12 months and NT-proBNP >2000 pg/ml within 30 days. This represents a more uniform, higher-risk population than the LIFE-HF model development cohort, as demonstrated by differences between cohorts in baseline characteristics. However, this did not compromise model discrimination relative to prior performance in the development and validation cohorts, thereby proving reliable results for this important subset of patients. Third, baseline NT-proBNP, the strongest predictor within the LIFE-HF model, was missing for 4.6% of participants, which was addressed via multiple imputation. Fourth, we used a broad range of threshold probabilities for the decision curve analysis due to the absence of accepted risk-based treatment thresholds for pharmacological and non-pharmacological therapies for HFrEF. Future studies establishing risk-based treatment thresholds based on patient preferences could inform the evaluation of clinical prediction models and individualized treatment decisions.
Conclusions
In a contemporary North American trial population of high-risk HFrEF patients, the LIFE-HF models demonstrated good discrimination, but systematically underpredicted 1-year risk, particularly in predicting the composite of death or HF hospitalization. Despite this miscalibration, decision curve analysis indicated that using the LIFE-HF mortality model to guide treatment decisions would yield net clinical benefit across a range of clinically relevant mortality risk thresholds. These findings support the potential utility of the LIFE-HF mortality model to inform decision-making. An impact study—an experimental study that assess the feasibility, acceptability, and impact on clinical outcomes of implementing a clinical prediction model—is the rational next step before widespread implementation of this model in practice.
Supplementary Material
Acknowledgements
This Manuscript was prepared using GUIDE-IT Research Materials obtained from the NHLBI Biologic Specimen and Data Repository Information Coordinating Center and does not necessarily reflect the opinions or views of the GUIDE-IT trialists or the NHLBI. We thank Drs Pascal M. Burger and Arend Mosterd for sharing the code used in the original LIFE-HF manuscript for score calculation and validation.
Contributor Information
Ricky D Turgeon, Collaboration for Outcomes Research and Evaluation, Faculty of Pharmaceutical Sciences, University of British Columbia, Room 5523-2405 Wesbrook Mall, Vancouver, British Columbia, Canada V6T 1Z3; Centre for Cardiovascular Innovation, Division of Cardiology, University of British Columbia, Vancouver, Canada.
Nathaniel M Hawkins, Centre for Cardiovascular Innovation, Division of Cardiology, University of British Columbia, Vancouver, Canada.
Mohsen Sadatsafavi, Collaboration for Outcomes Research and Evaluation, Faculty of Pharmaceutical Sciences, University of British Columbia, Room 5523-2405 Wesbrook Mall, Vancouver, British Columbia, Canada V6T 1Z3.
Douglas S Lee, ICES, Toronto, Canada.
Supplementary data
Supplementary data are available at ESC Heart Failure online.
Declarations
Disclosure of Interest
All authors declare no disclosure of interest for this contribution.
Data Availability
The authors do not have permission to share GUIDE-IT trial data; interested researchers may seek to obtain these from the NHLBI Biologic Specimen and Data Repository Information Coordinating Center. R code used for model validation is available on https://github.com/resplab/papercode/tree/main/LIFE-HF.
Funding
None. Ricky Turgeon is supported by a Health Professional–Investigator award by Michael Smith Health Research BC.
Ethical Approval
Ethics approval was obtained by the University of British Columbia Clinical Research Ethics Board. Informed consent was not required for the present analysis, as this was secondary use of de-identified data.
Pre-registered Clinical Trial Number
None supplied.
References
- 1. Savarese G, Becher PM, Lund LH, Seferovic P, Rosano GMC, Coats AJS. Global burden of heart failure: a comprehensive and updated review of epidemiology. Cardiovasc Res 2023;118:3272–87. 10.1093/cvr/cvac013 [DOI] [PubMed] [Google Scholar]
- 2. Heidenreich PA, Bozkurt B, Aguilar D, Allen LA, Byun JJ, Colvin MM, et al. 2022 AHA/ACC/HFSA guideline for the management of heart failure: a report of the American College of Cardiology/American Heart Association Joint Committee on Clinical Practice Guidelines. J Am Coll Cardiol 2022;79:e263–421. 10.1016/j.jacc.2021.12.012 [DOI] [PubMed] [Google Scholar]
- 3. Narayan M, Jones J, Portalupi LB, McIlvennan CK, Matlock DD, Allen LA. Patient perspectives on communication of individualized survival estimates in heart failure. J Card Fail 2017;23:272–7. 10.1016/j.cardfail.2016.12.008 [DOI] [PubMed] [Google Scholar]
- 4. Cavanagh CE, Rosman L, Spatz ES, Fried T, Gandhi PU, Soucier RJ, et al. Dying to know: prognosis communication in heart failure. ESC Heart Fail 2020;7:3452–63. 10.1002/ehf2.12941 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Allen LA, Yager JE, Funk MJ, Levy WC, Tulsky JA, Bowers MT, et al. Discordance between patient-predicted and model-predicted life expectancy among ambulatory patients with heart failure. JAMA 2008;299:2533–42. 10.1001/jama.299.21.2533 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Kitakata H, Kohno T, Kohsaka S, Fujisawa D, Nakano N, Shiraishi Y, et al. Prognostic understanding and preference for the communication process with physicians in hospitalized heart failure patients. J Card Fail 2021;27:318–26. 10.1016/j.cardfail.2020.10.009 [DOI] [PubMed] [Google Scholar]
- 7. Alba AC, Buchan TA, Saha S, Fan S, Demers C, Poon S, et al. Predicting 1-year mortality in outpatients with heart failure with reduced left ventricular ejection fraction: do empiric models outperform physician intuitive estimates? A multicenter cohort study. Circ Heart Fail 2023;16:e010312. 10.1161/CIRCHEARTFAILURE.122.010312 [DOI] [PubMed] [Google Scholar]
- 8. Lee DS, Lee JS, Schull MJ, Borgundvaag B, Edmonds ML, Ivankovic M, et al. Prospective validation of the emergency heart failure mortality risk grade for acute heart failure: the ACUTE study. Circulation 2019;139:1146–56. 10.1161/CIRCULATIONAHA.118.035509 [DOI] [PubMed] [Google Scholar]
- 9. Burger PM, Savarese G, Tromp J, Adamson C, Jhund PS, Benson L, et al. Personalized lifetime prediction of survival and treatment benefit in patients with heart failure with reduced ejection fraction: the LIFE-HF model. Eur J Heart Fail 2023;25:1962–75. 10.1002/ejhf.3028 [DOI] [PubMed] [Google Scholar]
- 10. Alba AC, Agoritsas T, Jankowski M, Courvoisier D, Walter SD, Guyatt GH, et al. Risk prediction models for mortality in ambulatory patients with heart failure: a systematic review. Circ Heart Fail 2013;6:881–9. 10.1161/CIRCHEARTFAILURE.112.000043 [DOI] [PubMed] [Google Scholar]
- 11. Rahimi K, Bennett D, Conrad N, Williams TM, Basu J, Dwight J, et al. Risk prediction in patients with heart failure: a systematic review and analysis. JACC Heart Fail 2014;2:440–6. 10.1016/j.jchf.2014.04.008 [DOI] [PubMed] [Google Scholar]
- 12. Simpson J, Jhund PS, Lund LH, Padmanabhan S, Claggett BL, Shen L, et al. Prognostic models derived in PARADIGM-HF and validated in ATMOSPHERE and the Swedish heart failure registry to predict mortality and morbidity in chronic heart failure. JAMA Cardiol 2020;5:432–41. 10.1001/jamacardio.2019.5850 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Foroutan F, Rayner DG, Ross HJ, Ehler T, Srivastava A, Shin S, et al. Global comparison of readmission rates for patients with heart failure. J Am Coll Cardiol 2023;82:430–44. 10.1016/j.jacc.2023.05.040 [DOI] [PubMed] [Google Scholar]
- 14. G-CHF Investigators; Joseph P, Roy A, Lonn E, Störk S, Floras J, et al. Global variations in heart failure etiology, management, and outcomes. JAMA 2023;329:1650–61. 10.1001/jama.2023.5942 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Felker GM, Anstrom KJ, Adams KF, Ezekowitz JA, Fiuzat M, Houston-Miller N, et al. Effect of natriuretic peptide-guided therapy on hospitalization or cardiovascular mortality in high-risk patients with heart failure and reduced ejection fraction: a randomized clinical trial. JAMA 2017;318:713–20. 10.1001/jama.2017.10565 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Moons KGM, Altman DG, Reitsma JB, Ioannidis JPA, Macaskill P, Steyerberg EW, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med 2015;162:W1–73. 10.7326/M14-0698 [DOI] [PubMed] [Google Scholar]
- 17. Inker LA, Eneanya ND, Coresh J, Tighiouart H, Wang D, Sang Y, et al. New creatinine- and cystatin C-based equations to estimate GFR without race. N Engl J Med 2021;385:1737–49. 10.1056/NEJMoa2102953 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. McLernon DJ, Giardiello D, Van Calster B, Wynants L, van Geloven N, van Smeden M, et al. Assessing performance and clinical usefulness in prediction models with survival outcomes: practical guidance for cox proportional hazards models. Ann Intern Med 2023;176:105–14. 10.7326/M22-0844 [DOI] [PubMed] [Google Scholar]
- 19. Collins GS, Dhiman P, Ma J, Schlussel MM, Archer L, Van Calster B, et al. Evaluation of clinical prediction models (part 1): from development to external validation. BMJ 2024;384:e074819. 10.1136/bmj-2023-074819 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Kamarudin AN, Cox T, Kolamunnage-Dona R. Time-dependent ROC curve analysis in medical research: current methods and applications. BMC Med Res Methodol 2017;17:53. 10.1186/s12874-017-0332-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Kattan MW, Gerds TA. The index of prediction accuracy: an intuitive measure useful for evaluating risk prediction models. Diagn Progn Res 2018;2:7. 10.1186/s41512-018-0029-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Van Calster B, Wynants L, Verbeek JFM, Verbakel JY, Christodoulou E, Vickers AJ, et al. Reporting and interpreting decision curve analysis: a guide for investigators. Eur Urol 2018;74:796–804. 10.1016/j.eururo.2018.08.038 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Sadatsafavi M, Adibi A, Puhan M, Gershon A, Aaron SD, Sin DD. Moving beyond AUC: decision curve analysis for quantifying net benefit of risk prediction models. Eur Respir J 2021;58:2101186. 10.1183/13993003.01186-2021 [DOI] [PubMed] [Google Scholar]
- 24. Stevens LA, Coresh J, Greene T, Levey AS. Assessing kidney function—measured and estimated glomerular filtration rate. N Engl J Med 2006;354:2473–83. 10.1056/NEJMra054415 [DOI] [PubMed] [Google Scholar]
- 25. Sud M, Sivaswamy A, Austin PC, Abdel-Qadir H, Anderson TJ, Khera R, et al. Validation of the European SCORE2 models in a Canadian primary care cohort. Eur J Prev Cardiol 2024;31:668–76. 10.1093/eurjpc/zwad352 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Hageman SHJ, McKay AJ, Ueda P, Gunn LH, Jernberg T, Hagström E, et al. Estimation of recurrent atherosclerotic cardiovascular event risk in patients with established cardiovascular disease: the updated SMART2 algorithm. Eur Heart J 2022;43:1715–27. 10.1093/eurheartj/ehac056 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Banerjee P, Watson C, Ali D. Discussing prognosis in heart failure: a questionnaire-based study of the patient’s view. JACC Heart Fail 2018;6:803–4. 10.1016/j.jchf.2018.04.001 [DOI] [PubMed] [Google Scholar]
- 28. Turgeon RD, Fernando S, Bains M, Code J, Hawkins NM, Koshman S, et al. Qualitative analysis of patient decisional needs for medications to treat heart failure. Circ Heart Fail 2024;17:e011445. 10.1161/CIRCHEARTFAILURE.123.011445 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. MacDonald BJ, Barry AR, Turgeon RD. Decisional needs and patient treatment preferences for heart failure medications: a scoping review. CJC Open 2023;5:136–47. 10.1016/j.cjco.2022.11.013 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Lee DS, Tu JV, Juurlink DN, Alter DA, Ko DT, Austin PC, et al. Risk-treatment mismatch in the pharmacotherapy of heart failure. JAMA 2005;294:1240–7. 10.1001/jama.294.10.1240 [DOI] [PubMed] [Google Scholar]
- 31. Swat SA, Helmkamp LJ, Tietbohl C, Thompson JS, Fitzgerald M, McIlvennan CK, et al. Clinical inertia among outpatients with heart failure: application of treatment nonintensification taxonomy to EPIC-HF trial. JACC Heart Fail 2023;11:1579–91. 10.1016/j.jchf.2023.06.022 [DOI] [PubMed] [Google Scholar]
- 32. Greene SJ, Bash LD, Tebbs KW, Hancock LN, Barlow SG, Coyle CR. Physician-reported reasons for not initiating guideline-directed medical therapy for heart failure. JACC Heart Fail 2024;12:2120–2. 10.1016/j.jchf.2024.08.002 [DOI] [PubMed] [Google Scholar]
- 33. Wahid M, Aghanya V, Sepehrvand N, Dover DC, Kaul P, Ezekowitz J. Use of guideline-directed medical therapy in patients aged ≥65 years after the diagnosis of heart failure: a Canadian population-based study. CJC Open 2022;4:1015–23. 10.1016/j.cjco.2022.08.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Saveski D, Kok M, Poon S, Rojas-Fernandez C, Virani SA, Honos G, et al. The Canadian Heart Failure (CAN-HF) registry: a Canadian multicentre, retrospective study of outpatients with heart failure. CJC Open 2025;7:1–9. 10.1016/j.cjco.2024.09.014 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Toll DB, Janssen KJM, Vergouwe Y, Moons KGM. Validation, updating and impact of clinical prediction rules: a review. J Clin Epidemiol 2008;61:1085–94. 10.1016/j.jclinepi.2008.04.008 [DOI] [PubMed] [Google Scholar]
- 36. Bobrowski D, Dorovenis A, Abdel-Qadir H, McNaughton CD, Alonzo R, Fang J, et al. Association of neighbourhood-level material deprivation with adverse outcomes and processes of care among patients with heart failure in a single-payer healthcare system: a population-based cohort study. Eur J Heart Fail 2023;25:2274–86. 10.1002/ejhf.3090 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The authors do not have permission to share GUIDE-IT trial data; interested researchers may seek to obtain these from the NHLBI Biologic Specimen and Data Repository Information Coordinating Center. R code used for model validation is available on https://github.com/resplab/papercode/tree/main/LIFE-HF.



