Predicting High-Risk Patients and High-Risk Outcomes in Heart Failure

“Prediction is very difficult...especially if it’s about the future.”

– Niels Bohr

Introduction

Over 6.5 million Americans are living with heart failure (HF) today and the prevalence continues to increase.¹ Despite substantial advances in medical therapy, many patients progress to end-stage disease with a 5-year absolute mortality rate of approximately 50%.² Moreover, HF is a significant burden on the United States health care system, accounting for ~800,000 hospitalizations in 2016 and projected to cost $69.7 billion annually by the year 2030.^1,3

Identifying high risk HF patients is therefore an important, yet challenging pursuit for clinicians and health care systems alike. Prognosis of individual patients with HF is highly variable in contemporary cohorts, and the risk of serious clinical outcomes such as mortality and hospitalization for HF can differ more than 20-fold.^4,5 Predicting adverse outcomes in patients with HF could theoretically help direct resources to patients at the highest levels of risk who might benefit the most from earlier and more intensive monitoring and treatment (e.g. targeted medications, cardiac devices, home monitoring systems, and social services), while avoiding unnecessary interventions and costs for patients at low risk.^6–9 Ideally, this would translate into improved outcomes and cost-efficiency in providing care for patients with HF. Given the importance of risk prediction for process improvement in HF, we aim to review the available evidence on prognostic variables and the current state of risk prediction for patients with HF. Additionally, we will discuss limitations of traditional risk modeling and provide a glimpse into the future of risk prediction in HF.

Established Risk Factors for Poor Outcomes in Patients with Heart Failure

A number of demographic and clinical variables have been explored as markers of increased risk for adverse outcomes in HF populations. Risk factors can vary substantially based on the outcome of interest or the population under study. Though a discussion of all of these risk factors is beyond the scope of this review, some of the prognostic features that have been consistently found to be significant drivers of clinically important outcomes, including mortality, hospitalization, and health-related quality of life (HRQOL) measures are summarized in Table 1. Risk factors are derived from various domains including demographics, clinical characteristics, functional status and HF grade (Table 2), comorbidities, vital signs, labs, imaging, hemodynamics, exercise capacity, medication and device therapy adherence, and social determinants of health.

Table 1.

Established Risk Factors for Poor Prognosis in Patients with Heart Failure

Risk Factors
Demographics14,15,77–81 – Age, sex, race/ethnicity
Systolic and diastolic dysfunction42–46,82
Heart failure etiology83,84 (e.g. ischemic vs.non-ischemic)
Functional class,85–87 heart failure stage,88 INTERMACS profile89
Body Mass Index90 – Cardiac cachexia and the “obesity paradox”
Comorbidities2,26,91–95 – Diabetes, renal dysfunction, liver disease, COPD, anemia, depression
Vital signs96–100 – Hypotension, HR, HR variability
Electrophysiologic risk factors101–105 – Arrhythmias (AF, VT/VF), QRS prolongation
Laboratory values106–111 – Sodium, biomarkers (natriuretic peptides, troponin, novel biomarkers - ST- 2, galectin-3, GDF-15)
Imaging 112–117 – Chamber enlargement, right ventricular dysfunction, strain imaging, delayed gadolinium enhancement
Hemodynamics118–124 – Elevated filling pressures, pulmonary hypertension, cardiac index, exercise hemodynamics
Exercise capacity125–127 – Peak VO2, 6MWT
Prior hospitalization for HF128–130
Underutilization of guideline directed medical therapies131–133
Social determinants of health134–137

Table 2.

ACCF/AHA HF Stage, NYHA Functional Class, and INTERMACS Profiles^85,88,89

ACCF/AHA Stages	NYHA Class	INTERMACS Profile
A – At risk for HF, but no structural heart disease
B – Structural heart disease, but no symptoms or limitations	I – Asymptomatic, no physical limitations
C – Structural heart disease with current or prior heart failure symptoms	II – Symptomatic with moderate exertion, slight limitation in activity
	III – Symptomatic with minimal exertion, marked limitation in activity	7- Advanced NYHA III
		6- Exertion limited
D – End-stage disease, refractory heart failure requiring specialized interventions	IV – Symptomatic at rest, inability to exert oneself	5- Exertion intolerant
		4- Resting symptoms
		3- Inotrope dependent, but stable
		2- Progressive decline
		1- Critical cardiogenic shock

Multivariable Risk Models For Patients with Heart Failure

Considerable effort has been devoted to developing multivariable risk scores to help summarize and simplify risk assessment so that it can be performed real-time in a clinical setting or embedded into systems of care. To date, hundreds of multivariable risk scores for predicting outcomes in HF populations have been developed. The majority of these risk models have been derived from multivariable statistical modeling such as logistic regression and Cox proportional hazards analysis. Some of these scores are derived directly from parameter estimates of regression models and involve complex calculations, while others have been translated into nomograms with integer scores given to different covariates based on their relative contribution to the overall risk.

As the risk factors that contribute to adverse outcomes can vary substantially based on the HF population (e.g HFrEF vs. HFpEF) or health care delivery setting (e.g. inpatient vs. outpatient), risk models have been developed for use in specific cohorts including those with chronic ambulatory heart failure, hospitalized heart failure (HHF), and HFpEF specifically. The most popular and well-validated clinical multivariable risk scores for HF are summarized in Table 3, and key features of these risk models are discussed in more detail below. Notably, many of the risk factors discussed previously are shared by several of these multivariable risk models.

Table 3.

Popular Clinical Multivariable Risk Prediction Models for Patients with Heart Failure

Risk Model	Multivariate Predictors	Outcomes
Chronic HF
Seattle Heart Failure Model 19 (2006)	Age, Sex, LVEF, NYHA Class, Weight, SBP, Labs (Hgb, Lymphs, UA, Chol, Na), Wide QRS/LBBB, Meds (including GDMT, allopurinol, statins, diuretics), Devices (ICD/CRT/PPM)	1, 2, and 5-year mortality
CHARM Risk Score 26 (2006)	Age, Sex, LVEF, NYHA Class, BMI, DM, AF, MR, MI, Prior HF hosp, HF duration, Current smoker, DBP, HR, Dependent edema, Pulmonary crackles, Dyspnea, Pulmonary edema, Cardiomegaly, ARB	All-cause mortality; CV death or HF hosp
CORONA Risk Score 27 (2009)	Age, Sex, LVEF, NYHA Class, BMI, DM, MI, CABG, AF, Claudication, HR, NT-proBNP, Cr, ApoA-1	CV death, non-fatal MI, or stroke
GISSI-HF Model 28 (2013)	Age, Sex, LVEF, NYHA Class, BMI, DM, COPD, AS, SBP, Labs (eGFR, Hgb, UA, NT-proBNP, hs-CTnT)	All-cause mortality
MAGGIC Risk Score 23 (2013)	Age, Sex, LVEF, NYHA class, BMI, DM, COPD, Recent diagnosis, Smoking status, SBP, Cr, Meds (B-blockers, ACEi/ARB)	1 and 3-year all-cause mortality
PARADIGM Risk Score 29 (2020)	Age, Sex, Race, LVEF, NYHA Class, BMI, DM, Region, HF duration, HF hosp, MI, Valve dz, BBB, PCI, PAD, SBP, Bili, AST, UA, Alb, K, Cl, Hgb, LDL, Trig, BUN, WBCs, NT-proBNP, B-blocker, ARNI	CV Death or HF Hosp; CV death; All-cause death
Hospitalized HF
EFFECT Model 138 (2003)	Age, COPD, Cirrhosis, Cancer, Dementia, Cerebrovascular dz, SBP, RR, Na, BUN, Hgb	30-day and 1-year mortality
OPTIME-CHF Model 9 (2004)	Age, NYHA Class, SBP, BUN, Na Prior HF hosp, SBP, BUN, h/o PCI	60-day mortality 60-day readmission
ADHERE CART Model 4 (2005)	SBP, BUN, Creatinine	In-hospital mortality
OPTIMIZE-HF Nomogram 38,39 (2008)	Age, LVEF, HF primary diagnosis, HR, SBP, Na, Cr Age, Weight, SBP, Na, Cr, Liver dz, Depression, Reactive airway dz	In-hospital mortality 60-day mortality or readmission
GWTG-HF Score 5 (2010)	Age, Race, COPD, SBP, HR, Na, BUN	In-hospital mortality
ESCAPE Risk Score 6 (2010)	Age, CPR/mechanical ventilation, Na, BNP, BUN, Diuretic, B-Blocker, 6MWT	6-month mortality
Refractory HF Risk Score 59 (2011)	Age, DM, Stroke, Arrhythmia, Na, BNP, BUN, Betablocker, KCCQ Score	Unfavorable QOL or death at 6 months
PROTECT HF Post-discharge Model 40 (2014)	Age, SBP, Na, Cr, BUN, Albumin, Prior HF Hosp, Peripheral edema	30/180-day mortality, All-cause Hosp, and/or CV Hosp
BIOSTAT-CHF Model 41 (2017)	Age, BUN, NT-proBNP, Hgb, B-blocker Age, prior HF hosp, edema, SBP, eGFR	All-cause mortality HF hosp
HFpEF
I-PRESERVE Score 46 (2011)	Age, LVEF, Ischemic etiology, Prior HF hosp, QOL, DM, MI, COPD/asthma, HR, eGFR, Neutrophils, NT-proBNP	All-cause mortality or CV hosp, all-cause mortality, HF hosp or mortality
ARIC Score 47 (2017)	Modified EFFECT score + Race, BMI, AF, HR, Hypoxia, Natriuretic peptides	28-day and 1-year mortality

The Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) Statement was a consortium document released in 2015 that aimed to improve the reporting of research surrounding the development and validation of risk scores.^10,11 The characteristics of model derivation and validation cohorts, outcome of interest, statistical analysis methods, handling of missing data, and measures of performance (e.g. discrimination and calibration) of the model should be clearly defined. Model discrimination, or the ability of a model to distinguish between cases and non-cases, is often measured by the concordance statistic (c-statistic) which is equal to the area under the receiver operator statistic curve (AUC) for a logistic regression model. By convention, an AUC <0.70 indicates inadequate discrimination for clinical use, while an AUC of 0.7–0.8 is considered “acceptable” and an AUC >0.8 is considered “excellent”.¹² Model calibration refers to a model’s ability to accurately predict absolute risk. This is often reported graphically, but there are also several statistical tests of overall calibration which are available.¹³ When available, we will comment on the features discussed above, however, frequently these are not reported for existing heart failure risk models.

Chronic Ambulatory Heart Failure

Numerous risk models have been developed to predict clinical outcomes in cohorts of ambulatory HF patients.^14–18 The majority of these models have focused on all-cause mortality as the outcome of interest and have examined more intermediate to long term outcomes given short term event rates are typically low in ambulatory populations. Notably, most of the risk models for patients with chronic HF share a core set of common risk factors, including age, sex, NYHA class, BMI, DM, SBP, and indices of renal function.

Perhaps the most popular and thoroughly validated risk model in the chronic ambulatory HF population is the Seattle Heart Failure Model (SHFM).¹⁹ The SHFM was derived from a clinical trial cohort (PRAISE1 trial)²⁰ of 1,125 heart failure patients with a reduced EF (≤30%) and NYHA class III or IV symptoms. The components of the risk model were selected via a multivariable stepwise Cox proportional hazards model designed to predict 1, 2, and 3-year mortality. The SHFM was externally validated in 5 separate clinical trial cohorts totaling 9,942 patients from varying populations (HFrEF only, mixed HFrEF/HFpEF, varying functional status and age) with discriminative performance ranging from 0.682 to 0.810 for predicting 1-year mortality and excellent calibration (correlation coefficient ≥ 0.97 between predicted and actual survival for all validation cohorts). The model incorporates 14 continuous variables and 10 categorical variables, including demographics, labs, EKG findings, medications, and device therapy. The authors created an interactive online calculator to facilitate clinical use (https://depts.washington.edu/shfm). Since the SHFM was derived from a HF clinical trial cohort almost 30 years ago prior to the widespread use of contemporary GDMT, performance in more modern cohorts, especially at the individual level, has been highly variable and modest at best²¹. Interestingly, the SHFM appears to overestimate survival in contemporary HF cohorts, likely explained by the fact that effect sizes for individual medications and ICD/CRT-D use were imputed from results of large randomized clinical trials available at the time.^18,22 Often, these trials were conducted in a cohort that largely was not on a background of contemporary GDMT, thus likely overestimating individual effect sizes (and consequently overall additive benefit) of these therapies.

More recently, the Meta-Analysis Global Group in Chronic Heart Failure (MAGGIC) developed a heart failure risk score using data on 39,372 patients from 30 cohort studies in an attempt to improve generalizability.²³ The derivation cohorts included patients with HFrEF and HFpEF from both randomized clinical trials and observational studies. Individual components of the model were determined using Poisson regression models with forward stepwise variable selection for the outcome of mortality. Missing data were handled via multiple imputation analysis, a more robust method to handle missing variables than the methods employed in most other risk prediction model derivation strategies. A total of 13 independent predictors were included in the final model, and separate models were derived for patients with HFpEF and HFrEF. Much like the SHFM, the authors provide an online calculator for computing the MAGGIC risk score at the point of care (www.heartfailurerisk.org). Subsequently, the MAGGIC score has been validated in 51,043 patients from a large HF registry cohort (AUC = 0.741 for predicting 3-year mortality)²⁴ and specifically in 407 HFpEF patients from a single institution real-world cohort (AUC = 0.74 for predicting mortality)²⁵. In general, calibration plots in these validation cohorts showed slight underestimation of risk of mortality in high-risk patients and overestimation of risk of mortality in low-risk patients, with better calibration for predicting risk of hospitalization events than the SHFM.

Other notable models that have been developed for predicting outcomes in chronic ambulatory HF include the Candesartan in Heart Failure Assessment of Reduction in Mortality and Morbidity (CHARM)²⁶, Controlled Rosuvastatin Multinational Trial in Heart Failure (CORONA)²⁷, and Gruppo Italiano per lo Studio della Streptochinasi nell’Infarto Miocardico-Heart Failure (GISSI-HF)²⁸ risk models, all of which were derived from clinical trial cohorts. Models derived in clinical trial cohorts are limited by non-representative patient populations due to significant under-recruitment of women and minorities as well as strict inclusion/exclusion criteria and thus have limited generalizability.

Recently, several popular risk scores for chronic ambulatory HF patients were directly compared in a large contemporary European HF registry of over 6,000 patients.¹⁸ The MAGGIC score had the best overall discriminatory power for estimating one-year mortality (AUC 0.743), but all scores were only modest predictors (AUC 0.714–0.743). Moreover, calibration was poor across all risk scores with the MAGGIC, CHARM, and GISSI-HF scores underestimating survival (given they were derived in patients with low background use of contemporary GDMT) and SHFM overestimating survival as explained previously.

Importantly, all of these risk scores were developed prior to the advent of angiotensin-receptor neprilysin inhibitors (ARNIs). A novel risk score was recently published using data from 8,399 patients in the Prospective Comparison of ARNI with ACEI to Determine Impact on Global Mortality and Morbidity in Heart Failure (PARADIGM-HF) trial of sacubitril-valsartan.²⁹ Three separate models were derived for outcomes of cardiovascular death, all-cause mortality, and the composite of cardiovascular death or HF hospitalization (AUC of 0.73, 0.71, and 0.74 at one year, respectively). The models were also validated using data from ~7,000 patients in the Aliskiren Trial to Minimize Outcomes in Patients with Heart Failure (ATMOSPHERE) and ~19,000 patients from the Swedish Heart Failure registry with similar performance. One of the risk factors included in all of the final models was lack of treatment with ARNI – this risk model may therefore be more appropriate for patients on contemporary GDMT, though given its novelty further validation in prospective cohorts will be necessary to support its clinical use.

Hospitalized Heart Failure

Although there has been progress in improving symptoms and in-hospital mortality for patients hospitalized with HF, events after discharge remain unacceptably high with up to 15% of patients dying after discharge and 1 in 4 patients being readmitted within 90 days.^30,31 Therefore, patients with hospitalized heart failure (HHF) represent a high-risk group for which processes of care designed to improve outcomes with increased resources, intensive monitoring, or targeted interventions may be especially effective. However, even among HHF patients, risk is highly variable. A number of multivariable risk scores have thus been developed specifically to predict subsequent outcomes in patients hospitalized with acute decompensated HF.^32–37 The majority of these scores focus on short term outcomes of in-hospital mortality or short-term mortality or readmission after discharge. Unlike risk models for chronic ambulatory HF, predictors of outcomes for patients with HHF are more varied, though a few risk factors – namely age, SBP, and renal indices – are shared by many of these models.

In-Hospital Mortality

A few popular clinical risk scores have focused on the outcome of in-hospital mortality, including the Acute Decompensated Heart Failure National Registry (ADHERE) classification and regression tree (CART) model,⁴ Organized Program to Initiate Lifesaving Treatment in Hospitalized Patients With Heart Failure (OPTIMIZE-HF) nomogram,³⁸ and Get With the Guidelines-Heart Failure (GWTG-HF) score.⁵ The ADHERE CART model was derived in a large national registry of ~33,000 patient records with HHF and validated in another ~33,000 patient records using a temporal validation scheme. The resulting ADHERE CART model is a simple tool for risk assessment at the bedside using only 3 predictive variables – blood urea nitrogen (BUN), systolic blood pressure (SBP), and serum creatinine. The discriminatory power of this score was quite modest (AUC 0.67–0.69 in derivation and validation cohorts), however, its allure lies in its simplicity and ease of use at the bedside. The OPTIMIZE-HF nomogram was derived from another national hospital-based registry of almost 50,000 patients in over 250 hospitals. A simplified nomogram for predicting in-hospital mortality was developed using the top 8 predictors from a multivariable logistic regression model and a separate nomogram was later developed for post-discharge outcomes of 60-day mortality and 60-day mortality or HF hospitalization.³⁹ The in-hospital mortality nomogram showed good discrimination with internal validation by bootstrapped resampling (AUC 0.753) as well as in external validation cohorts from the OPTIME-CHF clinical trial (AUC 0.756) and the ADHERE clinical registry (AUC 0.746). Performance was worse, however, for post-discharge nomograms, particularly for the prediction of HF readmission within 60-days. The GWTG-HF Score was derived from a more contemporary registry of approximately 40,000 patients with both HFrEF and HFpEF using a multivariable logistic regression model and internally validated in a subset of the patients from this same registry. Like the ADHERE CART model, age, SBP, and BUN were the strongest contributors to risk of in-hospital mortality, whereas HR, presence of COPD, and serum Na were weaker predictors in the final model. The model had good discriminatory power in both derivation and validation cohorts (AUC ~ 0.75) regardless of baseline LV systolic function. Calibration plots revealed that the model overestimated risk of mortality in high risk patients.

Post-discharge Outcomes

Though assessing risk of adverse inpatient outcomes may help triage patients who require more immediate attention or intensive monitoring/treatment while hospitalized, post-discharge outcomes remain the greatest source of morbidity in HHF patients. Thus, predicting post-discharge outcomes may be more consequential. Clinical risk prediction models for post-discharge outcomes in patients with HHF have focused on outcomes of all-cause mortality, cardiovascular mortality, all-cause readmission, readmission for HF or cardiovascular causes, or composite endpoints of these outcomes.

The Enhanced Feedback for Effective Cardiac Treatment (EFFECT) and The Evaluation Study of Congestive Heart Failure and Pulmonary Artery Catheterization Effectiveness (ESCAPE) risk scores focused on the outcome of post-discharge mortality. The EFFECT model was derived from a retrospective community-based cohort of 4,031 patients at multiple hospitals in Canada using a multivariable logistic regression model. Predictors of mortality at both 30-days and 1-year were similar, and a composite risk score was able to adequately discriminate very low risk (8% mortality at one year) from high risk (79% mortality at one year) patients with adequate discrimination in the derivation (AUC = 0.77) and internal validation (AUC=0.76) cohorts. The ESCAPE risk score was derived from a North American clinical trial cohort of 433 patients using a Cox proportional hazards model. A simplified risk score for mortality at 6 months was constructed using 8 clinical variables with integer values assigned for each variable’s contribution to the overall risk, with BUN and BNP being the most important variables in the model. The model showed good discrimination in derivation (AUC = 0.76) and internal validation with bootstrapping (AUC = 0.78) cohorts, but when externally validated in a clinical trial cohort from the FIRST (Flolan International Randomized Survival Trial) study (without values for BNP or diuretic use available) performed much worse (AUC = 0.65).

In general, models designed to predict non-fatal endpoints such as readmissions perform worse than those models designed to predict mortality. Additionally, predictors of mortality are typically different than those for readmissions. This may reflect the complexity of clinical and non-clinical factors that contribute to risk of hospital readmission and suggests that there may be risk factors that have not yet been identified or that are simply not captured in contemporary data sets. For example, the OPTIME-CHF (Outcomes of a Prospective Trial of Intravenous Milrinone for Exacerbations of Chronic Heart Failure) clinical trial cohort of 949 patients was used to derive two separate models to predict 60-day mortality or the composite of death and rehospitalization at 60 days.⁹ The model for predicting mortality had better discriminatory power (AUC 0.77) than the model for predicting the composite endpoint (AUC 0.69). Similarly, the Placebo-Controlled Randomized Study of the Selective A1 Adenosine Receptor Antagonist Rolofylline for Patients Hospitalized With Acute Decompensated Heart Failure and Volume Overload to Assess Treatment Effect on Congestion and Renal Function (PROTECT) derived risk models for 30-day mortality (AUC 0.79) and 180-day mortality (AUC 0.74) performed better overall than those for predicting 30-day all-cause mortality or hospitalization (AUC 0.66) and 30-day death or hospitalization for cardiovascular reasons (AUC 0.66).⁴⁰ More recently, the BIOSTAT-CHF project was developed specifically to derive and validate risk prediction models from a large European cohort of 2,516 patients across 69 hospital centers in 11 countries.⁴¹ Though this cohort did not exclusively include patients with HHF, the majority of patients were enrolled during an admission for worsening heart failure. Risk models for predicting outcomes of mortality, hospitalization for HF, and the composite outcome of these endpoints were derived from multivariable backward stepwise Cox proportional hazards regression models and validated in an external cohort of 1,738 patients from Scotland. Discriminatory power for the mortality model (AUC 0.73) was remarkably better than for the HF hospitalization model (AUC 0.64) in the external validation cohort and predictors included in respective models were markedly different (Table 3).

Heart Failure with Preserved Ejection Fraction

All of the aforementioned risk models either excluded patients with HFpEF or were derived from mixed populations of HFrEF and HFpEF enrolled in clinical trials, registries, or community cohorts. HFpEF is an increasingly common cause of HF hospitalization in an aging population.⁴² Moreover, the clinical characteristics of patients with HFpEF are distinct – risk models derived from mixed cohorts may fail to capture unique risk factors which contribute to adverse outcomes in this population.^42–45 Risk scores that have been developed specifically for patients with HFpEF are lacking, though there are a few models that warrant mention.

The I-PRESERVE score was derived from the Irbesartan in Heart Failure with Preserved Ejection Fraction clinical trial cohort of 4,128 patients with chronic HFpEF using a forward selection stepwise multivariable Cox proportional hazards model for predicting the outcomes of all-cause mortality, all-cause mortality or HF hospitalization, and HF death or hospitalization.⁴⁶ Interestingly, the variables included in the final derived models are quite similar to those included in the models developed in other chronic HF cohorts discussed previously (Table 3). The model was internally validated with bootstrapped resampling with good discrimination (AUC 0.711–0.765), but was not externally validated and has not been directly compared to other risk scores for chronic HF. In contrast, the Atherosclerosis Risk in Communities (ARIC) study-derived risk score was developed to specifically predict outcomes among patients with HFpEF hospitalized with acute decompensated HF.⁴⁷ The authors used a unique approach to build their model using the previously validated EFFECT score (modified to include variables available in the ARIC cohort) as a baseline and evaluating additional candidate variables that further improved prognostication of 28-day or 1-year mortality risk. Interestingly, the final model included several variables that were related to comorbidities implicated in the pathophysiology of the HFpEF syndrome. This model had good discrimination in an internal validation cohort (AUC 0.73 for 28-day and 0.71 for 1-year mortality) with a modest improvement over the EFFECT model alone (AUC 0.70 for 28-day and 0.68 for 1-year mortality). Given the evolving landscape of HF and the epidemiological impact of HFpEF, future research is clearly warranted to further refine risk prediction in this population.

Lack of Evidence for Clinical Utility of Risk Prediction Models

The literature is teeming with HF risk prediction models, however, there is comparatively very little evidence that incorporating risk prediction into clinical practice actually influences management or improves outcomes. Model development for the sake of prediction itself has become a popular “educational hobby”,⁴⁸ but in order for risk models to be used to improve processes of care, they should be interpretable and actionable. Most models include risk factors that are not modifiable (e.g. age) and it is unclear what, if any, interventions could be implemented to alter prognosis in patients identified as high risk. Additionally, it is unclear what thresholds of risk should be considered “high” or whether there are meaningful thresholds for intervention. Specific interventions to improve adherence to therapies of proven benefit, multidisciplinary disease management programs, and transitional hospital discharge programs have shown promise in improving outcomes (particularly reduction of readmission rates) for patients hospitalized with HF in general,^49–55 but whether basing these interventions on prognostic information derived from risk models provides any incremental benefit (or reduced costs/resource utilization) is unknown. In those circumstances where prognosis may not be appreciably altered through intervention, there is also limited evidence risk prediction may aid in patient-centered communication regarding risk, preparation for the possibility of death from HF, or referral for advanced HF therapies or interdisciplinary palliative care.⁵⁶

Horne et al. recently published the results of a clinical trial of risk score-guided multidisciplinary team-based care for approximately 6,000 patients hospitalized with HF in 8 hospitals from the Intermountain healthcare system.⁷ Patients who were deemed high risk were identified via an automated risk score calculator embedded in the electronic health record (EHR) and were managed via a distinct care pathway that included personalized changes in their inpatient care, higher-intensity post-discharge follow up, and a more precise discharge plan than lower risk patients. Compared to historical controls, the risk-score guided multidisciplinary care plan resulted in 21% lower 30-day readmission and 52% lower 30-day mortality among high risk patients. Importantly, outcomes among lower risk patients were similar in the intervention and historical control group. Additionally, though inpatient costs were higher in the intervention group, this was balanced by lower post-discharge costs and consequently, there was no difference in overall cost between intervention and control groups. This suggests that the risk-score guided strategy led to improved cost-efficiency in caring for HHF patients.

While these results are promising, this study unfortunately stands alone as the only large clinical trial of its kind. Moreover, reliability of the results may be limited by the use of historical controls rather than randomization, which may have led to confounding (though this was mitigated by the use of a staged crossover design and multivariable analysis that adjusted for potential confounders). Going forward, it is crucial that the value of risk prediction modeling for improving processes of care is rigorously evaluated in randomized clinical trials. The recent high-profile failure of the “hotspotting” program designed reduce spending and improve quality of care among healthcare “superutilizers” underscores the importance of confirming the utility of any presupposed pragmatic intervention in randomized clinical trials before it is adopted into routine practice.⁵⁷ The Risk Evaluation And Its Impact on ClinicAL Decision Making and Outcomes in Heart Failure (REVeAL-HF) randomized clinical trial is currently enrolling patients within the Yale New Haven Health System to evaluate the influence of an automatically generated risk prediction score “pop-up” in the EHR on the management and outcomes of HHF patients.⁵⁸ This is hopefully the first of many randomized clinical trials of risk score-based interventions to come.

Barriers to Implementation of Risk Prediction Models in Clinical Practice and Systems of Care

Despite all of the efforts to develop a well-validated risk prediction model for patients with HF, adoption of risk modelling to assess prognosis in clinical settings is quite low. Less than 1% of patients enrolled in a long-term heart failure registry were offered information regarding their prognosis calculated from a risk model by their treating clinician.¹⁸ One major reason for this is likely the lack of evidence for the clinical utility of implementation of HF risk scores, however, there are also several putative barriers to the utilization of heart failure risk predictions scores in practice.

First, the actual performance of these models, especially outside of their derivation cohorts, is disappointing. HF risk models are often derived from single-center populations or clinical trial cohorts without validation in an external cohort. Given the risk factors for adverse outcomes can vary substantially in HF cohorts with different characteristics (e.g. HFrEF vs. HFpEF, clinical trial vs. registry, race/ethnicity, sex, and socioeconomic diversity), the derivation cohort may be very different than the cohort to which the model is applied. Moreover, models derived from historical cohorts may fail to account for the changing landscape of HF and the effect that contemporary therapies have on outcomes - it can be difficult for risk model development to keep pace with therapeutic innovation. Outcomes modeled by different risk scores are often highly variable, including all-cause mortality, CV death, all-cause hospitalizations, HF hospitalizations, and composites of these individual endpoints. Each of these outcomes is associated with a different set of clinical features – indeed, even the 2 major causes of death from HF, sudden cardiac death and progressive pump failure, have distinct risk factors. Additionally, non-fatal events may be more difficult to predict than mortality. We must think carefully about which outcomes are important to predict and prevent – perhaps patient-centered outcomes would be a more appropriate metric. The refractory HF risk score (Table 3) introduced by Allen et al. is unique as it focused on a primary outcome of unfavorable future HRQOL (as assessed by the Kansas City Cardiomyopathy Questionnaire [KCCQ]) or death in the 6 months following admission.⁵⁹ Not only is this a more meaningful metric to patients than readmission rates, it may also identify patients who would benefit from end-of-life discussions or palliative care referral.

Second, HF risk scores rely on multivariable statistical modelling of population data. Selection of predictors is often based on a priori knowledge of “traditional” clinical risk factors for the outcome of interest and/or stepwise methods for multivariable selection based on statistical significance, both of which can be problematic and result in significant bias. Predictors included in models are often based on static measurements of clinical variables, rather than taking advantage of longitudinal dynamic changes in repeated measurements of risk factors which may be informative. Moreover, traditional statistical modeling lacks the ability to model complex, unstructured multi-dimensional data such as raw imaging data or free text from clinical reports. Most available models also fail to indicate how missing data is handled, with even fewer employing robust methods for handling missing data such as multiple imputations. Additionally, popular HF prediction models ignore the presence of competing risks such as from non-cardiac death, heart transplantation, or mechanical circulatory support – an especially salient issue in the advanced heart failure population where competing risks are common.^60,61 Given statistical models predict outcomes on a population level, it is unsurprising that in practice, these scores perform quite poorly on an individual level. Allen et al. found that popular HF risk models (including SHFM and MAGGIC) performed unacceptably poorly at the individual level, with the majority of patients who died within the next year having greater than a 75% model-predicted probability of survival.²¹

Finally, provider inertia and skepticism present a significant barrier to adoption of risk prediction models in practice, and a majority of providers believe that risk models would not change their practice or add value to their clinical evaluation.^62,63 Up to three quarters of clinicians surveyed rarely or never use cardiovascular risk prediction scores.⁶² One of the most common reasons provided for not utilizing risk models is the inconvenience and time required to manually calculate risk scores. Though many online calculators exist for popular HF risk scores, these still require manual input of individual risk factors, which can be prohibitively time-consuming especially in the context of increasing pressures on time constraints of the clinical encounter. An effective risk score should therefore be incorporated directly into the EHR with automatic calculation, but there are few current examples of such integration. Providers also express doubts over the utility of risk prediction models, believing these models often oversimplify the risk assessment process and that their own ability to predict risk is superior to that of any model. However, there is evidence that this belief is unfounded. Both providers and patients have proven notoriously bad at predicting prognosis, with providers tending to underestimate survival and patients tending to overestimate survival.^64–66

Promise of Artificial Intelligence For Risk Prediction in Heart Failure

Rapid advances in computational power, the digital “big data” revolution, and innovations in mathematical algorithms have led to a recent resurgence of enthusiasm surrounding the utility of artificial intelligence (AI) in a number of industries. The recent popularity of AI has been propelled by the development of novel machine learning (ML) models, which are computer algorithms designed to learn without explicit programming, instead relying on patterns and inference from data (Figure 1). ML has been applied to problems throughout various fields of medicine, particularly radiology, dermatology, and pathology. We are also starting to see the emergence of ML technologies in cardiovascular medicine.

Figure 1. Artificial Intelligence and Machine Learning.

Schema of the relationship between artificial intelligence, machine learning, and “deep” learning. SVM = Support vector machines, RF = random forests, KNN = k-nearest neighbor, PCA = principal components analysis, DNN = deep neural network, CNN = convolutional neural network, RNN = recurrent neural network, GAN = generative adversarial network, VAE = variational autoencoder

Given the disappointing performance and barriers to use of existing risk scores for HF, there has been substantial interest in novel methods for risk stratification. ML is able to model complex multidimensional interactions within data and ML models are designed to optimize prediction on an individual level rather than a population level.⁶⁷ Moreover, the learning aspect of ML means that these models are modular and adaptable – ML models can be updated to predict outcomes in different populations and more contemporary cohorts as newer data becomes available. Given that ML models are by definition computer programs, they are easily integrated into existing EHR and administrative systems – enabling automated calculation of risk scores towards relieving rather than increasing the burden of resource and time constraints on clinicians and health systems.

Machine Learning for Risk Prediction in Heart failure

Though ML for risk prediction in HF populations is still in its nascence, there are several promising examples of ML-derived risk prediction algorithms that have already been developed (Table 4). Compared to statistically derived risk models, the majority of ML HF risk models incorporate large amounts of features and are derived from very large data sets. Features incorporated into these models include demographic, clinical, physical exam, laboratory, ECG, echocardiographic, medication, procedure, hemodynamic, socioeconomic, quality of life, and administrative claims-based data. Additionally, the majority of these ML-based HF risk models used cross-validation only to assess model performance, with very few using a hold-out validation set and none evaluating model performance in a completely independent test set after training was complete.

Table 4.

Machine Learning Models For Predicting Outcomes in Patients with HF

Study	Population	Target (Outcomes)	Features (Predictors)	Models	Performance	Validation/Test sets
Shah et al. (2015) 68	Recently hospitalized outpatients with HFpEF – N = 420	CV Hosp, HF Hosp, and Death	46 features - clinical variables, physical characteristics, labs, ECG, and echo parameters	Phenomapping via model-based clustering + Risk prediction via SVM	Combined model AUC 0.67 for primary outcome	Hold-out validation -107 patients with HFpEF
Loghmanpour et al. (2016) 139	Pre-LVAD patients – N = 10,909	Right ventricular failure after LVAD	33 features - pre-op demographics, clinical features, labs, echo, hemodynamics, others	Tree-augmented naïve Bayesian network	AUC 0.903 for acute RVF (vs. Drakos 0.547 and RVFRS 0.498)	10-fold cross-validation
Mortazavi et al. (2016) 140	Recently discharged HF patients – N = 1,653	30 and 180-day all-cause readmission and readmission for HF	236 features – demographics, labs, physical exam, quality of life, SES	RF, GBM, RF/SVM, RF/LR vs. LR	All-cause hosp - RF (best) (AUC 0.628 vs. 0.533 LR) HF hosp – GBM (best) (AUC 0.678 vs. 0.543 LR)	50% random hold-out validation
Frizzel et al. (2017) 141	Inpatients with HF from GWTG-HF registry – N = 56,477	30-day all cause readmission	86 features - demographics, SES, medical history, characterization of HF, meds, vital signs, weights, labs, treatment, and discharge interventions	Tree-augmented naïve Bayesian network, LR with LASSO, RF, and GBM	AUC 0.607–0.618 vs. 0.624 for LR	30% random hold-out validation
Ahmad et al. (2018) 142	Swedish HF registry–N = 44,886	1-year survival	86 features - demographics, clinical, laboratory, and imaging data	RF, K-means clustering (using top 8 features from RF)	RF AUC 0.83, 8 cluster variables AUC 0.78	10-fold cross validation
Golas et al. (2018) 143	Patients with a history of HF admission – N = 11,510	30-day all cause readmission	3,512 features - demographics, admissions, diagnosis, procedure, medication, and lab data; ûnstructured text data from clinical notes	Deep unified neural networks, bag-of-words model for text	AUC 0.705 vs. 0.664 for LR	10-fold cross validation
Adler et al. (2019) 144	Patients with initial occurrence of HF – N = 5,882	Death within 90 days (high-risk) or no HF event after 800 days (low-risk)	8 features - DBP, Cr, BUN, Hgb, WBC, Platelets, Albumin, RDW	GBM	AUC 0.88 (0.81 and 0.84 in external test cohorts)	50% random hold out validation, external test sets in in 2 separate cohorts
Angraal et al. (2019) 145	Outpatients with HFpEF in TOPCAT – N = 1,767	All-cause mortality and HF Hosp through 3 years follow-up	86 features – demographic, clinical, laboratory, and electrocardiography data; KCCQ Scores	LR with LASSO, RF, GBM, SVM	RF (best) – AUC 0.72 mortality and 0.76 HF Hosp (vs. 0.66 and 0.73 for LRR)	5-fold cross-validation
Desai et al. (2020) 146	Patients with HF by Medicare claims – N = 9,502	Mortality, HF Hosp, High Cost, Home Days Lost	54 features - demographics, HF-related variables, meds, comorbidities, and claims-based frailty-index and SES-index	LR with LASSO, CART, RF, GBM	AUC for GBM (best) vs. LR Mortality 0.727 vs. 0.7124 HF Hosp 0.745 vs. 0.707 High Cost 0.733 vs. 0.734 Home Days Lost 0.790 vs. 0.781	10-fold cross validation, Hold-out test set 3,389 patients

Hosp = hospitalization, SES = socioeconomic status, DBP = diastolic blood pressure, Cr = creatinine, BUN = blood urea nitrogen, Hgb = hemoglobin, WBC = white blood cells, RDW = red cell distribution width, KCCQ = Kansas City Cardiomyopathy Questionnaire, RF = random forest, LR = logistic regression, GBM = gradient boosting machine, LASSO = least absolute shrinkage and selection operator, CART = classification and regression tree, SVM = support vector machine

Most efforts to develop a ML-derived HF risk model have employed supervised learning algorithms. Several authors have compared HF risk prediction models derived using supervised learning to traditional statistical methodology (namely logistic regression [LR]). Some have found that ML models provided no improvement over LR, while others found modest improvement for ML over LR. Typically, the highest performing ML models are derived from algorithms using ensembles of decision trees, including random forests (RFs) and gradient boosting machines (GBMs). It should be noted, however, that even the best performing ML models showed discriminative power (AUC 0.62 to 0.73) and calibration for prediction of mortality and readmission that is on par with previously reported statistical models in the literature.

Shah et al. utilized a combined approach of unsupervised and supervised learning to identify distinct phenotypes, then predict risk of disease outcomes among a cohort of 397 patients with HFpEF.⁶⁸ Clinical, laboratory, ECG, and echocardiographic data was prospectively collected from 397 patients with HFpEF enrolled in an observational study at a single institution. A total of 46 continuous features (after filtering those that were highly correlated) were utilized in a penalized model-based clustering algorithm. From this analysis 3 clinically distinct phenotypes of HFpEF were identified, each with significantly different risks of mortality or cardiovascular hospitalization. Additionally, using the supervised learning technique of support vector machines (SVMs), a model that included the initial 46 features plus the derived phenogroup feature had fair discriminative ability in predicting the composite outcome of HF hospitalization, cardiovascular hospitalization, or death (AUC 0.67). Ahmad et al. also used a combined approach to develop a risk prediction model among patients in the Swedish Heart Failure Registry by applying supervised learning BEFORE unsupervised learning to model risk.⁶⁹ First, a RFs algorithm was used to identify predictors of 1-year survival. Discrimination based on the full RFs model (AUC 0.828) was excellent and higher than that historically reported in popular statistical models for predicting 1-year mortality (e.g. SHFM and MAGGIC). Next, unsupervised learning via K-means clustering was applied to the 8 most predictive features identified from the supervised learning algorithm to identify 4 clinically relevant subgroups of HF with distinct risk profiles. Patients in the highest risk cluster had only a 69% one-year survival, compared to 93% one-year survival in the lowest risk group. There are a few other recent examples in the literature of unsupervised learning to identify clusters of patients with distinct risk profiles for clinically meaningful outcomes among patients with chronic heart failure and those hospitalized with acute decompensated heart failure.

Limitations of Machine Learning Risk Models

Thus far, we have yet to see any real adoption of ML risk models for HF in clinical practice or systems of care. ML-derived risk models have generally failed to live up to inflated expectations, showing only modest improvement over traditional statistical models. While some of these shortcomings may simply be due to the fallacy of believing one can predict the future, there are other considerations regarding the potential pitfalls and limitations of ML risk models that should be noted.

Many of the problems that have plagued their statistical predecessors have also troubled ML-derived models. Existing ML models for risk prediction in HF have generally failed to account adequately for time to event outcomes, competing risks, or missing data; though strategies and algorithms to address these problems exist and research in this area is active.^70,71 Additionally, the majority of ML models utilize discrete values from a single time-point as input features, ignoring the predictive ability of longitudinal data that changes over time. Many ML models have an exceptional ability to analyze time-series data and adjust risk probabilities as new information becomes available, thereby accounting for some of the intrinsically stochastic nature of risk prediction. Further, we should not expect any significant performance boost from simply employing a novel methodology for analyzing data when using the same features/predictors (demographics, vital signs, labs) that were typically used in statistical models⁷². Indeed, one of the strengths of the complexity of ML models, particularly deep learning models, is the ability to handle complex, raw data as inputs as long as the quantity of data is sufficient for the model to learn meaningful representations.⁷³

There are also new considerations presented by ML techniques. Given their complexity, ML models are prone to overfitting to training data, which can limit generalizability to external datasets.^67,73 It is therefore crucial that any ML model is validated multiple times in independent data sets before being adopted for clinical use. Moreover, there are concerns regarding the interpretability of ML models as they can be perceived as being a “black box” due to their focus on the result of the model rather than transparency of the model itself.⁷⁴ Further, ML models often consider thousands of features so it may be difficult to decide which risk factors are most important and consequently, how one could intervene to modify this risk. To address these concerns, new visualization techniques are being developed in order to codify importance of individual features in ML algorithms (e.g. heatmap visualization of layer class activation in convolutional neural networks) and make these models more actionable.⁷⁵ As with statistically derived risk models, randomized clinical trials assessing the influence of using ML-based HF risk modelling on management and clinically important outcomes are imperative.

Finally, the era of ML brings new cultural and sociological concerns that should be noted.⁷⁴ ML models, particularly supervised learning models, often rely on massive amounts of well-labeled data that requires a significant degree of human input, though there are many strategies to reduce this reliance on human “micro-work” such as semi-supervised learning, data augmentation, transfer learning, fine-tuning, and feature extraction. There is also inherent risk in introducing human implicit bias when developing a ML model, and there are countless examples of unintended and often discriminatory results of ML models.⁷⁶ Lastly, the human-machine interface has become an important consideration and popular culture has promoted a general fear and resistance to the adoption of these technologies that can be perceived to be replacing human input, especially among healthcare providers. However, there are abundant examples in the literature that “augmented intelligence”, that is a human being with the assistance of an artificially intelligent system, almost always outperforms a human or machine alone.⁷²

Summary and Future Directions

In summary, predicting outcomes in patients with HF has proven difficult. While many individual risk factors for adverse future outcomes are known, attempts at developing high-performing risk models in various HF populations using statistically based methods have been generally underwhelming. Moreover, there is currently a lack of evidence to support the notion that implementation of these risk scores into clinical practice or systems of care actually improves outcomes. Newer methods of risk prediction utilizing machine learning technologies offer promise, but thus far have failed to live up to the hype of expectations.

While artificial intelligence may play a role in the future of risk prediction in HF, many key challenges remain. We must think more broadly about the predictors or features included in risk models, including “source” data from unstructured clinical notes text, imaging, pathology slides, hemodynamic and ECG tracings, sensor data (ambulatory monitors, pulmonary artery sensors, and consumer wearables), and other “omics” data (genomics, proteomics, metabolomics, etc.). More robust models can be built using agnostic deep learning algorithms, and perhaps the key to predicting the as-of-yet unpredictable is in some of the unrecognized or “hidden” features in this raw data. Additionally, future research is needed to develop better strategies to communicate risk to patients and families in a way that enhances comprehension, identify clinically meaningful thresholds to implement clinical action items (e.g. telemonitoring, pulmonary artery pressure monitoring, multidisciplinary care initiatives), and rigorously evaluate risk-score based strategies in randomized clinical trials. We should strive to constantly assess for bias and ensure that risk models are both equitable and actionable on an individual level – that we are not just predicting the future, but that we are able to intervene and change that future.

Synopsis:

Identifying patients with heart failure at high risk for poor outcomes is important for patient care, resource allocation, and process improvement. Although numerous risk models exist to predict mortality, hospitalization, and patient-reported health status, they are infrequently used for several reasons, including modest performance, lack of evidence to support routine clinical use, and barriers to implementation. Artificial intelligence has the potential to enhance the performance of risk prediction models, but has its own limitations and remains unproven.

Key Points:

• Identifying patients with heart failure who are at high risk for poor outcomes is important for patient care, resource utilization, and process improvement

• Several risk models exist to identify high risk patients with heart failure, based on traditional statistical risk modeling methodology

• Risk models for heart failure are infrequently utilized in practice due to their modest performance outside validation cohorts, lack of evidence to support that risk modeling in heart failure improves outcomes, and barriers to implementation

• Machine learning may offer an alternative solution for a precision medicine approach to personalized risk prediction in heart failure patients, but has its own limitations

• The future of risk prediction in HF will likely involve more sophisticated risk modeling, incorporating diverse data sources not typically included in existing risk scores