A Multi-State Model to Predict Heart Failure Hospitalizations and All-Cause Mortality in Outpatients With Heart Failure With Reduced Ejection Fraction: Model Derivation and External Validation

Abstract

Background

Outpatients with heart failure (HF) who are at high risk for HF hospitalization and/or death may benefit from early identification. We sought to develop and externally validate a model to predict both HF hospitalization and mortality that accounts for the semi-competing nature of the two outcomes and captures the risk associated with the transition from the stable outpatient state to the post-HF hospitalization state.

Methods and Results

A multi-state model to predict HF hospitalization and all-cause mortality was derived using data (n=3834) from the Heart Failure Endpoint evaluation of Angiotensin II Antagonist Losartan (HEAAL) study, a multinational randomized trial in symptomatic patients with reduced left ventricular ejection fraction. Twelve easily and reliably obtainable demographic and clinical predictors were pre-specified for model inclusion. Model performance was assessed in the Sudden Cardiac Death in Heart Failure Trial (SCD-HeFT) cohort (n=2521). At one year, the probability of being alive without HF hospitalization was 94% for a typical patient in the lowest risk quintile and 77% for a typical patient in the highest risk quintile and this variability in risk continued through 7 years of follow up. The model c-index was 0.72 in the derivation cohort, 0.66 in the validation cohort and 0.69 in the implantable cardiac defibrillator arm of the validation cohort. There was excellent calibration across quintiles of predicted risk.

Conclusions

Our findings illustrate the advantages of a multi-state modeling approach - providing estimates of HF hospitalization and death in the same model, comparison of predictors for the different outcomes and demonstrating the different trajectories of patients based on baseline characteristics and intermediary events.

There are nearly 6 million adults living with heart failure (HF) in the United States with an estimated 670,000 new annual diagnoses¹. Due, in part, to aging of the American population, the prevalence of HF is expected to increase by upwards of 25% over the next 20 years¹. Although pharmacologic and device therapies have improved the survival and quality of life for many patients with HF, the median survival after a HF diagnosis remains only 3-5 years ^{2, 3}, and hospitalizations and re-hospitalizations are common^{4, 5}. Discriminating between high and low risk ambulatory patients with HF can improve care by preventing delays in appropriate treatment for high-risk patients including referral for consideration of advanced therapies. In addition, prediction models can help direct costly disease management services to high-risk patients, select patients for clinical trials, and risk-adjust patient populations for quality improvement efforts.

Currently available prediction models in ambulatory patients with HF either focus on mortality alone ^6-22, a composite of death or hospitalization^{11, 19} or HF hospitalization alone²³. Most ambulatory HF models have not been externally validated^{9, 11-22} or contain variables that are not routinely collected in clinical practice ^{6-8, 13, 14, 16-18, 20, 22}. Several models have been developed using cohorts of patients with end-stage HF either with New York Heart Association IIIb-IV symptoms⁷ or in patients referred for heart transplant evaluation⁶ and thus may not pertain to the great majority of patients who have less severe symptoms. In addition, many models were derived in patient cohorts enrolled more than 20 years ago when evidence based treatment for HF did not include routine beta-blockers, aldosterone antagonists or implantable cardiovertersdefibrillators (ICDs) ^{6, 7, 12, 13, 15, 16}. Some models developed from more contemporary patient cohorts undergoing ICD implantation were restricted to the Medicare population¹⁰.

Recently, statistical ²⁴ and analytic approaches^{24, 25} have made it feasible to construct multi-state prediction models that can simultaneously account for terminal and non-terminal events, so called “semi-competing risks”, such as HF hospitalization and death. Unlike other models of HF hospitalization ²³ that failed to account for the competing risk of death, and thus may be predisposed to biased estimates due to informative censoring, multistate models allow for unbiased estimates of each outcome separately or as a composite²⁵. Thus, we developed a multi-state model employing easily and reliably obtainable demographic and clinical variables for the purposes of predicting HF hospitalization and all-cause mortality in outpatients with symptomatic HF with reduced ejection fraction, based on a population receiving current evidence-based care. We then validated the model in a separate cohort of patients with heart failure with reduced ejection fraction.

Methods

Model Derivation Cohort

The prediction model was developed from the Heart Failure Endpoint evaluation of Angiotensin II Antagonist Losartan (HEAAL) study cohort ²⁶. HEAAL was an international, multicenter, randomized trial of low dose (50mg) versus high dose (150mg) losartan in stable outpatients with HF with reduced ejection fraction. The study methods and population characteristics have been described previously^{26, 27}. To be enrolled in the trial, participants had to have New York Heart Association (NYHA) Class II-IV HF with left ventricular ejection fraction (LVEF) ≤40%, be on stable cardiovascular therapy for at least 2 weeks at the time of enrollment and be intolerant to angiotensin-converting enzyme inhibitor therapy. The complete list of inclusion and exclusion criteria has been published previously²⁷. Participants were enrolled from 55 sites in 30 countries and 5 geographical regions including Western Europe, Eastern Europe, the Middle East and Africa, Asia and the Pacific Region, and Latin America between November 2001 and March 2005. Participants were randomized on a 1:1 basis to the receipt of either 50mg or 150mg of losartan and followed for a median of 4.7 years with the study ending on March, 2009. The study was approved by the institutional review board at each site and all patients provided written informed consent.

Model Validation Cohort

Prediction model performance was assessed using the Sudden Cardiac Death in Heart Failure Trial (SCD-HeFT) cohort. SCD-HeFT enrolled adults with NYHA Class II or III chronic, stable HF with LVEF ≤35%²⁸. Participants were enrolled from 125 sites in North America, Australia and New Zealand from September 1997 to July 2001 and were randomized 1:1:1 to receive a single-chamber ICD, amiodarone or placebo and followed for a median of 3.8 years. The study was approved by the institutional review board at each site and all patients provided written informed consent. Model performance was assessed in the entire SCD-HeFT cohort, the intention to treat ICD arm and the intention to treat medical therapy arm (placebo and amiodarone).

Outcomes

HF hospitalization and all-cause mortality were selected as the primary outcomes of interest for the prediction model. The HEAAL study captured all hospitalizations and deaths throughout the duration of follow up²⁶. All mortality and hospitalization endpoints were adjudicated by an Endpoint Classification Committee. The current analysis was restricted to the patient's first HF hospitalization after enrollment.

SCD-HeFT collected information on mortality and all hospitalizations throughout the duration of the study. Unlike the HEAAL study, HF hospitalization was not a pre-specified endpoint and thus the causes of hospitalizations were investigator reported and not independently adjudicated. For purposes of this analysis, the investigator reported cause of each hospitalization was used to determine the heart failure hospitalization outcome.

Predictors

Candidate predictor variables were selected a priori based upon 3 characteristics. First, the predictor variable must have been consistently associated with HF hospitalization or mortality in prior studies. Second, binary predictors could not be rare (<5%). Lastly, predictors were only selected if, by a consensus of the investigators, it was believed they could be reliably assessed with little inter-observer variability in routine clinical practice and are routinely collected in a stable heart failure population. The following variables, which were collected at the time of enrollment, were included in the multi-state model: age, gender, NYHA class (binary outcome III vs II), LVEF as assessed by echocardiogram (%), serum creatinine (mg/dl) and serum sodium (mEq/L), systolic blood pressure (SBP) (mmHg), weight (kg), history of diabetes (DM), ischemic heart disease (IHD), atrial fibrillation (AF) and prior stroke.

Sample Size

There were three total transitions in the model. We did not use any data-driven variable selection procedures and ensured that there were more than 20 events per degree of freedom per transition to achieve model parsimony and prevent overfitting.

Missing Data

Since there were few missing data in the model development cohort (<1%), a complete case analysis was used.

Multi-state model

Since HF hospitalization and death are semi-competing risks in that death precludes a subsequent HF hospitalization but death can still occur after a HF hospitalization, an illness-death, acyclic, multi-state model was used^{24, 25}. In this model, all participants are in the initial state of prevalent HF and are at risk of a HF hospitalization (transition 1) or death without a preceding HF hospitalization (transition 2). In addition, those who were hospitalized for HF are also at risk for death after a HF hospitalization (transition 3). To demonstrate model use, patients in the derivation cohort were grouped by quintile of predicted risk of transition 2 and the predicted probabilities of the patient with the median risk in each quintile were calculated and plotted. To assess model calibration, predicted probabilities for heart failure hospitalization and death were calculated for participants in the external validation cohort, divided into quintiles and compared to observed outcomes of heart failure hospitalization and death at 1,2,3,4 and 5 years of follow-up.

Statistical analysis

Cox proportional hazards regression was used to model the effect of covariates on the cause-specific hazards of the three state transitions with separate (stratified) non-parametric baseline hazards for transitions into the hospitalization state and into the death state.²⁴. For each of the 12 predictors, univariate multistate models were used to compare the model fit with transition-specific coefficients versus identical coefficients for each transition and decisions of whether to include transition-specific coefficients were made by likelihood-ratio testing and comparison of the Akaike information criterion (AIC). No data-driven variable selection procedures were used. Linearity assumptions were assessed for all continuous variables and appropriate transformations performed as necessary. The proportional hazards assumptions were evaluated using log(-log(Survival)) plots and Schoenfeld residuals. Discrimination was assessed by the c-index for time-to-event data with censoring. For both the derivation and validation cohorts, the c-index was calculated for the overall model as well as for hospitalization and death separately. Internal validation was performed by creating 200 bootstrap samples from the original dataset, re-estimating the model coefficients in each bootstrap sample and applying the bootstrapped model coefficients to the original dataset. The optimism of the c-index was calculated as the average difference between the c-index of the 200 bootstrap samples and that of the original sample; this value was subtracted from the c-index to correct for optimism. Calibration was assessed in the validation cohort by dividing the cohort into quintiles of predicted risk of transition 2 and comparing model predicted outcomes of heart failure hospitalization and death to observed outcomes at 1,2,3,4 and 5 years of follow-up. All analyses were performed using R version 3.0.2 and the mstate and survival packages for multistate modeling.

Results

Study Population

Model Development Cohort

Baseline characteristics of the model development and validation cohorts are shown in Table 1. There were a total of 3834 participants enrolled in the HEAAL trial between 2001 and 2005. All patients were treated with losartan and there was a high use of beta-blockers (72%), moderately high use of aldosterone antagonists (38%) and fairly low use of ICDs (5%) at the time of study enrollment (Table 1). Given the small number of New York Heart Association (NYHA) Class IV participants (0.5% of the total population), these participants were excluded as well as 2 participants with NYHA Class I HF and one patient with missing information on NYHA Class, leaving a total of 3809 participants available for model development (Figure 1). Of the 12 predictors of interest, there were few missing data (24 total participants missing data for one or more covariates, 0.6% of the population), thus a complete case analysis was used and the model derived in the remaining 3786 participants.

Table 1.

Baseline characteristics of the derivation and validation cohorts

	Derivation Cohort (HEAAL) n=3786	HEAAL missing data: no. (%)	Validation Cohort (SCD-HeFT) n=2521	SCD-HeFT missing data: no. (%)
Age (yrs): mean ±SD	64 ± 12	0 (0)	59 ± 12	0 (0)
Female: n(%)	1143 (30)	0 (0)	588 (23)	0 (0)
Race/Ethnicity:
White: n(%)	2321 (61)	0 (0)	1932 (77)	0 (0)
Asian: n(%)	856 (22)
Multi: n(%)	354 (9)
Hispanic: n(%)	212 (5)
Other: n(%)	103 (3)
Left ventricular ejection fraction (%): median (IQR)	33 (27-37)	1(0.02)	25 (20-30)	2(0.1)
New York Heart Association Class:
I: n(%)	2 (<0.01)	1 (0.02)		0 (0)
II: n(%)	2657 (69)		1761 (70)
III: n(%)	1152 (30)		760 (30)
IV: n(%)	22 (0.5)
Ischemic Heart Disease: n(%)	2456 (64)	0 (0)	1305 (52)	0 (0)
Diabetes: n(%)	1199 (31)	0 (0)	767 (30)	0 (0)
Atrial Fibrillation: n(%)	1070 (28)	0 (0)	390 (16)	0 (0)
Stroke: n(%)	307 (8)	0 (0)	166 (7)	0 (0)
Systolic Blood Pressure (mmHg): mean ±SD	126 ± 18	1 (0.02)	120 ± 20	9 (0.4)
Heart Rate (bpm): mean ±SD	73± 12	5 (0.1)	75± 14	9 (0.4)
Weight (kg): mean ±SD	76 ± 17	5 (0.1)	88 ± 20	0 (0)
Serum Sodium (mEq/L): mean ±SD	140 ± 4	16 (0.4)	139 ± 3	20 (0.8)
Creatinine (mg/dl): mean ±SD	1.2 ± 0.3	16 (0.4)	1.2 ± 0.7	21 (0.8)
ACEi/ARB: n (%)	3786 (100)	0 (0)	2432 (97)	0 (0)
Beta blocker: n (%)	2758 (72)	0 (0)	1738 (69)	0 (0)
Aldosterone antagonist: n (%)	1436 (38)	0 (0)	484 (19)	1 (<−0.1)
Diuretics: n(%)	3071 (80)	0 (0)	2064 (82)	0 (0)
Aspirin: n (%)	2138 (56)	0 (0)	1415 (56)	0 (0)
Statin: n (%)	1511 (39)	0 (0)	965 (38)	0 (0)
Calcium channel Blocker: n (%)	450 (12)	0 (0)	279 (11)	0 (0)
ICD at time of enrollment n(%)	172 (4.5)	0 (0)	0 (0)	0 (0)

Figure 1.

Flow diagram of model development cohort selection and illness-death multistate model transitions. Selection of model development cohort and the number of participants with each of the illness-death multistate model transitions. HEAAL, Heart Failure Endpoint evaluation of Angiotensin II Antagonist Losartan; NYHA, New York Heart Association; HF, heart failure

Validation Cohort

There were a total of 2521 participants enrolled in SCD-HeFT between 1997 and 2001. Patients were well treated with angiotensin-converting enzyme or angiotensin II-receptor blockers (97%) and beta-blocker therapy (69%) with fewer participants on aldosterone antagonist therapy (19%). In general, baseline characteristics of the two cohorts were similar (Table 1). As in the derivation cohort, participants with missing data for one or more covariates were excluded (39 participants missing data for one or more covariates, 1.5% of the population), leaving 2482 participants for model validation.

Model development and specification

Of the 3786 participants in the model development cohort, 944 were hospitalized for worsening HF at least once, 757 died without a HF hospitalization, and 528 died at any time after HF hospitalization (Figure 1). Of the 1274 deaths, 926 were adjudicated as CV deaths, 314 as HF deaths and 481 as sudden deaths. Transition-specific covariates were associated with lower model AIC and better model fit by likelihood ratio testing for all predictors except serum sodium. The proportional hazards assumption was met for all 12 predictors. For all 6 continuous variables a linear association appeared reasonable based on Locally Weighted Scatterplot Smoothing curves and was associated with better model fit than piecewise linear or other transformations in univariate multistate analysis. The multivariate effects of the 12 predictors for the three transitions are shown in Table 2. NYHA Class, sodium, SBP and weight had similar effects on all three transitions in multivariate analysis with higher NYHA Class increasing the risk and higher sodium, SBP and weight decreasing the risk for all three transitions. For the majority of predictors, there was a similar effect of the predictor on the risk of transition 1 (HF hospitalization) and transition 2 (Death) with an attenuated effect of the predictor on the risk of transition 3 (Death after HF hospitalization). Lower LVEF values significantly increased the risk of transitions 1 and 2 with a similar yet attenuated and non-significant trend for transition 3. Similar findings were seen with age, male gender, diabetes, AF and prior stroke. Ischemic etiology increased the risk of death (transitions 2 and 3) with no effect on HF hospitalization risk (transition 1).

Table 2.

Transitional hazard ratios for multi-state prediction model:

Predictor	T1: Prevalent HF to HF hospitalization		T2: Prevalent HF to Death		T3: HF hospitalization to Death
	HR	95% CI	HR	95% CI	HR	95% CI
Age per 10 year increase	1.09	1.02-1.16	1.22	1.13-1.32	1.09	1.00-1.19
Female gender	0.81	0.69-0.96	0.69	0.57-0.83	0.93	0.75-1.17
NYHA III vs II	1.62	1.41-1.85	1.41	1.21-1.64	1.38	1.16-1.65
LVEF per 10% increase	0.73	0.66-0.80	0.76	0.68-0.85	0.88	0.77-1.01
Creatinine per 1mg/dl increase	2.22	1.83-2.68	1.62	1.30-2.01	1.78	1.39-2.29
Sodium (per 10mEQ/L increase)	0.77	0.68-0.88	0.77	0.68-0.88	0.77	0.68-0.88
Systolic Blood Pressure per 10mmHg increase	0.91	0.87-0.94	0.98	0.94-1.02	0.94	0.89-0.99
Weight per 10kg increase	0.92	0.88-0.96	0.86	0.82-0.91	0.84	0.79-0.90
Diabetes mellitus	1.50	1.30-1.72	1.47	1.26-1.71	1.13	0.94-1.36
Ischemic etiology	0.90	0.78-1.03	1.15	0.97-1.35	1.13	0.93-1.37
Atrial fibrillation	1.21	1.05-1.40	1.28	1.09-1.50	0.89	0.73-1.08
Prior stroke	1.28	1.03-1.60	1.52	1.22-1.89	1.16	0.87-1.53

HR indicates hazard ratio, CI confidence interval, HF heart failure, NYHA New York Heart Association, LVEF left ventricular ejection fraction

Risk prediction

The baseline characteristics of the patient with median risk from each quintile of predicted risk for transition two are shown in Table 3 with the predicted probability of being in each outcome state over 7 years of follow up shown in Figure 2. In addition, the predicted probability of being in each of the four possible states (alive without HF hospitalization, alive having experienced a HF hospitalization, death having experienced a HF hospitalization, death without HF hospitalization) are listed in Table 3 for the five patients at 1, 3 and 6 years of follow up. As can be seen, the probability of transitioning to these different clinical states varied tremendously in these typical patients. At one year of follow up, patient A (the patient with the median risk from the lowest risk quintile) has a 2.1% predicted probability of death with or without a preceding HF hospitalization and 3.5% probability of HF hospitalization without subsequent death. The same predicted probabilities are 10.4% and 12.5%, respectively for patient E in the highest risk quintile. After six years of follow up, patient A has a 16.8% predicted probability of death either with or without a preceding HF hospitalization and a 10.7% probability HF hospitalization without subsequent death. The same probabilities for patient E at six years are 61.1% and 15.5%, respectively. In addition, composite outcomes can be obtained from the predicted probabilities directly. For patient A, the predicted probability of being alive with or without a HF hospitalization (state 1 or state 2) at 1 year is 98% and the risk of HF hospitalization or death (states 2 and 4) is 4.5% at 1 year. For patient E the predicted probability of being alive with or without HF hospitalization (state 1 or 2) is 89.5% and HF hospitalization or death (state 2 or 4) is 22.1% at 1 year.

Table 3.

Baseline characteristics and predicted outcomes of the patient with median risk from each quintile of predicted risk of transition two

		Patient A (Q1)	Patient B (Q2)	Patient C (Q3)	Patient D (Q4)	Patient E (Q5)
Age (yrs)		66	64	69	65	75
Gender (M/F)		M	M	F	M	M
NYHA Class (II or III)		II	II	III	II	III
LVEF (%)		38	35	30	29	31
Creatinine (mg/dl)		1.6	2.0	0.7	1.2	1.1
Serum sodium (mEq/L)		141	135	141	140	135
Systolic blood pressure (mmHg)		152	160	122	140	115
Weight (kg)		110	115	84.3	78	105
Diabetes (Y/N)		−	−	+	+	+
Ischemic etiology (Y/N)		−	+	+	+	+
Atrial fibrillation (Y/N)		−	−	−	−	+
Prior stroke (Y/N)		−	−	−	−	−
1 yr probabilities of being in a given outcome state	Alive without HFH	94.3%	91.9%	89.0%	87.1%	77.1%
	HFH, still alive	3.6%	4.6%	6.1%	6.3%	12.5%
	Death without HFH	1.9%	3.0%	4.1%	5.7%	8.5%
	HFH followed by death	0.2%	0.5%	0.8%	0.9%	1.9%
3 yr probabilities of being in a given outcome state	Alive without HFH	84.9%	78.8%	72.0%	67.3%	48.0%
	HFH, still alive	7.8%	9.3%	11.1%	11.1%	19.4%
	Death without HFH	5.7%	8.9%	11.7%	15.9%	11.0%
	HFH followed by death	1.6%	3.0%	5.2%	5.7%	21.6%
6 yr probabilities of being in a given outcome state	Alive without HFH	72.5%	62.5%	52.3%	45.4%	23.4%
	HFH, still alive	10.7%	11.1%	11.5%	10.6%	15.5%
	Death without HFH	11.4%	16.8%	21.3%	28.1%	33.7%
	HFH followed by death	5.4%	9.6%	14.9%	15.9%	27.4%

Q1 refers to quintile 1 (lowest risk), Q2 quintile 2, Q3 quintile 3, Q4 quintile 4, Q5 quintile 5 (highest risk), M male, F female, NYHA New York Heart Association, LVEF left ventricular ejection fraction, HFH heart failure hospitalization, (+) refers to present and (−) refers to absent at baseline

Figure 2.

Prediction probabilities for the patient with the median risk from each quintile of risk for transition 2. The development cohort was divided into quintile of predicted risk for transition 2 (from prevalent heart failure to death). The median patient from each quintile was selected and the predicted probabilities of being alive without heart failure hospitalization, alive after heart failure hospitalization, dead after heart failure hospitalization and dead without heart failure hospitalization are shown. Patient A is the median risk patient in quintile 1 (lowest risk), patient B quintile 2, patient C quintile 3, patient D quintile 4 and patient E quintile 5 (highest risk).

Model performance in Development Cohort

The c-index was 0.72 for the entire model. Discrimination was better for the transition states into death (c-index = 0.75) than for HF hospitalization (c-index = 0.68). Dividing the cohort into quintiles of predicted risk and plotting the cumulative incidence of each state revealed a separation of risk of HF hospitalization and death (Figure 3). Patients in the lowest risk quintile had a 1-, 2- and 5-year risks of death of 0.7%, 1.6% and 4.1% respectively, while patients in the highest risk quintile had a 1-,2- and 5 year risks of dying of 10.2%, 21.6% and 46%.

Figure 3.

Observed cumulative incidence of being in each health state by quintiles of predicted risk in the derivation cohort. Quintile 1 (Q1) refers to the lowest risk quintile and quintile 5 (Q5) refers to the highest risk quintile. HFH refers to heart failure hospitalization. Patients were divided into quintile of predicted risk of transition 2 and the cumulative incidence of being in each health state was plotted.

Internal validation and optimism-corrected performance

The optimism-corrected c-index for the entire model from re-estimating the model parameters on 200 bootstrap samples and reapplying these to the original dataset was 0.72

Model performance in External Validation Cohort

In the entire SCD-HeFT validation cohort (n=2482), the c-index was 0.66 for the overall model. As in the derivation cohort, discrimination was better for the transitions into death (c-index = 0.71) than for HF hospitalization (c-index = 0.62). In the subset of the cohort randomized to ICD (n=829), the c-index was 0.69 for the entire model, 0.77 for death and 0.63 for HF hospitalization. In the subset of the cohort randomized to medical therapy (n=1692), the c-index was 0.63 for the entire model, 0.68 for death and 0.62 for HF hospitalization. Calibration by quintiles of predicted risk of transition 2 in the non-ICD medical therapy arms of the validation cohort (Figure 4), the ICD arm only (Figure 5) and the entire validation cohort (Supplemental Figure 1) showed overall excellent calibration. For example, in quintile 1 of the medical therapy arm the 1, 2 and 5 year predicted risks of death were 2.3%, 5.3% and 14.4% and observed were 3.6%, 6.9% and 16.1%, respectively. The same predicted risks in quintile 5 of the medical therapy arm were 14.1%, 30.1% and 63.7% and observed were 14%, 30.4% and 60.0% at 1,2 and 5 years respectively. Calibration plots of the “death without HF hospitalization” and “death after HF hospitalization” states combined into one “death” state are shown in Supplemental Figures 2 and 3.

Figure 4.

Quintiles of predicted probability and observed outcomes in the non-ICD medical therapy arms of the SCD-HeFT external validation cohort at 1, 2 and 5 years of follow up. Patients were divided into quintile of predicted risk of transition 2 . Quintile 1 (Q1) refers to the lowest risk quartile and quintile 5 (Q5) refers to the highest risk quartile. HFH refers to heart failure hospitalization.

Figure 5.

Quintiles of predicted probability and observed outcomes in the ICD arm of the SCDHeFT external validation cohort at 1, 2 and 5 years of follow up. Patients were divided into quintile of predicted risk of transition 2. Quintile 1 (Q1) refers to the lowest risk quartile and quintile 5 (Q5) refers to the highest risk quartile. HFH refers to heart failure hospitalization.

Updated model with the addition of ICD effect on mortality and implementation tool

The addition of a coefficient (β = −0.26, HR=0.77) for the ICD effect on mortality to the prediction model improved the calibration for mortality in the ICD arm of the Validation Cohort (Supplemental Figure 4) and thus was added to the model as a 13^th covariate. We have created an online calculator that can be found at: tuftsmedicalcenter.org/HFcalc

Discussion

We describe the first multistate model for risk prediction in ambulatory patients with HF, which we have called the Tufts MC HF Risk Model. Using routinely collected demographic, clinical, and laboratory information, we have developed a model that can predict HF hospitalization, all-cause death or the composite of HF hospitalization or death over 7 years of follow-up. We found wide variation in risk with the typical patient in the lowest risk quintile having an excellent 1-year prognosis, while the typical patient in the highest risk quintile had much higher event rates at 1 year. This discrimination between low and high-risk patients continued throughout the 7-year duration of follow up of this cohort. Model discrimination was slightly worse in the validation cohort (c-index=0.66) as compared with the derivation cohort (c-index=0.72) but still in a range of providing clinically useful separation in risk. Since there were few ICDs within the HEAAL population, it was reassuring that discrimination was slightly better in the SCD-HeFT ICD cohort (c-index=0.68). Calibration of mortality was excellent in the validation cohort, particularly in the medical therapy arm. The addition of a coefficient for ICD therapy improved model calibration in the ICD arm, correcting for the near-absence of this evidence-based therapy in the derivation cohort. Several studies have documented that less than 50% of eligible patients receive a primary prevention ICD^29-31. The Tufts MC HF Risk Model includes a variable for ICD use, derived by recalibrating the model using the SCD-HeFT cohort, and thus can be used in patients with or without an ICD, although the performance of the updated model with a variable for ICD use requires additional external validation. Finally, while the model appeared to underestimate HF hospitalizations in the validation cohort, the derivation data was based on adjudicated HF hospitalizations whereas our process of interpreting investigator reported causes of hospitalization in the validation cohort may have overestimated the prevalence of this outcome.

Another advantage of using a multistate modeling approach is the ability to compare the effect of predictor variables on each of the individual component outcomes. This study demonstrates the similar effect of many predictors on the outcomes of HF hospitalization and all-cause mortality with an attenuated effect of these predictors on subsequent risk of death after HF hospitalization. Variables were pre-specified for model inclusion because of previous associations with mortality; however, the associations of these variables with HF hospitalization are less well described. We showed that other than ischemic etiology, all the variables that are associated with an increased risk of death are also associated with an increased risk of HF hospitalization. The attenuated effect of known predictors of death in HF on transition 3 may be related to index event bias, whereby conditioning on a first event dilutes the effect of shared risk factors on related subsequent events—since all patients who experience hospitalization are by definition at higher risk ³². Effect modification or bias related to the fact that all predictors were measured at baseline may also contribute to the attenuated effect of some predictors on the risk of death after HF hospitalization. Notably, this effect is not apparent for NYHA Class, sodium, SBP and weight, in which similar effects were seen for all three transitions.

While there are several published models to predict HF readmission in patients admitted with acute HF, there is only one published model to predict HF hospitalization in ambulatory patients with HF and reduced ejection fraction²³, and no model includes the competing risk of death thus predisposing to biased risk estimates due to informative censoring²⁵. In contrast, multistate models such as the one described here account for transition states, such as HF hospitalization, and absorbing/terminal states, such as death, and allow for valid predictions in the presence of semi-competing risks²⁵. This report demonstrates the utility and feasibility of using multistate models to improve prediction of semi-competing events and to gain additional insight into factors that modify risk of outcomes in patients with HF.

Meta-analyses^33-35 have identified 15-20 published models for the prediction of mortality in ambulatory patients with HF, 4 of which have been externally validated^{6-8, 10}. Some of these models have limited applicability because they include variables that are not routinely collected in the general HF population such as cardiopulmonary exercise stress testing^{6, 14, 16, 22}, six minute walk testing⁸, magnetic resonance imaging¹⁷ or research based laboratory tests such as cytokine levels^{20, 22}. Several models ^{7, 11} have more than 20 variables, which may be burdensome to calculate in clinical practice. In addition, some models include variables without precise definitions or with considerable inter-observer variability in routine clinical practice such as physical exam findings^{11, 12} or the appearance of left ventricular hypertrophy on electrocardiogram¹³. Many models^{6, 7, 12, 13, 15, 16} were developed in cohorts that were not treated with current evidence-based therapy such as beta-blockers or aldosterone antagonists, which may lead to poor performance in current practice. In contrast, our model was developed in a cohort of patients with high rates of beta-blocker and aldosterone antagonist therapy and only used variables that are routinely collected and reliably assessed in clinical practice. The discrimination of our overall model (c-index 0.72) and for the mortality endpoint (c-index 0.75) are similar to those reported for other mortality models on external validation: the Heart Failure Survival Score (c-index 0.74)⁶, the Seattle Heart Failure Model (c-index 0.73)⁷ and the “SHOCKED” model (c-index 0.75)¹⁰. These models all contain variables not collected in the SCD-HeFT validation cohort; therefore we were not able to directly compare model performance in the same dataset.

We anticipate that our model will be useful to identify patients who are at high risk for a HF hospitalization, who may benefit from more frequent clinic visits, disease management services or aggressive titration of neurohormonal therapy. In addition, patients with an elevated predicted risk for mortality over the next one to two years may benefit from referral to a transplant center in addition to the above interventions. This model can also be used to select patients for inclusion or explore heterogeneity of treatment effect in clinical trials. Finally, this model can be used for risk adjustment for hospitals or providers to identify areas for quality improvement.

Study Limitations

This model was developed in a population with mild to moderate HF symptoms (NYHA Class II-III) and should not be applied to NYHA Class I or IV symptoms or those with a reduced ejection fraction without HF or patients with HF with preserved LVEF. The derivation cohort was enrolled more than 10 years ago, before aldosterone antagonists were routinely used for patients with NYHA Class II symptoms and prior to the use of neprilysin inhibitors or ivabradine. Evaluation of discrimination and calibration on a more contemporary cohort, with model updating if appropriate, would enhance the usefulness of this model. While we recognize this as a limitation of our study, we also recognize that other externally validated models for the prediction of mortality were derived from cohorts enrolled in the same time frame¹⁰ or even older^{6, 7} and thus have the same limitations. While our 12 variable model was externally validated in the SCD-HeFT validation cohort, the ICD recalibration effect was derived from the SCD-HeFT population directly and thus the performance of the updated model with a variable for ICD use requires additional external validation. Because of the complexities of modeling recurrent events, our model only includes the first HFH per patient. Finally, some potentially important predictor variables were not collected such as natriuretic peptide levels, baseline diuretic dose and prior HF hospitalization data, which if available may have improved model performance.

In conclusion, we demonstrate the utility of a multi-state modeling approach to predict clinically meaningful semi-competing risks, such as HF hospitalization and death. Multi-state models, such as the one we developed, are not subject to informative censoring bias, allow for predictions of related outcomes individually and as a composite and can be useful to explore the effect of predictors on the different outcomes of interest.

Clinical Perspective.

The rapidly expanding population of patients with heart failure (HF) is continually at risk for hospitalization and death. Predicting these events is increasingly important in individualizing treatments from disease management to advanced therapies to palliative care. Statistical models that do not account for the competing risk of death can lead to biased prediction of hospitalization. Existing conventional models for predicting hospitalization are limited to the short term, with variables collected during HF admission. We describe the derivation and external validation of a multi-state model for long-term prediction of both HF hospitalization and death in a cohort of patients with symptomatic HF and reduced left ventricular ejection fraction. We observe substantial heterogeneity in risk for both hospitalization and death, across the population. This model contains 13 readily accessible variables, and we have made it openly available, as an on-line risk calculator. This approach offers improved guidance to clinicians and patients in selecting from among the various available and emerging complex HF management strategies.