Key Points
Question
Can the Prospective Comparison of ARNI With ACEI to Determine Impact on Global Mortality and Morbidity in Heart Failure trial (PARADIGM-HF) trial predict outcome in patients with heart failure (HF)?
Findings
In this prognostic study, there were 1546 deaths and 2031 cardiovascular death or HF hospitalization events of the 8399 patients in the PARADIGM-HF trial, and models were derived for a composite of cardiovascular death or HF hospitalization, cardiovascular death, and all-cause death. The models were externally validated using data from the Aliskiren Trial to Minimize Outcomes in Patients with Heart Failure and the Swedish Heart Failure registry.
Meaning
The PREDICT-HF models can be used to accurately predict outcome of patients with chronic HF.
Abstract
Importance
Accurate prediction of risk of death or hospitalizations in patients with heart failure (HF) may allow physicians to explore how more accurate decisions regarding appropriateness and timing of disease-modifying treatments, advanced therapies, or the need for end-of-life care can be made.
Objective
To develop and validate a prognostic model for patients with HF.
Design, Setting, and Participants
Multivariable analyses were performed in a stepwise fashion. Harrell C statistic was used to assess the discriminative ability. The derivation cohort was Prospective Comparison of ARNI With ACEI to Determine Impact on Global Mortality and Morbidity in Heart Failure trial (PARADIGM-HF) participants. The models were validated using the Aliskiren Trial to Minimize Outcomes in Patients with Heart Failure Trial (ATMOSPHERE) study and in the Swedish Heart Failure Registry (SwedeHF). A total of 8399 participants enrolled in PARADIGM-HF. Data were analyzed between June 2016 and June 2018.
Main Outcomes and Measures
Cardiovascular death, all-cause mortality, and the composite of cardiovascular death or HF hospitalization at both 1 and 2 years.
Results
Complete baseline clinical data were available for 8011 patients in PARADIGM-HF. The mean (SD) age of participants was 64 (11.4) years, 78.2% were men (n = 6567 of 8399), and 70.6% were New York Heart Association class II (n = 5919 of 8399). During a mean follow-up of 27 months, 1546 patients died, and 2031 had a cardiovascular death or HF hospitalization. The common variables were: male sex, race/ethnicity (black or Asian), region (Central Europe or Latin America), HF duration of more than 5 years, New York Heart Association class III/ IV, left ventricular ejection fraction, diabetes mellitus, β-blocker use at baseline, and allocation to sacubitril/valsartan. Ranked by χ2, N-terminal pro brain natriuretic peptide was the single most powerful independent predictor of each outcome. The C statistic at 1 and 2 years was 0.74 (95% CI, 0.71-0.76) and 0.71 (95% CI, 0.70-0.73) for the primary composite end point, 0.73 (95% CI, 0.71-0.75) and 0.71 (95% CI, 0.69-0.73) for cardiovascular death, and 0.71 (95% CI, 0.69-0.74) and 0.70 (95% CI, 0.67-0.74) for all-cause death, respectively. When validated in ATMOSPHERE, the C statistic at 1 and 2 years was 0.71 (95% CI, 0.69-0.72) and 0.70 (95% CI, 0.68-0.71) for the primary composite end point, 0.71 (95% CI, 0.69-0.74) and 0.70 (95% CI, 0.69-0.72) for cardiovascular death, and 0.71 (95% CI, 0.69-0.74) and 0.70 (95% CI, 0.68-0.72) for all-cause death, respectively. An online calculator was created to allow calculation of an individual’s risk (http://www.predict-hf.com).
Conclusions and Relevance
Predictive models performed well and were developed and externally validated in large cohorts of geographically representative patients, comprehensively characterized with clinical and laboratory data including natriuretic peptides, who were receiving contemporary evidence-based treatment.
This study developed and validated a prognostic prediction model for patients with heart failure.
Introduction
Rates of morbidity and mortality are high in patients with heart failure (HF), but predicting prognosis is difficult.1,2,3,4 Predicting risk of death or hospitalizations in patients with HF can allow physicians and patients to make important decisions regarding appropriateness and timing of treatments or the need for end-of-life care.1,2,3,4 Risk estimation can be used to alter care, with closer follow-up and intensified therapy. It may also aid physicians in assisting patients to make fully informed decisions about their care. Furthermore, identifying factors common to patients at high risk of death or hospitalization may reveal potential targets for interventions to improve prognosis.1,2,3,4
Individual measures, such as left ventricular ejection fraction (LVEF) or New York Heart Association (NYHA) classification, are associated with morbidity and mortality but are not sensitive enough to be used to predict prognosis. For example, in the Prospective Comparison of ARNI With ACEI to Determine Impact on Global Mortality and Morbidity in Heart Failure trial (PARADIGM-HF), although 70% of patients were deemed to have mild symptoms (NYHA class II), when stratified into quintiles using a predictive risk score, the rate of cardiovascular (CV) death or HF hospitalization ranged from 7.6 to 20.6 per 100 patient-years of follow-up.5 Using multiple variables and calculating risk by regression models more accurately predicts risk.1,2,3,4
Several multivariable models exist for predicting adverse outcomes in patients with HF.1,2,3,4 The Seattle Heart Failure Model (SHFM) remains the most extensively validated of these6 but was derived from (and validated in) a historical cohort not receiving contemporary evidence based guideline-recommended therapies for HF. Furthermore, the SHFM (and most other models) did not include biomarkers in the derivation model and, in particular, did not take into account levels of natriuretic peptides, which are strongly associated with prognosis in HF.1,2,3,4
We used patient-level data (including natriuretic peptides) from the PARADIGM-HF trial to develop prognostic models for morbidity and mortality in contemporary patients with HF treated with evidence-based therapies for HF.7,8 The models were validated using data from the Aliskiren Trial to Minimize Outcomes in Patients with Heart Failure Trial (ATMOSPHERE) study and in the Swedish Heart Failure Registry (SwedeHF) containing data on an unselected nationwide cohort.9,10,11
Methods
The PARADIGM-HF patient cohort (n = 8399) was used for model derivation. The ATMOSPHERE (n = 7016) and SwedeHF cohorts were used for model validation.
Design of Trials
The study design and main results of both PARADIGM-HF and ATMOSPHERE are published.7,8,9,10 The inclusion and exclusion criteria of the 2 trials were almost identical and are described in eAppendix 1 in the Supplement. Briefly, patients were eligible at screening if they had NYHA class II through IV, LVEF of 35% or less (changed from ≤40% initially in PARADIGM-HF by amendment), had an elevated natriuretic peptide level (see eAppendix 1 in the Supplement for levels) and were receiving an angiotensin-converting enzyme (ACE) inhibitor or angiotensin receptor blocker (ARB), along with a β-blocker (unless contraindicated or not tolerated) and a mineralocorticoid receptor antagonist (MRA), if indicated. The trial was approved by ethics committees at all participating centers in PARADIGM-HF and ATMOSPHERE, and all patients provided written informed consent. In PARADIGM-HF, patients were randomized to sacubitril/valsartan or enalapril and in ATMOSPHERE to enalapril, aliskiren, or both following a run in period in both trials (eAppendix 1 in the Supplement).
The primary outcome in each trial was the composite of death from CV causes or a first hospitalization for HF. The median duration of follow-up was 27 months in PARADIGM-HF and 36.6 months in ATMOSPHERE.
Swedish Heart Failure Registry
Patients with HF included in SwedeHF between July 2005 and December 2012 were studied.11 This nationwide internet-based voluntary registry has been described in detail.11 Patients are included in the registry based on a physician-based clinical diagnosis of HF and registered at discharge from hospital or following an outpatient visit. For the purposes of externally validating the PARADIGM Risk of Events and Death in the Contemporary Treatment of Heart Failure (PREDICT-HF) model for all-cause mortality, only patients registered following an outpatient encounter (rather than at hospital discharge) were included, and those with an LVEF greater than or equal to 50% were excluded. The LVEF is reported in the registry as less than 30%, 30% to 39%, and 40% to 49%, and each group were considered to have an LVEF of 25%, 35%, and 40%, respectively. Race/ethnicity was not available in SwedeHF, so all patients were classified as white. Heart failure duration was coded in SwedeHF as less than or greater than 6 months; this was treated as a patient with HF duration of greater than 1 year for the purposes of model validation (PARADIGM-HF categories for duration of HF: 1-2 years and >2 years). The remaining 10 baseline covariates in the PREDICT-HF model were available in SwedeHF. Serum potassium, hemoglobin, and N-terminal pro brain natriuretic peptide (NTproBNP) levels were available in SwedeHF. For the remaining laboratory values, the mean or median values from the PARADIGM-HF cohort were used because this is the default procedure for the online calculator. The variables included albumin, uric acid, percentage monocytes, total bilirubin, absolute neutrophils, chloride, low-density lipoprotein cholesterol, aspartate aminotransferase, urea, and triglycerides. Only patients with complete data across the covariates for the model for all-cause mortality were included in the primary analyses. Establishment of SwedeHF and this analysis assessing prognostic scores were approved by a multisite ethics committee. In SwedeHF, individual patient consent was not required, but patients are informed of registration in national registries and allowed to opt out.
Statistical Analysis
The main analysis includes only patients who had complete baseline data. The outcomes studied were the primary composite end point, CV death, and all-cause mortality.
The multivariable analyses were performed in a stepwise fashion. In step one, 63 baseline candidate variables potentially associated with outcomes were examined. Continuous variables were inspected using restricted cubic splines for nonlinear effects. Left ventricular ejection fraction and heart rate were included as continuous variables. Age, systolic blood pressure, and body mass index were included with a cutoff value, at which point the effect of the variable had negligible effect on the outcome. A backward stepwise Cox proportional hazards model was built for each outcome, with a 2-sided P value less than .01 as the initial significance level and selected variables were then taken forward to step 2.
In step 2, all initial candidate variables were evaluated for significant interactions for each outcome. For the interaction terms, a 2-sided P value of less than .001 was considered statistically significant. Significant interaction terms were added and selected again using a backward stepwise procedure, and the final model taken forward to step 3.
In step 3 laboratory test results (excluding natriuretic peptides) were added. Laboratory test results were evaluated by visual inspection of restricted cubic splines to identify nonlinearity. Cutoff values were selected for each variable, at which point the value did not alter the risk of the outcome. Laboratory test results that were statistically significant in the model were added and taken forward to step 4.
In step 4, the association between natriuretic peptides and outcome was considered in 5 categories (1, NTproBNP <400 pg/mL; 2, 400 pg/mL-799 pg/mL; 3, 800 pg/mL-1599 pg/mL; 4, 1600 pg/mL-3199 pg/mL; 5, ≥3200 pg/mL). The NTproBNP and BNP measurements were available in 8385 and 8340 patients, respectively.
The Cox model was used to obtain baseline survival at 1 and 2 years. The predicted vs actual outcome at 1 and 2 years was compared in quintiles. Kaplan-Meier survival curves for each outcome by quintile of risk were estimated and compared directly with baseline survival function for each outcome using Cox proportional hazards method.
The C statistic was used to assess the discriminative ability of the model, and predicted scores from the PARADIGM-HF cohort were applied to the ATMOSPHERE cohort to estimate survival at 1 and 2 years. Predicted vs observed outcomes in ATMOPSHERE at 1 and 2 years were examined in quintiles and deciles (shown in eAppendix 8C and D in the Supplement). Sensitivity analyses were also carried out for missing data using the multiple imputation technique chained method for missing data in SwedeHF. All analyses were conducted using Stata, version 14.0 (StataCorp) and SAS, version 9.4 (SAS Institute Inc).
Results
Complete baseline data were available for 8011 of 8399 patients in PARADIGM-HF. During a mean follow-up of 27 months, 1546 patients died and 2031 had a CV death or HF hospitalization.
Baseline characteristics of candidate variables for the prognostic models are shown in eAppendix 2 in the Supplement according to outcome (no outcome of interest, primary composite end point, CV death, or all-cause death). Patients experiencing an adverse outcome were older, more likely to be male, had higher resting heart rates, evidence of more advanced HF, more comorbidity, worse renal function, and higher natriuretic peptide levels than those not experiencing one of these outcomes.
Predictive Models
Primary Composite End Point
A total of 7683 patients had complete data for all variables in the final multivariable model, and a total of 1853 experienced the primary composite end point of CV death or hospitalization for HF. Twenty-three characteristics were found to be independently associated with the primary composite end point on multivariate analysis (eAppendix 3A in the Supplement). The final prognostic model for the primary composite end point, based on backward stepwise Cox proportional hazard regression, is shown in Table 1.
Table 1. Final Model for Primary Composite End Point Based on Backward Stepwise Cox Proportional Hazard Regressiona.
| Characteristic | HR (95% CI) | χ2 | Coefficient (SE) | P Value |
|---|---|---|---|---|
| Baseline characteristic | ||||
| Diabetes mellitus | 1.36 (1.24-1.50) | 39.6 | 0.307 (0.049) | <.001 |
| HF duration >5 y | 1.50 (1.31-1.70) | 36.5 | 0.402 (0.067) | <.001 |
| Prior HF hospitalization | 1.35 (1.21-1.51) | 27.3 | 0.302 (0.058) | <.001 |
| HF duration 1-5 y | 1.37 (1.21-1.54) | 25.2 | 0.313 (0.062) | <.001 |
| Race/ethnicity | ||||
| Black | 1.67 (1.36-2.04) | 24.5 | 0.511 (0.103) | <.001 |
| Asian | 1.43 (1.23-1.66) | 21.3 | 0.357 (0.077) | <.001 |
| Not prescribed sacubitril/valsartan | 1.24 (1.13-1.36) | 20.9 | 0.214 (0.047) | <.001 |
| Region (central Europe) | 1.34 (1.18-1.52) | 19.9 | 0.291 (0.065) | <.001 |
| Ejection fraction (per 5% decrease <40%) | 1.09 (1.05-1.13) | 19.5 | 0.083 (0.019) | <.001 |
| NYHA III/IV | 1.24 (1.11-1.38) | 15.5 | 0.214 (0.054) | <.001 |
| Region (Latin America) | 1.50 (1.21-1.85) | 14.2 | 0.403 (0.110) | <.001 |
| Valvular heart disease | 1.33 (1.14-1.55) | 13.2 | 0.286 (0.079) | <.001 |
| Not prescribed β-blocker | 1.34 (1.14-1.58) | 12.7 | 0.295 (0.083) | <.001 |
| Peripheral arterial disease | 1.28 (1.11-1.47) | 11.4 | 0.245 (0.073) | .001 |
| Male | 1.22 (1.08-1.38) | 9.9 | 0.200 (0.064) | .002 |
| Bundle branch block | 1.16 (1.05-1.28) | 8.6 | 0.147 (0.051) | .004 |
| Prior MI | 1.12 (1.02-1.23) | 5.3 | 0.113 (0.049) | .02 |
| Interaction terms | ||||
| Prior hospitalization for HF from Latin America | 1.41 (0.78-2.50) | 7.8 | 0.344 (0.132) | .005 |
| Laboratory values | ||||
| Total bilirubin (for every 0.29 mg/dL increase greater than 0.58 mg/dL) | 1.11 (1.08-1.15) | 42.0 | 0.109 (0.017) | <.001 |
| Uric acid (for every 0.84 mg/dL increase greater than 6.72 mg/dL) | 1.08 (1.05-1.11) | 26.7 | 0.075 (0.014) | <.001 |
| Albumin (for every 0.1 g/dL decrease less than 4.2 g/dL) | 1.05 (1.03-1.07) | 17.5 | 0.048 (0.012) | <.001 |
| Potassium (for every 0.1 mEq/L decrease less than 4 mEq/L) | 1.07 (1.03-1.10) | 13.7 | 0.064 (0.017) | <.001 |
| Absolute neutrophils (for every 1000/mL increase below 6000/mL) | 1.07 (1.03-1.11) | 10.6 | 0.065 (0.020) | .001 |
| Hemoglobin (for every 1 g/dL decrease less than 14 g/dL) | 1.08 (1.03-1.13) | 10.5 | 0.076 (0.024) | .001 |
| LDL (for every 38.61 mg/dL increase greater than 115.83 mg/dL) | 1.15 (1.04-1.26) | 8.0 | 0.135 (0.048) | .005 |
| Urea (for every 2.8 mg/dL increase greater than 14.01 mg/dL) | 1.02 (1.00-1.04) | 5.9 | 0.021 (0.009) | .02 |
| Absolute lymphocytes (for every 1000/mL decrease less than 2500/mL) | 1.06 (1.02-1.09) | 5.2 | 0.054 (0.048) | .02 |
| Natriuretic peptides | ||||
| NTproBNP categoryb | 1.34 (1.28-1.40) | 149.1 | 0.292 (0.0240) | <.001 |
Abbreviations: BP, blood pressure; HF, heart failure; HR, hazard ratio; LDL, low-density lipoprotein; MI, myocardial infarction; NYHA, New York Heart Association; NTproBNP, N-terminal pro brain natriuretic peptide.
SI conversion factor: To convert albumin to grams per liter, multiply by 10; bilirubin to micromoles per liter, multiply by 17.104; hemoglobin to grams per liter, multiply by 10; LDL cholesterol to millimoles per liter, multiply by 0.0259; lymphocytes to ×109 per liter, multiply by 0.001; neutrophils to ×109 per liter, multiply by 0.001; potassium to millimoles per liter, multiply by 1; urea to millimoles per liter, multiply by 0.375; uric acid to micromoles per liter, multiply by 59.485.
There was a significant interaction between region, specifically Latin America and a prior history of heart failure hospitalization. Specifically, those patients with a history of HF hospitalization and from Latin America were 1.41 times more likely to experience the primary composite outcome than those from outside Latin America without a history of HF hospitalization.
NTproBNP categories: 1, <400 pg/mL; 2, 400 pg/mL-799 pg/mL; 3, 800 pg/mL-1599 pg/mL; 4, 1600 pg/mL-3199 pg/mL; 5, ≥3200 pg/mL.
Cardiovascular Death
A total of 7804 patients had complete data across all variables in the final multivariable model, and 1176 patients died. Twenty-one predictors of CV death were identified on multivariate analysis (eAppendix 3B in the Supplement). The final prognostic model for CV death based on backward stepwise Cox proportional hazard regression is shown in Table 2.
Table 2. Final Model for CV Death Based on Backward Stepwise Cox Proportional Hazard Regressiona.
| Variable | HR (95% CI) | χ2 | Coefficient (SE) | P Value |
|---|---|---|---|---|
| Baseline characteristics | ||||
| Region (Latin America) | 1.84 (1.53-2.21) | 43.3 | 0.610 (0.093) | <.001 |
| Race/ethnicity (Asian) | 1.89 (1.57-2.30) | 43.2 | 0.639 (0.097) | <.001 |
| Region (central Europe) | 1.63 (1.37-1.92) | 32.6 | 0.487 (0.085) | <.001 |
| NYHA III/IV | 1.32 (1.16-1.51) | 17.3 | 0.278 (0.067) | <.001 |
| Ejection fraction (per 5% decrease <40%) | 1.10 (1.05-1.15) | 16.7 | 0.095 (0.023) | <.001 |
| Male | 1.37 (1.16-1.60) | 14.7 | 0.311 (0.081) | <.001 |
| HF duration >5 y | 1.36 (1.16-1.60) | 14.5 | 0.309 (0.081) | <.001 |
| Peripheral arterial disease | 1.41 (1.18-1.68) | 14.4 | 0.342 (0.090) | <.001 |
| Not prescribed sacubitril/valsartan | 1.25 (1.11-1.40) | 14.4 | 0.221 (0.058) | <.001 |
| Diabetes mellitus | 1.26 (1.12-1.42) | 14.5 | 0.232 (0.061) | <.001 |
| Age (per 10 y >60 y) | 1.15 (1.06-1.25) | 10.6 | 0.137 (0.042) | .001 |
| Prior MI | 1.23 (1.08-1.39) | 10.0 | 0.204 (0.064) | .002 |
| Not prescribed β-blocker | 1.36 (1.12-1.65) | 9.7 | 0.307 (0.099) | .002 |
| Bundle-branch block | 1.20 (1.06-1.35) | 8.2 | 0.180 (0.063) | .004 |
| HF duration 1-5 y | 1.21 (1.04-1.40) | 6.1 | 0.187 (0.076) | .01 |
| No previous PCI | 1.21 (1.03-1.41) | 5.3 | 0.187 (0.081) | .02 |
| Systolic BP (per 10 mm Hg decrease <120 mm Hg) | 1.10 (1.01-1.19) | 5.2 | 0.092 (0.040) | .045 |
| Race/ethnicity (black) | 1.34 (1.03-1.78) | 4.7 | 0.298 (0.137) | .03 |
| Laboratory values | ||||
| Total bilirubin (for every 0.29 mg/dL increase greater than 0.58 mg/dL) | 1.10 (1.06-1.14) | 26.5 | 0.097 (0.019) | <.001 |
| Potassium (for every 0.1 mEq/L decrease less than 4 mEq/L) | 1.09 (1.05-1.13) | 17.9 | 0.086 (0.020) | <.001 |
| Uric acid (for every 0.84 mg/dL increase greater than 6.72 mg/dL) | 1.07 (1.03-1.11) | 15.0 | 0.068 (0.018) | <.001 |
| Albumin (for every 0.1 g/dL decrease less than 4.2 g/dL) | 1.05 (1.02-1.08) | 12.8 | 0.049 (0.014) | <.001 |
| Hemoglobin (for every 1 g/dL decrease less than 14 g/dL) | 1.10 (1.04-1.17) | 10.8 | 0.097 (0.030) | .001 |
| Total cholesterol (for every 38.61 mg/dL increase greater than 115.83 mg/dL) | 1.09 (1.03-1.16) | 7.8 | 0.089 (0.032) | .005 |
| Urea (for every 2.8 mg/dL increase greater than 14.01 mg/dL) | 1.03 (1.01-1.05) | 6.8 | 0.029 (0.011) | .009 |
| Natriuretic peptides | ||||
| NTproBNP categoryb | 1.40 (1.32-1.48) | 122.3 | 0.0333 (0.030) | <.001 |
Abbreviations: BP, blood pressure; HF, heart failure; HR, hazard ratio; LDL, low-density lipoprotein; MI, myocardial infarction; NTproBNP, N-terminal pro brain natriuretic peptide; NYHA, New York Heart Association; PCI, percutaneous coronary intervention.
SI conversion factor: To convert albumin to grams per liter, multiply by 10; bilirubin to micromoles per liter, multiply by 17.104; hemoglobin to grams per liter, multiply by 10; total cholesterol to millimoles per liter, multiply by 0.0259; lymphocytes to ×109 per liter, multiply by 0.001; neutrophils to ×109 per liter, multiply by 0.001; potassium to millimoles per liter, multiply by 1; urea to millimoles per liter, multiply by 0.375; uric acid to micromoles per liter, multiply by 59.485.
No significant interaction terms were identified for this model.
NTproBNP categories: 1, <400 pg/mL; 2, 400 pg/mL-799 pg/mL; 3, 800 pg/mL-1599 pg/mL; 4, 1600 pg/mL-3199 pg/mL; 5, ≥3200 pg/mL.
All-Cause Death
A total of 7876 patients had complete data for all variables in the final multivariable model, and 1453 patients died. Twenty variables were independent predictors of all-cause death on multivariate analysis (eAppendix 3C in the Supplement). The final prognostic model for all-cause death based on backward stepwise Cox proportional hazard regression is shown in Table 3.
Table 3. Final Model for All-Cause Mortality Based on Backward Stepwise Cox Proportional Hazard Regression.
| Variable | HR (95% CI) | χ2 | Coefficient (SE) | P Value |
|---|---|---|---|---|
| Baseline characteristics | ||||
| Region (Latin America) | 1.69 (1.43-2.00) | 37.7 | 0.524 (0.085) | <.001 |
| Age (per 10 y >60 y) | 1.20 (1.11-1.30) | 22.9 | 0.184 (0.038) | <.001 |
| Region (central Europe) | 1.42 (1.23-1.65) | 21.9 | 0.353 (0.075) | <.001 |
| NYHA III/IV | 1.30 (1.15-1.47) | 17.8 | 0.262 (0.062) | <.001 |
| Diabetes mellitus | 1.26 (1.12-1.41) | 15.9 | 0.230 (0.058) | <.001 |
| HF duration >5 y | 1.35 (1.16-1.56) | 15.8 | 0.300 (0.075) | <.001 |
| Male | 1.34 (1.16-1.55) | 15.3 | 0.290 (0.074) | <.001 |
| Peripheral arterial disease | 1.36 (1.16-1.60) | 14.2 | 0.311 (0.082) | <.001 |
| Race/ethnicity (Asian) | 1.42 (1.18-1.71) | 13.4 | 0.348 (0.095) | <.001 |
| Ejection fraction (per 5% decrease <40%) | 1.08 (1.03-1.12) | 11.8 | 0.075 (0.022) | <.001 |
| No previous PCI | 1.29 (1.12-1.50) | 11.7 | 0.257 (0.075) | .001 |
| HF duration 1-5 y | 1.25 (1.09-1.44) | 10.1 | 0.224 (0.070) | .001 |
| Not prescribed sacubitril/valsartan | 1.18 (1.07-1.31) | 9.9 | 0.168 (0.063) | .002 |
| Not prescribed β-blocker | 1.27 (1.06-1.53) | 6.8 | 0.243 (0.092) | .009 |
| Prior MI | 1.15 (1.03-1.29) | 5.9 | 0.143 (0.059) | .02 |
| Systolic BP (per 10 mm Hg decrease <120 mm Hg) | 1.09 (1.01-1.17) | 5.0 | 0.083 (0.037) | .03 |
| BMI (for every kg/m2 decrease less than 30 kg/m2) | 1.02 (1.00-1.04) | 4.4 | 0.019 (0.001) | .04 |
| Laboratory values | ||||
| Albumin (for every 0.1 g/dL decrease less than 4.2 g/dL) | 1.06 (1.03-1.08) | 20.5 | 0.057 (0.012) | <.001 |
| Uric acid (for every 0.84 mg/dL increase greater than 6.72 mg/dL) | 1.07 (1.04-1.11) | 18.1 | 0.071 (0.017) | <.001 |
| Total bilirubin (for every 0.29 mg/dL increase greater than 0.58 mg/dL) | 1.08 (1.04-1.13) | 16.9 | 0.080 (0.020) | <.001 |
| % Monocytes for every % greater than 7% | 1.04 (1.02-1.07) | 16.4 | 0.042 (0.010) | <.001 |
| Absolute neutrophils (for every 1000/mL increase below 6000/mL) | 1.10 (1.05-1.15) | 16.3 | 0.095 (0.024) | <.001 |
| Hemoglobin (for every 1 g/dL decrease less than 14 g/dL) | 1.11 (1.05-1.17) | 15.3 | 0.106 (0.027) | <.001 |
| Chloride (for every 1 mEq/L decrease 100 mEq/L) | 1.06 (1.03-1.09) | 13.0 | 0.054 (0.015) | <.001 |
| LDL (for every 38.61 mg/dL increase greater than 115.83 mg/dL) | 1.19 (1.07-1.33) | 9.9 | 0.176 (0.056) | .002 |
| Potassium (for every 0.1 mEq/L decrease less than 4 mEq/L) | 1.05 (1.02-1.10) | 7.8 | 0.056 (0.020) | .005 |
| AST (For every 598.8 U/L increase greater than 1796.4 U/L) | 1.07 (1.01-1.13) | 5.6 | 0.069 (0.029) | .02 |
| Urea (for every 2.8 mg/dL increase greater than 14.01 mg/dL) | 1.02 (1.00-1.04) | 5.4 | 0.024 (0.010) | .02 |
| Triglycerides (for every 88.5 mg/dL decrease less than 221.24 mg/dL) | 1.12 (1.01-1.24) | 4.8 | 0.112 (0.051) | .03 |
| Natriuretic peptides | ||||
| NTproBNP categorya | 1.33 (1.24-1.38) | 93.3 | 0.270 (0.028) | <.001 |
Abbreviations: AST, aspartate aminotransferase; BMI, body mass index (calculated as weight in kilograms divided by height in meters squared); BP, blood pressure; HF, heart failure; HR, hazard ratio; LDL, low-density lipoprotein; MI, myocardial infarction; NTproBNP, N-terminal pro brain natriuretic peptide; NYHA, New York Heart Association; PCI, percutaneous coronary intervention.
SI conversion factor: To convert albumin to grams per liter, multiply by 10; AST to microkatals per liter, multiply by 0.0167; bilirubin to micromoles per liter, multiply by 17.104; chloride to millimoles per liter, multiply by 1; hemoglobin to grams per liter, multiply by 10; LDL cholesterol to millimoles per liter, multiply by 0.0259; lymphocytes to ×109 per liter, multiply by 0.001; neutrophils to ×109 per liter, multiply by 0.001; potassium to millimoles per liter, multiply by 1; urea to millimoles per liter, multiply by 0.375; uric acid to micromoles per liter, multiply by 59.485.
NTproBNP categories: 1, <400 pg/mL; 2, 400 pg/mL-799 pg/mL; 3, 800 pg/mL-1599 pg/mL; 4, 1600 pg/mL-3199 pg/mL; 5, ≥3200 pg/mL.
Comparison of Predictors
Most of the clinical/demographic predictive variables were the same in each of the 3 models. The common variables were: sex, race/ethnicity, region, duration of HF, NYHA class, LVEF, diabetes mellitus, prior myocardial infarction, peripheral arterial disease, β-blocker use at baseline, and allocation to sacubitril/valsartan.
Prior HF hospitalization was only predictive of the primary composite outcome as was valvular heart disease. Bundle-branch block predicted the primary end point and CV death but not all-cause death. Age predicted CV death and all-cause mortality, and prior percutaneous coronary intervention predicted CV and all-cause death but not the primary composite outcome. Body mass index was predictive of all-cause death only.
Among the routine laboratory variables, the top 3 predictors for all 3 outcomes were bilirubin, uric acid, and albumin. Ranked by χ2, NTproBNP was the most powerful predictor of each of the 3 outcomes.
Missing Data
The models for all 3 end points were developed on complete cases. The extent of missing data for baseline characteristics, laboratory values, and natriuretic peptides are shown in eAppendix 4 in the Supplement. Data were complete for 50 of 63 candidate variables. The proportion of missing data for the other 13 variables ranged from 0.01% to 0.2%. There were also few missing data for laboratory variables (ranging from 1.8% to 5.6%).
Predicting an Individual’s Risk
Model calibration for each end point examined was explored by comparing the predicted and observed probabilities across patient risk quintiles at 1 and 2 years (Figure 1 and Figure 2). For the primary composite end point there was a strong gradient in risk, with patients in the highest risk quintile having 9 times the risk of HF hospitalization or CV death than patients in the lowest-risk quintile (Figure 1A). This was similar for CV death and all-cause death, with patients in the fifth quintile having 9 and 8 times the risk of the outcome in question when compared with patients in the first quintile (Figures 1B and C). A similar trend was seen across the quintiles for predicted and observed risk at 2 years with gradient risk between the first and fifth quintile of 6 for all outcomes (Figure 2). Kaplan-Meier event-free survival curves over 3 years for the primary composite end point, CV death, and all-cause mortality by quintile of risk were estimated and compared directly with baseline survival function using Cox proportional hazards method (eAppendix 5 and 6 in the Supplement).
Figure 1. Observed and Predicted Risk in Prospective Comparison of ARNI With ACEI to Determine Impact on Global Mortality and Morbidity in Heart Failure trial (PARADIGM-HF) at 1 Year for Primary Composite End Point (A), Cardiovascular Death (B), and All-Cause Mortality for Patients Categorized by Quintile of Risk Scores (C).
A, Predicted vs observed risk of primary composite end point at 1 year. B, Predicted vs observed risk of cardiovascular death at 1 year. C, Predicted vs observed risk of all-cause mortality at 1 year.
Figure 2. Observed and Predicted Risk in Prospective Comparison of ARNI With ACEI to Determine Impact on Global Mortality and Morbidity in Heart Failure trial (PARADIGM-HF) at 2 Years for Primary Composite End Point (A), Cardiovascular Death (B), and All-Cause Mortality for Patients Categorized by Quintile of Risk Score (C).
A, Predicted vs observed risk of primary composite end point at 2 years. B, Predicted vs observed risk of cardiovascular death at 2 years. C. Predicted vs observed risk of all-cause mortality at 2 years.
The C statistic at 1 and 2 years for the primary composite end point was 0.74 (95% CI, 0.71-0.76) and 0.71 (95% CI, 0.70-0.75), respectively. For cardiovascular death, the C statistic for 1 and 2 years was 0.73 (95% CI, 0.71-0.75) and 0.71 (95% CI, 0.69-073), respectively, and for all-cause death it was 0.71 (95% CI, 0.69-0.74) and 0.70 (95% CI, 0.67-0.72), respectively.
The models performed similarly well without the inclusion of natriuretic peptides, although performance was better with their inclusion and was substantially better than natriuretic peptides on their own (eAppendix 7 in the Supplement). The model performed better than currently available scores (eAppendix 8 in the Supplement).
External Validation of the Models
The PARADIGM-HF prognostic model was prospectively validated in ATMOSPHERE. Baseline characteristics for patients in the ATMOPSHERE cohort according to outcome are shown in the eAppendix 2B in the Supplement. The C statistic for the model applied to the ATMOSPHERE cohort for the primary composite end point at 1 and 2 years was 0.71 (95% CI, 0.69-0.72) and 0.70 (95% CI, 0.68-0.71), respectively. For CV death, the Harrell C statistic for 1 and 2 years was 0.71 (95% CI, 0.69-0.74) and 0.70 (95% CI, 0.69-0.72), respectively, and for all-cause death was 0.71 (95% CI, 0.69-0.74) and 0.70 (95% CI, 0.68-0.72), respectively.
The PARADIGM-HF model was also validated in SwedeHF for all-cause death. Baseline characteristics for patients in the SwedeHF cohort by outcome are shown in the eAppendix 2C in the Supplement. The C statistic for the model applied to SwedeHF for all-cause death at 1 and 2 years was 0.79 (95% CI, 0.75-0.81) and 0.78 (95% CI, 0.75-0.80) (see eAppendix 9B in the Supplement).
Sensitivity Analyses
The models were developed using BNP instead of NTproBNP. The C statistic for the model using BNP for the primary end point, CV death, and all-cause death were 0.72, 0.73, and 0.72 and 0.70, 0.72, and 0.70 at 1 and 2 years, respectively. The final models including BNP are shown in eAppendix 10 in the Supplement (eAppendix 10 in the Supplement).
Calculation of Individual Estimated Survival
An online calculator (http://www.predict-hf.com) was created for each outcome to allow calculation of an individual’s risk in the outpatient setting. These tools allow calculation of risk at each model step (demographics/clinical characteristics, routine laboratory variables, and natriuretic peptide level) and, if natriuretic peptide levels are available, either NTproBNP or BNP.
Discussion
Our prognostic models for patients with chronic ambulatory HF reduced ejection fraction (HF-REF) were developed in the PARADIGM-HF cohort and validated in the ATMOSPHERE cohort (and for all-cause death in SwedeHF). The models for the composite outcome of CV death or HF hospitalization (1853 derivation events and 1988 validation events), CV death alone (1176 of 1371), and all-cause death (1453 of 1627) demonstrated good calibration and discrimination. To our knowledge, this is one of the few studies of prognostic models in HF to report external validation, is one of a minority that included natriuretic peptides, and is by far the largest to include either of these features.1,2,3,4 The cohorts used had more extensive and complete collection of clinical and laboratory data at baseline, were better treated with contemporary evidence-based treatment (100% treated with ACE inhibitor/ARB, 93% β-blockers, and 56% MRA in PARADIGM-HF) than in any prior study of a similar nature, and were more geographically representative than in previous reports. Because PARADIGM-HF was a global clinical trial, enrolling patients from 1043 centers in 47 countries (and ATMOSPHERE from 789 centers in 43 countries), we could examine both geographical region and race/ethnicity in our models, which has rarely been possible previously.1,2,3,4 One model, derived in the Biology Study to Tailored Treatment in Chronic Heart Failure cohort (n = 2516) did incorporate natriuretic peptides and was externally validated12 but only enrolled European patients and those not treated with (or with ≤50% of the target dose of) an ACE inhibitor/ARB and/or a β-blocker at baseline. Most patients were enrolled during a hospital admission with worsening heart failure, and, although a higher risk cohort, only 657 deaths were reported (compared with 1546 in PARADIGM-HF), presumably because of the relatively small sample size.
Most of the predictors identified in our models are established prognostic markers in HF, although some are less so because of incomplete collection in prior studies. These less-well-recognized predictors include peripheral arterial disease, β-blocker use, bundle-branch block, prior percutaneous coronary intervention, race/ethnicity, and region.
The presence of peripheral arterial disease and electrocardiogram QRS configuration has not been documented in many HF trials, but where they have been, they have consistently been predictors of poor outcome.13,14 β-Blockers (and even ACE inhibitors/ARBs and MRAs) were much less frequently taken by patients included in some of the studies, giving rise to established predictive models. In the derivation cohort used to generate the widely used SHFM, only 3% of patients were taking an MRA and none were prescribed a β-blocker.6 Overall, in Meta-Analysis Global Group in Chronic Heart Failure (MAGGIC), only 67% of patients were treated with an ACE inhibitor/ARB, 34% with a β-blocker, and 21% with an MRA.15 In the CHARM and 3C-HF data sets, 55% and 67% of patients, respectively, were prescribed a β-blocker, but only 17% of CHARM patients were prescribed an MRA (not reported in 3C-HF).16,17 Although pharmacologic therapy has been more comprehensive in some models, and a few of these included biomarkers, the patients enrolled came from a handful of countries or even a single country.18 Each of these limitations make the generalizability of these prior models to contemporary patients, especially those not living in Western Europe or North America, uncertain. This is especially important because both region and race/ethnicity featured prominently among the predictive variables in each of our 3 models.
Routine laboratory tests added valuable prognostic information in our models. Few studies have measured each of these biomarkers together, and many prior cohorts had a high proportion of patients with missing data; eg, in validation of the SHFM, 65% of uric acid levels and up to 100% of lymphocyte values were missing (compared with more than 90% completeness in our derivation and validation cohorts).6 Consequently, we describe, to our knowledge, the most comprehensive and complete simultaneous evaluation of the relative predictive value of routine biomarkers in patients with HF-REF. Additionally, the SHFM is known to underestimate risk in high-risk HF, when risk prediction is particularly important for selecting patients for scarce and/or expensive therapy, such as heart transplantation or durable mechanical circulatory support.19 In contrast, our model discriminated equally well regardless of risk, although the actual rate of the primary composite end point was minimally greater than the predicted in the highest-risk quintile. Creatinine and estimated glomerular filtration rate (eGFR) were predictors of outcome in our univariable models but notably missing from the multivariable models, although blood urea nitrogen (BUN) was retained. Surprisingly, although widely considered an important predictor of outcome in heart failure, eGFR has not been included in many prior models and usually not simultaneously with BUN (although it has been used with creatinine).1,2,3,4,20 The Biology Study to Tailored Treatment in Chronic Heart Failure did and found, as we did, that BUN displaced eGFR/creatinine.12
A particular strength of our model was the inclusion of both NTproBNP and BNP. Despite the inclusion of a wide range of demographic and clinical variables, and as alluded to, a comprehensive routine laboratory panel, NTproBNP and BNP had the highest predictive value for each outcome. Our main model was developed using NTproBNP because this value was available in 99.8% of patients in the derivation cohort (n = 8385) and 90.5% of patients in the validation cohort (n = 6352). However, BNP performed similarly.
Another strength was external validation of our models in a similarly contemporaneous population (the ATMOSPHERE trial). Our models performed well in the ATMOSPHERE data set. We were also able to validate our all-cause mortality model in a third, real-world data set, the SwedeHF registry. Again, our models performed well, indeed, at least as well as in the ATMOSPHERE data set. Therefore our model, using contemporary patients from around the world, receiving contemporary levels of guideline-recommended therapies and using natriuretic peptides, and that has been externally validated in a real-world cohort, may be considered the strongest of currently available models. To make our models clinically applicable, we have created an online calculator.
Limitations
Our study also had limitations. Both the derivation and main validation data sets were obtained from clinical trials and therefore included selected patients (for example, both trials excluded individuals with markedly reduced renal function); however, this limitation was overcome to some extent by additional validation of one of the outcomes in SwedeHF, where prescribing rates and doses are different to the 2 clinical trials. Both trial data sets included only patients with HF-REF, and we do not know the value of our models in patients with HF with preserved ejection fraction. Implantable cardioverter defibrillators and cardiac resynchronization therapy were used relatively infrequently in PARADIGM-HF and ATMOSPHERE, and we do not know how well our models would perform in patients extensively treated with such devices. Additional prognostic variables (such as functional capacity and measures of frailty), and biomarkers were not available in PARADIGM-HF. The accessibility and clinical application of our models might be easier if we created an integer score, but this would have resulted in loss of some predictive accuracy.
Conclusions
In summary, the PARADIGM-HF models can accurately predict morbidity and mortality in ambulatory patients with chronic HF-REF at 1 and 2 years. The models are based on readily available clinical and laboratory characteristics and were derived from a robust, complete, data set where most patients were receiving evidence-based therapy for HF. External validation in a similarly large cohort confirmed the model’s discriminative ability. Clinically, the model can be used to predict patient outcomes and has potential wider uses in the organization and targeting of services and therapies in patients with HF-REF.
eAppendix 1. Inclusion and exclusion criteria of PARADIGM-HF and ATMOSPHERE
eAppendix 2. Baseline characteristics
eAppendix 3. Models with baseline characteristics only
eAppendix 4. Extent of missing data
eAppendix 5. Observed vs predicted risk by by deciles
eAppendix 6. Kaplan-Meier plots for observed versus predicted event rate in PARADIGM-HF
eAppendix 7. Incremental value of natriuretic peptides
eAppendix 8. Comparison of performance by C-statistic of PREDICT-HF with existing risk prediction models
eAppendix 9. Model validation
eAppendix 10. Models with BNP
References
- 1.Alba AC, Agoritsas T, Jankowski M, et al. . Risk prediction models for mortality in ambulatory patients with heart failure: a systematic review. Circ Heart Fail. 2013;6(5):881-889. doi: 10.1161/CIRCHEARTFAILURE.112.000043 [DOI] [PubMed] [Google Scholar]
- 2.Ouwerkerk W, Voors AA, Zwinderman AH. Factors influencing the predictive power of models for predicting mortality and/or heart failure hospitalization in patients with heart failure. JACC Heart Fail. 2014;2(5):429-436. doi: 10.1016/j.jchf.2014.04.006 [DOI] [PubMed] [Google Scholar]
- 3.Rahimi K, Bennett D, Conrad N, et al. . Risk prediction in patients with heart failure: a systematic review and analysis. JACC Heart Fail. 2014;2(5):440-446. doi: 10.1016/j.jchf.2014.04.008 [DOI] [PubMed] [Google Scholar]
- 4.Ferrero P, Iacovoni A, D’Elia E, Vaduganathan M, Gavazzi A, Senni M. Prognostic scores in heart failure: critical appraisal and practical use. Int J Cardiol. 2015;188:1-9. doi: 10.1016/j.ijcard.2015.03.154 [DOI] [PubMed] [Google Scholar]
- 5.Simpson J, Jhund PS, Silva Cardoso J, et al. ; PARADIGM-HF Investigators and Committees . Comparing LCZ696 with enalapril according to baseline risk using the MAGGIC and EMPHASIS-HF risk scores: an analysis of mortality and morbidity in PARADIGM-HF. J Am Coll Cardiol. 2015;66(19):2059-2071. doi: 10.1016/j.jacc.2015.08.878 [DOI] [PubMed] [Google Scholar]
- 6.Levy WC, Mozaffarian D, Linker DT, et al. . The Seattle Heart Failure Model: prediction of survival in heart failure. Circulation. 2006;113(11):1424-1433. doi: 10.1161/CIRCULATIONAHA.105.584102 [DOI] [PubMed] [Google Scholar]
- 7.McMurray JJ, Packer M, Desai AS, et al. ; PARADIGM-HF Committees and Investigators . Dual angiotensin receptor and neprilysin inhibition as an alternative to angiotensin-converting enzyme inhibition in patients with chronic systolic heart failure: rationale for and design of the Prospective comparison of ARNI with ACEI to Determine Impact on Global Mortality and morbidity in Heart Failure trial (PARADIGM-HF). Eur J Heart Fail. 2013;15(9):1062-1073. doi: 10.1093/eurjhf/hft052 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.McMurray JJ, Packer M, Desai AS, et al. ; PARADIGM-HF Investigators and Committees . Angiotensin-neprilysin inhibition versus enalapril in heart failure. N Engl J Med. 2014;371(11):993-1004. doi: 10.1056/NEJMoa1409077 [DOI] [PubMed] [Google Scholar]
- 9.Krum H, Massie B, Abraham WT, et al. ; ATMOSPHERE Investigators . Direct renin inhibition in addition to or as an alternative to angiotensin converting enzyme inhibition in patients with chronic systolic heart failure: rationale and design of the Aliskiren Trial to Minimize Outcomes in Patients with Heart Failure (ATMOSPHERE) study. Eur J Heart Fail. 2011;13(1):107-114. doi: 10.1093/eurjhf/hfq212 [DOI] [PubMed] [Google Scholar]
- 10.McMurray JJ, Krum H, Abraham WT, et al. ; ATMOSPHERE Committees Investigators . Aliskiren, enalapril, or aliskiren and enalapril in heart failure. N Engl J Med. 2016;374(16):1521-1532. doi: 10.1056/NEJMoa1514859 [DOI] [PubMed] [Google Scholar]
- 11.Jonsson A, Edner M, Alehagen U, Dahlström U. Heart failure registry: a valuable tool for improving the management of patients with heart failure. Eur J Heart Fail. 2010;12(1):25-31. doi: 10.1093/eurjhf/hfp175 [DOI] [PubMed] [Google Scholar]
- 12.Voors AA, Ouwerkerk W, Zannad F, et al. . Development and validation of multivariable models to predict mortality and hospitalization in patients with heart failure. Eur J Heart Fail. 2017;19(5):627-634. doi: 10.1002/ejhf.785 [DOI] [PubMed] [Google Scholar]
- 13.Cannon JA, Collier TJ, Shen L, et al. . Clinical outcomes according to QRS duration and morphology in the Eplerenone in Mild Patients: Hospitalization and SurvIval Study in Heart Failure (EMPHASIS-HF). Eur J Heart Fail. 2015;17(7):707-716. doi: 10.1002/ejhf.303 [DOI] [PubMed] [Google Scholar]
- 14.Inglis SC, Hermis A, Shehab S, Newton PJ, Lal S, Davidson PM. Peripheral arterial disease and chronic heart failure: a dangerous mix. Heart Fail Rev. 2013;18(4):457-464. doi: 10.1007/s10741-012-9331-1 [DOI] [PubMed] [Google Scholar]
- 15.Pocock SJ, Ariti CA, McMurray JJ, et al. ; Meta-Analysis Global Group in Chronic Heart Failure . Predicting survival in heart failure: a risk score based on 39 372 patients from 30 studies. Eur Heart J. 2013;34(19):1404-1413. doi: 10.1093/eurheartj/ehs337 [DOI] [PubMed] [Google Scholar]
- 16.Pocock SJ, Wang D, Pfeffer MA, et al. . Predictors of mortality and morbidity in patients with chronic heart failure. Eur Heart J. 2006;27(1):65-75. doi: 10.1093/eurheartj/ehi555 [DOI] [PubMed] [Google Scholar]
- 17.Senni M, Parrella P, De Maria R, et al. . Predicting heart failure outcome from cardiac and comorbid conditions: the 3C-HF score. Int J Cardiol. 2013;163(2):206-211. doi: 10.1016/j.ijcard.2011.10.071 [DOI] [PubMed] [Google Scholar]
- 18.Barlera S, Tavazzi L, Franzosi MG, et al. ; GISSI-HF Investigators . Predictors of mortality in 6975 patients with chronic heart failure in the Gruppo Italiano per lo Studio della Streptochinasi nell’Infarto Miocardico-Heart Failure trial: proposal for a nomogram. Circ Heart Fail. 2013;6(1):31-39. doi: 10.1161/CIRCHEARTFAILURE.112.967828 [DOI] [PubMed] [Google Scholar]
- 19.Sartipy U, Goda A, Yuzefpolskaya M, Mancini DM, Lund LH. Utility of the Seattle Heart Failure Model in patients with cardiac resynchronization therapy and implantable cardioverter defibrillator referred for heart transplantation. Am Heart J. 2014;168(3):325-331. doi: 10.1016/j.ahj.2014.03.025 [DOI] [PubMed] [Google Scholar]
- 20.Cleland JG, Chiswell K, Teerlink JR, et al. . Predictors of postdischarge outcomes from information acquired shortly after admission for acute heart failure: a report from the Placebo-Controlled Randomized Study of the Selective A1 Adenosine Receptor Antagonist Rolofylline for Patients Hospitalized With Acute Decompensated Heart Failure and Volume Overload to Assess Treatment Effect on Congestion and Renal Function (PROTECT) Study. Circ Heart Fail. 2014;7(1):76-87. doi: 10.1161/CIRCHEARTFAILURE.113.000284 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eAppendix 1. Inclusion and exclusion criteria of PARADIGM-HF and ATMOSPHERE
eAppendix 2. Baseline characteristics
eAppendix 3. Models with baseline characteristics only
eAppendix 4. Extent of missing data
eAppendix 5. Observed vs predicted risk by by deciles
eAppendix 6. Kaplan-Meier plots for observed versus predicted event rate in PARADIGM-HF
eAppendix 7. Incremental value of natriuretic peptides
eAppendix 8. Comparison of performance by C-statistic of PREDICT-HF with existing risk prediction models
eAppendix 9. Model validation
eAppendix 10. Models with BNP