Validation of Heart Failure-Specific Risk Equations in 1.3 Million Israeli Adults and Usefulness of Combining Ambulatory and Hospitalization Data from a Large Integrated Health Care Organization

Abstract

Heart failure prevalence is increasing worldwide and is associated with significant morbidity and mortality. Guidelines emphasize prevention in those at risk, but HF-specific risk prediction equations developed in US population-based cohorts lack external validation in large, real-world datasets outside of the US. The purpose of this study was to assess the model performance of the Pooled Cohort Equations to Prevent Heart Failure (PCP-HF) within a contemporary electronic health record for 5-year and 10-year risk. Utilizing a retrospective cohort study design of Israeli residents between 2008 and 2018 with continuous membership until end of follow-up, HF or death, we quantified 5-year and 10-year estimated risk of HF using the PCP-HF equations, which integrate demographics (age, sex, race) and risk factors (body mass index, systolic blood pressure, glucose, medication use for hypertension or diabetes, and smoking status). Of 1,394,411 patients included, 56% were women with mean age of 49.6 (standard deviation 13.2) years. Incident HF occurred in 1.2% and 4.5% of participants over 5- and 10-years of follow-up. The PCP-HF model had excellent discrimination for 5-year and 10-year prediction of incident HF (C statistic 0.82 [0.82, 0.82] and 0.84 [0.84, 0.84]), respectively. In conclusion, HF-specific risk equations (PCP-HF) accurately predict the risk of incident HF in ambulatory and hospitalized patients using routinely available clinical data.

INTRODUCTION

Heart failure (HF) is a significant and growing public health problem with an estimated prevalence of 26 million people globally.^1,2 Despite remarkable advances in guideline-directed medical, surgical, and device therapies, morbidity and mortality due to HF remain high, with 17 to 45% of deaths occurring within one year of diagnosis.^3–5 Focused efforts on prevention of HF are needed.⁶ While several HF-specific risk scores have been developed^7,8, none are currently utilized in clinical practice due to lack of generalizability and external validation. We recently developed and validated a new HF-specific risk estimation model, the Pooled Cohort Equations to Prevent Heart Failure (PCP-HF) that utilizes readily available risk factor levels in the primary care setting.⁹ However, only hospitalized HF cases were included and all individuals were participants from cohort studies, which may not be representative of the general population. Therefore, we sought to examine the predictive accuracy of the PCP-HF tool in a contemporary, large, integrated health system that comprised of a real-world, non-US sample and to examine the utility of the PCP-HF with inclusion of ambulatory and hospitalized HF cases.

METHODS

As a result of the National Health Insurance Law, Israeli citizens are required to enroll in one of four payer-provider health funds and receive free, basic health care. Clalit Health Services provides health insurance coverage to approximately 50% of the eligible population in Israel and represents about 4.6 million insured members. We conducted a retrospective cohort study of participants insured by Clalit, including those with continuous membership at least for 1 year prior to the index date (June 1, 2013 for 5-year [Figure 1] and June 1, 2008 for 10-year [Online Figure 1] cohorts). Participants were excluded for ages <30 years or >80 years or a history of pre-existing cardiovascular disease, defined as HF, coronary heart disease (myocardial infarction, unstable angina pectoris, angioplasty, coronary artery bypass graft, and stable angina), stroke, or atrial fibrillation using the International Classification of Diseases, ninth Revision (ICD-9, Online Table 1). Medical diagnoses as of index date were primarily defined based on ICD-9 codes extracted from hospital discharge records or ambulatory medical records. Ambulatory records were verified by analyzing available written text. Among the eligible participants, further exclusions included missing data for calculation of the PCP-HF risk score (body mass index [BMI], systolic blood pressure [SBP], total cholesterol [TC], high density lipoprotein cholesterol [HDL-C], or random glucose level) or insufficient follow-up for complete case ascertainment. This study was approved by the Clalit Health Service Institutional Review Board. Receipt of vital status from the Ministry of Interior ensured 100% follow-up for mortality.

Figure 1. Primary analytic sample.

Selection of the primary prevention retrospective cohort at-risk for HF with 5-year follow-up

Baseline characteristics, comorbidities, and risk factor levels were ascertained based on the last documented results prior to the index date in the Clalit electronic health record data. Laboratory values included random blood glucose, TC, and HDL-C. Clinically extreme or improbable values were set to missing (BMI <16 kg/m² or >50 kg/m², SBP<80 mm Hg or >200 mm Hg, or glucose <60 mg/dL or >400 mg/dL). Treatment of diabetes or hypertension was coded based on filled prescription records using the Anatomical Therapeutic Chemical classification system. Medication use for diabetes and hypertension was based on at least one purchase in the year before the index date (Online Table 2).

The PCP-HF model was developed from sex- and race-specific proportional hazards models from 5 US population-based cohorts, including a total of 23,541 participants, and has been validated to-date in 2 population-based cohorts⁹ and a single US health system¹⁰. The included covariates were chosen based on their known association with incident HF. The PCP-HF model without QRS, which was validated in the same derivation population as the model with QRS,¹⁰ was applied to predict risk of HF in the Clalit population at 5-years and 10-years of follow-up. A new diagnosis of HF, including hospital admission and ambulatory diagnosis, was ascertained through use of ICD-9 codes and new prescription of medical therapy for HF (including β-blocking agents, agents acting on the renin-angiotensin system, or diuretic therapy, each defined as a patient being prescribed ≥1 type of medication during follow-up). The use of prescription data is utilized to enhance the specificity of the outcome of HF, as demonstrated previously in the REGARDS cohort.¹¹

Baseline characteristics of the study population were described using means with standard deviations (SDs) for continuous and with proportions for categorical variables. Termination of follow-up was defined as incident HF event, death, or end of study period. To assess model performance, the model was first linearly recalibrated on a training set (a random 50% of the sample) using previously described methods.^12,13 Specifically, the logistic regression model was analyzed on the training set with the logits of the PCP-HF predictions as the sole variable, and the resulting intercept and slope were used to adjust the predictions on the test set (remaining 50%). Model calibration was visually assessed. Predicted 5- and 10- year sex-specific risk equations for HF were applied (PCP-HF) for each participant and absolute HF risk was calculated. Model discrimination was assessed using area under the curve (AUC). The AUC was defined a priori based on prior publications¹⁴ with less than 0.70 as inadequate, 0.70 to 0.80 as acceptable, and greater than 0.80 as excellent. Confidence intervals for the AUC were calculated using the bootstrap methods with 2,000 repetitions. All statistical analyses were performed using R version 3.6.

RESULTS

There were 1,394,411 and 760,750 members in Clalit Health Service between the ages of 30–80 years old who were included based on required 5-years and 10-years of follow-up, respectively. Baseline characteristics were similar for the 5-year (Table 1) and 10-year samples (Online Table 3) including a well-balanced sample by sex. Mean BMI, SBP, and fasting glucose were higher for those with higher predicted risk compared with lower predicted risk of HF at both 5- and 10-years (Online Tables 4–5). Baseline characteristics of participants excluded for lack of follow-up demonstrate a slightly younger population (Online Table 6).

Table 1.

Baseline characteristics of eligible participants aged 30–80 years free of cardiovascular disease in an integrated health system and 5-years of follow-up

Variables	5-Year Cohort N=1,394,411
	Men N=615,251	Women N=779,160
Age (years), Mean±SD	48.6±12.8	50.3±13.4
Current smoker	28.6%	13.9%
Diabetes mellitus	9.6%	9.5%
Fasting glucose (mg/dL), mean±SD	99±28	97±26
Systolic blood pressure (mm Hg), mean±SD	123±13	119±14
Body mass index (kg/m2), mean±SD	26.9±4.4	27.0±5.7
Total cholesterol (mg/dL), mean±SD	185±37	190±37
HDL cholesterol (mg/dL), mean±SD	44±11	55±13

SD reflects standard deviation; HDL high-density lipoprotein

There were 16,351 (1.9%) and 34,505 (4.5%) incident HF events in the 5-year and 10-year follow-up cohorts, respectively. Linear recalibration improved visual calibration in subgroups stratified by sex and age categories (Figure 2, Online Figures 2–4). Overall AUC was similar in the 5-year and 10-year samples (Table 2). The AUC was statistically significantly higher in women compared with men (for both 5-year and 10-year risk prediction of HF. AUC was statistically better among the younger subgroup aged 30–65 years at baseline compared with those aged 65–80 years for 5-year and 10-year HF risk prediction.

Figure 2.

Sex-specific calibration plots demonstrating non-recalibrated (A,C) and calibrated (B, D) model performance of the PCP-HF score for 5-year risk prediction of ambulatory and hospitalized heart failure

Table 2.

C Statistics for 5-Year and 10-Year Risk of Heart Failure in the Overall Population and Stratified by Sex and Age Subgroups

	All	Men	Women	30–65 years	65–80 years
5-year Risk Prediction of HF
All participants	N=1,394,411	N=615,251	N=779,160	N=1,201,949	N=192,462
Events no.	16,351	8193	8158
C Statistic (95% CI)	0.84 (0.84, 0.84)	0.82 (0.81, 0.82)	0.86 (0.86, 0.87)	0.82 (0.81, 0.82)	0.69 (0.68, 0.69)
10-year Risk Prediction of HF
All participants	N=760,750	N=309,845	N=450,905	N=624,953	N=135,797
Events no.	34,505	18,843	15,662
C Statistic (95% CI)	0.82 (0.82, 0.82)	0.79 (0.78, 0.79)	0.84 (0.84, 0.84)	0.78 (0.78, 0.78)	0.69 (0.68, 0.69)

DISCUSSION

In this retrospective cohort study of 1.3 million adults in Israel, the PCP-HF model performed well at predicting incident HF in the short-term (5- to 10-years) across a broad range of ages. Superior performance was noted in younger adults who may derive the greatest absolute benefit from earlier initiation of targeted HF prevention. This builds upon our prior work demonstrating good discrimination of the PCP-HF in a single-center study in the US of ~30,000 patients.¹⁵ Specifically, we further extend the generalizability and demonstrate utility of the PCP-HF prediction model in >1 million adults across multiple centers in a non-US population with the use of both billing codes and prescription data. This is especially important given the growing burden of HF outside the US in Europe⁴ and Asia¹⁶.

While several HF-specific risk scores have been developed¹⁷, no risk scores are widely utilized in clinical practice. Some risk scores were derived in specific populations (e.g. DM¹⁸ or older adults¹⁹) or utilize trial data²⁰, which are not representative of a primary care population. Further, some risk scores integrate use of biomarkers and echocardiography²¹, which limit widespread implementation as these are not routine tests and may not be cost effective. It is important to distinguish the role of the PCP-HF tool without biomarkers for risk prediction in asymptomatic individuals in contrast with the key role of biomarkers and imaging in the diagnosis of HF in symptomatic individuals.

Application of risk scores in populations that were not included in the derivation is often hampered by poor calibration.²² Recalibration, as was performed in this analysis, can optimize a tool for a new population. Importantly, discrimination does not change with recalibration and the PCP-HF tool demonstrated excellent discrimination in this sample with an AUC>0.80, which was consistent or better than AUC values observed in other risk prediction tools, such as the Pooled Cohort Equations to predict atherosclerotic cardiovascular disease. The current study expands upon prior work in a Flemish population that demonstrated that the PCP-HF tool accurately identified adverse cardiac remodeling (e.g., left ventricular concentric remodeling, diastolic dysfunction), key precursors to incident HF.²³ Lastly, the PCP-HF score has also been associated with global CVD events and all-cause mortality in a nationally representative sample of US adults in the National Health and Nutrition Examination Survey III cohort.²⁴

Recently a new class of medications, sodium-glucose co-transporter-2 inhibitors (SGLT2i), have emerged as a promising strategy for HF prevention.²⁵ Across several large randomized trials in patients with diabetes or chronic kidney disease, SGLT2i have consistently been associated with an approximately 30% reduction in risk of incident HF²⁶. The pleotropic effects of SGLT2i have been hypothesized to be due to multiple mechanisms, independent of glucose lowering, including favorable myocardial remodeling and natriuretic properties. A current clinical trial is ongoing focusing on the prevention of clinical HF in those individuals with asymptomatic left ventricular dysfunction without clinical signs or symptoms of HF with dapagliflozin (DAPA-MI [NCT04564742). In addition to SGLT2i, risk factor control approaches may be emphasized in those at high predicted risk. In a post-hoc analysis of the SPRINT trial, intensive BP lowering had a greater relative benefit in reducing HF events in those with the highest predicted risk of HF as estimated by the PCP-HF tool.²⁷

Limitations of our study must be noted. First, the Clalit population may not be representative of other populations globally. However, our sample represented a national, contemporary cohort of primary prevention adults outside of the US. Second, there is the potential for misclassification of incident HF, but we included initiation of prescription of commonly utilized therapies for HF to improve precision in outcome ascertainment. Complete follow-up for mortality allowed for robust ascertainment of vital status. Third, we were not able to adjudicate subtypes of HF in our prediction, but prevention of HF is not subtype specific and focuses on key risk factors shared by both HF with preserved and reduced ejection fraction.

In summary, we demonstrate that the PCP-HF model performs well for short-term risk prediction of HF in the next 5–10 years across a broad range of ages in a primary prevention population of 1.3 million adults in Israel.