Evaluation and Comparison of the PREVENT and Pooled Cohort Equations for 10‐Year Atherosclerotic Cardiovascular Risk Prediction

Abstract

Background

We compared the atherosclerotic cardiovascular disease (ASCVD) risk prediction performance of the American Heart Association's Predicting Risk of Cardiovascular Disease Events (PREVENT) Base and PREVENT Full equations (includes urine albumin/creatinine ratio, glycated hemoglobin, and social deprivation index) with the pooled cohort equations (PCEs).

Methods

We included adults, aged 40 to 75 years, with no history of ASCVD, diabetes, or statin use in 2009 from Kaiser Permanente Southern California and followed up through 2019. ASCVD was defined as myocardial infarction, fatal coronary heart disease, and fatal and nonfatal ischemic stroke. We compared model discrimination (Harrell C), mean calibration (estimated as the ratio of predicted/observed event rates), and calibration curve among the overall population and stratified by sex and race and ethnicity.

Results

Of the 559 241 adults (mean age, 54 years; 11% Asian, 11% non‐Hispanic Black, and 32% Hispanic), 10 695 developed an ASCVD event (median follow‐up, 10 years). Harrell C was 0.741 (95% CI, 0.736–0.745) for PREVENT Base, 0.743 (95% CI, 0.738–0.748) for PREVENT Full, and 0.741 (95% CI, 0.736–0.746) for the PCEs. Compared with the PCEs, both PREVENT equations improved Harrell C in men but not women, and in non‐Hispanic Black adults but not in other races and ethnicities. Both PREVENT equations were well calibrated (mean calibration, 0.85–1.36; calibration slope, 0.69–1.27), whereas the PCEs overestimated 10‐year ASCVD risk (mean calibration, 1.80–2.18; calibration slope, 0.32–0.45).

Conclusions

Compared with the PCEs, PREVENT Base and Full equations better predict absolute 10‐year ASCVD risk across sex and racial and ethnic groups in a contemporary US adult population.

Nonstandard Abbreviations and Acronyms

KPSC

Kaiser Permanente Southern California

PCE

pooled cohort equation

PREVENT

Predicting Risk of Cardiovascular Disease Events

SDI

social deprivation index

Clinical Perspective.

What Is New?

• In a population from a large integrated health care system, the Predicting Risk of Cardiovascular Disease Events (PREVENT) equations showed better absolute 10‐year atherosclerotic cardiovascular disease risk prediction than the pooled cohort equations.

• However, the PREVENT equations demonstrated only a slight improvement in discrimination compared with the pooled cohort equations mainly in men and non‐Hispanic Black adults.

• The PREVENT Full model, which adds albumin/creatinine ratio, glycated hemoglobin, and social deprivation index, showed better discrimination than the PREVENT Base model.

What Are the Clinical Implications?

• The PREVENT equations may change statin initiation decisions; only one‐third of individuals who may be eligible for statin therapy by the pooled cohort equations and current guideline recommendations (10‐year atherosclerotic cardiovascular disease risk ≥7.5%) may be eligible using PREVENT.

• Establishing appropriate risk thresholds for primary prevention statin therapy recommendations using PREVENT is necessary.

The pooled cohort equations (PCEs) were recommended for assessing individual atherosclerotic cardiovascular disease (ASCVD) risk and guiding preventive treatment decisions in US adults aged 40 to 75 years starting in 2013. ¹ The PCEs were subsequently shown to overestimate 10‐year ASCVD risk in some US cohorts ² , ³ , ⁴ , ⁵ and underestimate risk in certain racial and ethnic groups ⁶ and individuals with lower socioeconomic status. ¹ , ⁷ , ⁸ , ⁹ , ¹⁰ The American Heart Association recently published Predicting Risk of Cardiovascular Disease Events (PREVENT) equations, which are meant to replace the PCEs. PREVENT equations used more contemporary data, removed race as a predictor, and incorporated new predictors, including kidney biomarkers and social deprivation index (SDI). ¹¹ , ¹² In addition, PREVENT equations provide predictions for total cardiovascular disease (CVD), including ASCVD and heart failure, which were not available in the PCEs. ¹²

Although the PREVENT development cohort included a diverse population, <7% of validation sample were either Asian or Hispanic, and it is important to independently evaluate prediction performance in other populations outside of the development cohort. Using electronic health records from a large diverse US population, we aimed to assess ASCVD risk prediction performance of the 2 PREVENT equations: (1) PREVENT Base and (2) PREVENT Full (PREVENT Base plus urine albumin/creatinine ratio, glycated hemoglobin, and SDI, compared with the PCEs in the overall population and across sex and racial and ethnic groups).

METHODS

Study Setting

We conducted a retrospective cohort study in Kaiser Permanente Southern California (KPSC), a large integrated health care system that provides care to >4.6 million members across 9 counties in Southern California. Membership is approximately representative of the general population in Southern California. ¹³ , ¹⁴ All aspects of care provided to KPSC members, including claims data on care received outside of KPSC, are recorded in the electronic health records. The study protocol was approved by the KPSC Institutional Review Board. Waiver of consent was granted because of use of retrospective data. We followed the Strengthening the Reporting of Observational Studies in Epidemiology reporting guideline. ¹⁵ Deidentified data that support the findings of this study may be made available from the corresponding author pursuant to a written request with appropriate approvals and assurances to maintain data in accordance with security requirements and documented evidence of human subjects protections.

Study Population

Among adults, aged 40 to 75 years, who were actively enrolled on September 30, 2009, with continuous 12‐month membership (baseline), we identified a primary prevention cohort for ASCVD, for whom the current American Heart Association/American College of Cardiology cholesterol guidelines recommend assessment of 10‐year ASCVD risk using the PCEs to guide lipid‐lowering treatment strategies. ² We included adults without a history of ASCVD or diabetes, and with ≥1 lipid test result. We excluded adults with low‐density lipoprotein cholesterol <70 or ≥190 mg/dL at baseline or history of statin use. To align with the requirement for 10‐year ASCVD prediction using PCEs and PREVENT equations, individuals without available measurements of total cholesterol, high‐density lipoprotein cholesterol, systolic blood pressure, or estimated glomerular filtration rate at baseline were excluded. For tobacco use at baseline, we assumed all missing values (13%) indicated nonsmoking because most individuals reported being “never” smokers during follow‐up. We had 3.2% missing data for area‐level socioeconomic factors; however, these individuals were included in the analyses as PREVENT provides a coefficient for the missing SDI indicator. Only individuals with low‐density lipoprotein cholesterol of 70 to 189 mg/dL and estimated glomerular filtration rate of 15 to 140 mL/min per 1.73 m² were included in the final cohort and followed up for up to 10 years (Figure S1).

Outcome

The primary outcome of interest was incident ASCVD, defined as nonfatal myocardial infarction, fatal coronary heart disease, or fatal or nonfatal ischemic stroke over a 10‐year period. For the secondary analysis, the outcome of interest was incident total CVD, defined as a composite event of ASCVD, intracranial hemorrhage, and heart failure. Specifically, myocardial infarction was identified by principal hospital discharge diagnoses with International Classification of Diseases, Ninth Revision (ICD‐9), codes 410.x0 and 410.x1 and International Classification of Diseases, Tenth Revision (ICD‐10), code I21.x, ¹⁶ and ischemic stroke by principal hospital discharge diagnoses with ICD‐9 codes 433.x1, 434.x1, and 436.x, and ICD‐10 codes I63.x, G46.3, and G46.4. ¹⁷ , ¹⁸ Intracranial hemorrhage was identified by hospital discharge diagnosis with ICD‐9 codes 430, 431, and 432.x, and ICD‐10 codes I60.x, I61.x, and I62.x in any positions. ¹⁹ Heart failure was identified by hospital discharge diagnosis with ICD‐9 code 428.x, and ICD‐10 code I50.x in any positions. ¹¹ Deaths from coronary heart disease and stroke were identified using ICD‐10 codes I20.x to I25.x and I60.x to I69.x, from membership files, hospital records, and death files from state sources. Follow‐up time was censored at the earliest date of incident ASCVD event, death, membership disenrollment, or December 31, 2019.

Risk Factors

All variables for this study were collected from KPSC electronic health records. Baseline characteristics, including age, vital signs, laboratory results, smoking status, and medications dispensed, were determined before the index date. Specifically, sex (male and female), race, and ethnicity were based on a combination of self‐report and administrative data. Antihypertensive medication was ascertained through outpatient pharmacy dispensing records. Systolic blood pressure, high‐density lipoprotein cholesterol, total cholesterol, and estimated glomerular filtration rate were collected from outpatient laboratory records, and the measurements closest to baseline were kept. For calculations using the PREVENT Full equation, the closest measurements to the index date of urine albumin/creatinine ratio and glycated hemoglobin were used, but those without such measurements were kept and categorized as “unknown” for these 2 variables. Smoking status was self‐reported by members. SDI was derived by linking members' addresses via geocoding to 2008 to 2012 US Census tract data. ²⁰

Statistical Analysis

We described characteristics of the study population at baseline and the distribution of predicted risk by the 2 PREVENT equations and the PCEs. The predictive performance of the 3 models was compared in terms of discrimination and calibration for predicting 10‐year ASCVD events in primary analysis and 10‐year total CVD events in secondary analysis. We calculated the Harrell C‐index for assessing model discrimination. ²¹ Model calibration was assessed using mean calibration, calculated as the ratio of average predicted risk/the observed overall event rate, as well as graphical assessment of calibration slope and intercept with predicted risk on the x axis and the observed event rate on the y axis using linear regression. ²² Observed risk was estimated using cause‐specific cumulative incidence, accounting for non–coronary heart disease mortality as a competing event. We performed nonparametric bootstrapping 100 times to estimate the 95% CIs for these values. All analyses were conducted on the overall population, as well as stratified by sex, or race and ethnicity. All analyses were performed using SAS, version 9.4 (SAS Institute, Cary, NC).

RESULTS

We identified 559 241 adults (mean age, 53.5 years; 59.4% women; 42.9% non‐Hispanic White, 10.6% non‐Hispanic Black, 31.7% Hispanic, and 10.5% Asian and Pacific Islanders) (Table 1). The proportion of patients classified as intermediate or high ASCVD risk (10‐year risk ≥7.5%) was 22.5%, 7.3%, and 3.3% according to the PCEs, PREVENT Base, and PREVENT Full equations, respectively. The proportion of patients classified as intermediate or high ASCVD risk (10‐year risk ≥7.5%) was higher among men than women using all 3 equations (Table S1). According to the PCEs, PREVENT Base, and PREVENT Full equations, the proportion of patients classified as having intermediate or high ASCVD risk (10‐year risk ≥7.5%) was 32.6%, 8.5%, and 4.6% among non‐Hispanic Black adults, and 26.4%, 9.3%, and 3.7% among non‐Hispanic White adults, respectively.

Table 1.

Demographic and Clinical Characteristics of the Study Population

Characteristics*	Total (N=559 241)
Age, y	53.5±8.9
Age group, N (%)
40–59.9	416 261 (74.4)
60–75	142 980 (25.6)
Sex, N (%)
Female	332 458 (59.4)
Male	226 783 (40.6)
Race and ethnicity, N (%)
Non‐Hispanic White	239 636 (42.9)
Non‐Hispanic Black	59 262 (10.6)
Hispanic	177 080 (31.7)
Asian or Pacific Islander	58 943 (10.5)
Other or unknown†	24 310 (4.3)
Systolic blood pressure, mm Hg	122.7±14.0
Total cholesterol, mg/dL	202.8±31.4
Non‐HDL cholesterol, mg/dL	149.1±30.9
HDL cholesterol, mg/dL	53.6±14.6
Body mass index, kg/m2	28.8±6.1
Current smoking, N (%)	34 819 (6.2)
Antihypertensive treatment, N (%)	170 570 (30.5)
eGFR, mL/min per 1.73 m2	90.9±16.2
uACR, mg/g	29.2±186.1
Glycated hemoglobin, %	5.7±0.8
Social deprivation index decile (1–10)	5.4±2.9
PCE risk group, N (%)
Low risk (<5%)	368 261 (65.8)
Borderline risk (5% to <7.5%)	65 374 (11.7)
Intermediate risk (7.5% to <20%)	109 367 (19.6)
High risk (≥20%)	16 239 (2.9)
PREVENT Base equation risk group, N (%)
Low risk (<5%)	463 067 (82.8)
Borderline risk (5% to <7.5%)	55 595 (9.9)
Intermediate risk (7.5% to <20%)	40 537 (7.3)
High risk (≥20%)	42 (0.01)
PREVENT Full equation risk group, N (%)
Low risk (<5%)	502 832 (89.9)
Borderline risk (5% to <7.5%)	37 742 (6.8)
Intermediate risk (7.5% to <20%)	18 502 (3.3)
High risk (≥20%)	165 (0.03)

eGFR indicates estimated glomerular filtration rate; HDL, high‐density lipoprotein; PCE, pooled cohort equation; PREVENT, Predicting Risk of Cardiovascular Disease Events; and uACR, urine albumin/creatinine ratio.

^*Described as mean±SD unless otherwise specified.

^† Includes Native American or Alaska Native, multiple, or other races and ethnicities, as well as those unreported.

A total of 10 695 adults experienced an incident ASCVD event during follow‐up (mean, 7.4 years; median, 10 years). Among the overall population, the Harrell C‐index was 0.741 (95% CI, 0.736–0.746) for the PCEs, 0.741 (95% CI, 0.736–0.745) for the PREVENT Base, and 0.743 (95% CI, 0.738–0.748) for the PREVENT Full equations (Table 2). When stratified by sex, the C‐index difference was greater among male adults than female adults, increasing from 0.708 (PCE) to 0.712 (PREVENT Full), with a ΔC‐index of 0.0049 (95% CI, 0.0026–0.0073). When stratified by race and ethnicity, the biggest C‐index difference was among non‐Hispanic Black adults, increasing from 0.705 (PCE) to 0.717 (PREVENT Full), with a ΔC‐index of 0.012 (95% CI, 0.008–0.018).

Table 2.

Harrell C‐Index and Mean Calibration Comparison Among the PCEs, PREVENT Base, and PREVENT Full Equations

Variable	Overall	Sex		Race and ethnicity
		Female	Male	Non‐Hispanic White	Non‐Hispanic Black	Hispanic	Asian or Pacific Islander
No. of adults	559 241	332 458	226 783	239 636	59 262	177 090	58 943
No. of ASCVD events	10 695	4676	6019	5621	1444	2513	952
PCEs
C‐index (95% CI)	0.741 (0.736 to 0.746)	0.739 (0.731 to 0.746)	0.708 (0.701 to 0.713)	0.738 (0.731 to 0.743)	0.705 (0.692 to 0.720)	0.742 (0.734 to 0.754)	0.735 (0.720 to 0.746)
Mean calibration (95% CI)	1.99 (1.95 to 2.03)	1.80 (1.74 to 1.85)	2.12 (2.07 to 2.17)	1.91 (1.87 to 1.96)	2.10 (2.00 to 2.21)	2.05 (1.97 to 2.13)	2.18 (2.02 to 2.32)
PREVENT Base equation
C‐index (95% CI)	0.741 (0.736 to 0.745)	0.738 (0.730 to 0.745)	0.711 (0.703 to 0.716)	0.737 (0.729 to 0.742)	0.713 (0.698 to 0.727)	0.743 (0.732 to 0.754)	0.733 (0.719 to 0.744)
ΔC‐index (95% CI) [PREVENT Base–the PCEs]	−0.0005 (−0.0018 to 0.0007)	−0.0007 (−0.0022 to 0.0012)	0.0030 (0.0016 to 0.0045)	−0.0013 (−0.0032 to −0.0001)	0.0078 (0.0027 to 0.0131)	0.0001 (−0.0020 to 0.0023)	−0.0023 (−0.0062 to 0.0018)
Mean calibration (95% CI)	1.19 (1.16 to 1.20)	1.28 (1.25 to 1.32)	1.09 (1.06 to 1.11)	1.14 (1.11 to 1.16)	1.03 (0.98 to 1.09)	1.28 (1.23 to 1.33)	1.36 (1.26 to 1.45)
PREVENT Full equation
C‐index (95% CI)	0.743 (0.738 to 0.748)	0.741 (0.734 to 0.748)	0.712 (0.706 to 0.718)	0.739 (0.732 to 0.744)	0.717 (0.702 to 0.731)	0.745 (0.736 to 0.757)	0.733 (0.719 to 0.743)
ΔC‐index (95% CI) [PREVENT Full–the PCEs]	0.0017 (0.0002 to 0.0033)	0.0018 (−0.0005 to 0.0045)	0.0049 (0.0026 to 0.0073)	0.0008 (−0.0014 to 0.0026)	0.0122 (0.0077 to 0.0181)	0.0026 (−0.0002 to 0.0061)	−0.0025 (−0.0078 to 0.0026)
Mean calibration (95% CI)	0.91 (0.89 to 0.92)	0.97 (0.94 to 1.00)	0.85 (0.83 to 0.86)	0.84 (0.82 to 0.86)	0.83 (0.79 to 0.87)	1.03 (1.00 to 1.07)	1.04 (0.97 to 1.11)

ASCVD indicates atherosclerotic cardiovascular disease; PCE, pooled cohort equation; and PREVENT, Predicting Risk of Cardiovascular Disease Events.

The PCEs predicted nearly twice as many ASCVD events as were observed (mean calibration, 1.99 [95% CI, 1.95–2.03]) in the overall population, with the largest overestimation in men (mean calibration, 2.12 [95% CI, 2.07–2.17]) and Asian and Pacific Islanders (mean calibration, 2.18 [95% CI, 2.02–2.32]) (Table 2). The overestimation was reduced to 19% overall using the PREVENT Base equation, whereas the risk was underestimated by 9% using the PREVENT Full equation. The calibration slope changed from 0.40 for the PCEs to 0.85 for the PREVENT Base equation, and to 1.09 for the PREVENT Full equation while all intercepts were close to 0 (Figure 1). These findings were consistent across different sexes, and racial and ethnic subgroups. The calibration slopes ranged from 0.32 to 0.42 for the PCEs, from 0.69 to 0.90 for the PREVENT Base equation, and from 0.87 to 1.19 for the PREVENT Full equation (Figures 2 and 3).

Figure 1. Calibration plots applying the PCE, the PREVENT Base equation, and the PREVENT Full equation.

ASCVD indicates atherosclerotic cardiovascular disease; PCE, pooled cohort equation; and PREVENT, Predicting Risk of Cardiovascular Disease Events.

Figure 2. Calibration plots applying the PCE, the PREVENT Base equation, and the PREVENT Full equation stratified by sex.

ASCVD indicates atherosclerotic cardiovascular disease; F, female; M, male; PCE pooled cohort equation; and PREVENT, Predicting Risk of Cardiovascular Disease Events.

Figure 3. Calibration plots applying the PCE, the PREVENT Base equation, and the PREVENT Full equation stratified by race and ethnicity.

ASCVD indicates atherosclerotic cardiovascular disease; PCE, pooled cohort equation; and PREVENT, Predicting Risk of Cardiovascular Disease Events.

In secondary analysis, 20 023 adults developed an incident total CVD event during follow‐up. The Harrell C‐index for predicting total CVD was 0.742 (95% CI, 0.739–0.745) for the PCEs, 0.747 (95% CI, 0.744–0.750) for the PREVENT Base, and 0.750 (95% CI, 0.747–0.753) for the PREVENT Full equations (Table S2). The calibration slope changed from 0.77 for the PCEs to 0.97 for the PREVENT Base equation (Figure S2).

DISCUSSION

In this study of nearly 560 000 adults from a large US health care system, the PREVENT equations improved calibration in the prediction of 10‐year ASCVD risk compared with the older PCEs across sex, and racial and ethnic groups. However, the PREVENT equations demonstrated only a slight improvement in discrimination when compared with the PCEs. These findings suggest that the PREVENT equations are equivalent to the PCEs in ranking individuals from low to high risk and more accurately predict absolute 10‐year ASCVD risk than the PCEs.

Since 2013, the PCEs have been recommended for assessing ASCVD risk, becoming key in statin treatment decisions for primary ASCVD prevention in the United States. However, subsequent studies showed that the PCEs often overestimated ASCVD risk in contemporary US cohorts, possibly because they were developed using relatively older data. ²³ , ²⁴ Furthermore, the PCEs tend to underestimate risk in certain subgroups, such as South Asians. ¹ , ⁹ , ¹⁰ Our study demonstrated that PREVENT equations substantially improved calibration, providing more accurate estimates of 10‐year absolute ASCVD risk. The accuracy of ASCVD risk predication could have substantial impacts on clinical care; we estimated that only one‐third of individuals who might be eligible for statin therapy by the PCEs might be eligible using the PREVENT Base equations. ⁹ Consistent with our analysis, recent National Health and Nutrition Examination Survey analyses estimate that 14 to 17 million US adults who may be eligible for statins based on the PCEs would no longer be eligible based on the PREVENT equations. ²⁵ , ²⁶ Before implementing the PREVENT equations more broadly in clinical practice, future studies should determine the most effective and cost‐effective risk thresholds for primary prevention statin therapy recommendations using this new tool.

Compared with the PCEs, the PREVENT equations removed race as a predictor, aiming to avoid perpetuating inequities or racial disparities in health care. The PREVENT equations also included kidney biomarkers to inform a holistic approach to cardiovascular‐kidney‐metabolic health. The recent National Health and Nutrition Examination Survey analysis showed that the PREVENT Base equations had strong discrimination for CVD mortality, with a C‐index of 0.890, comparable to the PCE's C‐index of 0.880. ²⁷ Consistent with these findings, the current study further extends previous reports by using ASCVD and total CVD events as outcomes, and found that the PREVENT Base equations had good discrimination properties but did not significantly improve model discrimination compared with the PCEs in the overall population. In subgroup analyses stratified by sex and race and ethnicity, we found that the C‐index improved in male individuals and in non‐Hispanic Black individuals but not in other subgroups. Additionally, we validated the PREVENT Full equations, which included metabolic risk factors as well as SDI, as a new predictor to address potential impact of social determinants to cardiovascular risk. The current study found a modest improvement in C‐index in the overall population when using the PREVENT Full equations compared with the PCEs, with the largest improvement seen in non‐Hispanic Black adults. These results suggest that PREVENT equations' inclusion of additional risk factors better identifies high‐risk individuals within specific subgroups. These findings were consistent when we compared the prediction performance for total CVD, including heart failure. The significant improvement in discrimination using both the PREVENT Base and Full equations compared with the PCEs reinforces the importance of considering SDI and new biomarkers whenever possible.

This study has the strength of externally validating 2 PREVENT equations using a large, diverse, contemporary population from a US integrated healthcare system. However, this study also has several limitations. The study cohort, limited to nondiabetics, was healthier than the populations used to develop the PREVENT equations. Despite this, we focused on the primary prevention cohort, the major population of interest for cardiovascular risk assessment, to determine statin initiation. We also examined how population at intermediate or high ASCVD risk (10‐year risk ≥7.5%, recommended for statin initiation) changed when applying the PREVENT equations instead of the PCEs. Although the PREVENT equations cover a broader age range (30–79 versus 40–79 years for the PCEs), we focused on adults aged 40 to 75 years, consistent with the 2018 American Heart Association/American College of Cardiology cholesterol guidelines ⁹ and to facilitate a direct comparison between the PREVENT and the PCEs. Unlike the PCEs, PREVENT used a competing risk approach, which may explain PREVENT's superior calibration compared with the PCEs. Direct comparisons of mean calibration that account for competing risk are likely to favor PREVENT. Although there are separate equations for predicting 10‐year total CVD, the PCEs focus on predicting 10‐year ASCVD. Therefore, the better performance of PREVENT versus PCEs for total CVD may be expected.

CONCLUSIONS

Compared with the PCEs, the PREVENT equations improved calibration accuracy in predicting 10‐year ASCVD risk across sex and all race and ethnicity groups. Discrimination did not improve substantively, although modest improvement in discrimination was also observed in male and non‐Hispanic Black adults. Overall, these findings suggest PREVENT equations provide a more accurate assessment of 10‐year ASCVD risk than the PCEs in a contemporary US adult population.

Sources of Funding

This work was supported by National Institutes of Health grants R01HL155081 and R01HL168379 (Drs Zhang and An). The funder/sponsor had no role in the design and conduct of the study; the collection, management, analysis, and interpretation of the data; the preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication.

Disclosures

Dr An receives research support from Bayer unrelated to this work. S. K. Choi received research support from Bayer unrelated to this work. Dr Colantonio received research support from Amgen unrelated to this work. Dr Reynolds receives research support from Merck Sharp & Dohme LLC unrelated to this work. The remaining authors have no disclosures to report.

Supporting information

Tables S1–S2

Figures S1–S2