Abstract
Importance
The American College of Cardiology/American Heart Association Pooled Cohort risk equations were developed to estimate atherosclerotic cardiovascular disease (ASCVD) risk and guide statin initiation.
Objective
Assess calibration and discrimination of the Pooled Cohort risk equations in a contemporary US population.
Design, Setting, and Participants
Adults 45-79 years enrolled in the REasons for Geographic And Racial Differences in Stroke study between January 2003 and October 2007 and followed through December 2010. We studied participants for whom ASCVD risk may trigger a discussion of statin initiation (those without clinical ASCVD or diabetes, LDL-C between 70-189 mg/dL, not taking statins; n=10,997).
Main Outcomes and Measures
Predicted risk and observed adjudicated ASCVD incidence (non-fatal myocardial infarction, coronary heart disease [CHD] death, non-fatal or fatal stroke) at 5 years as REGARDS participants have not been followed 10 years. Additional analyses, limited to Medicare beneficiaries (n=3,333), added ASCVD events identified in claims data.
Results
There were 338 adjudicated events (192 CHD events, 146 strokes). The observed and predicted 5-year ASCVD incidence per 1,000 person-years for participants with 10-year predicted ASCVD risk <5% was 1.9 (95%CI: 1.3 – 2.7) and 1.9, risk 5% to <7.5% was 4.8 (95%CI: 3.4 – 6.7) and 4.8, risk 7.5% to <10% was 6.1 (95%CI: 4.4 – 8.6) and 6.9, and risk ≥10% was 12.0 (95%CI: 10.6 – 13.6) and 15.1 (Hosmer-Lemeshow χ2 19.9 p-value=0.01). The c-index was 0.72 (95%CI: 0.70–0.75). There were 234 ASCVD events (120 CHD events, 114 strokes) among Medicare-linked participants and the observed and predicted 5-year ASCVD incidence per 1,000 person-years for participants with predicted risk <7.5% was 5.3 (95% CI: 2.8 – 10.1) and 4.0, risk 7.5% to <10% was 7.9 (95% CI: 4.6 – 13.5) and 6.4, and risk ≥10% was 17.4 (95% CI: 15.3–19.8) and 16.4 (Hosmer-Lemeshow χ2 5.4 p-value=0.71). The c-index was 0.67 (95%CI: 0.64 – 0.71)
Conclusions and Relevance
In this cohort of US adults for whom statin initiation is considered based on the ACC/AHA Pooled Cohort risk equations, observed and predicted 5-year ASCVD risks were similar, indicating that these risk equations were well calibrated in the population for which they were designed to be used.
The American College of Cardiology (ACC) and the American Heart Association (AHA) recently published the 2013 Guideline on the Assessment of Cardiovascular Risk.1 As part of this guideline, a working group developed new equations for the prediction of 10-year atherosclerotic cardiovascular disease (ASCVD) risk, the “Pooled Cohort risk equations.” These equations were derived in several population-based cohorts that included large samples of blacks and whites and were aimed at estimating 10-year risk for non-fatal myocardial infarction (MI), coronary heart disease (CHD) death and non-fatal or fatal stroke (“hard” ASCVD events). The 2013 ACC/AHA cholesterol treatment guidelines recommend using the Pooled Cohort risk equations to estimate ASCVD risk and help guide the decision to initiate statin therapy for primary prevention in adults without clinical ASCVD or diabetes, and with a low density lipoprotein cholesterol (LDL-C) between 70 and 189 mg/dL.2
Many of the studies used to develop the new ASCVD risk equations recruited and followed participants before 2000, and there have been marked declines in CHD and stroke incidence over the past two decades.3,4 In light of the need for new prediction models to undergo external validation, we evaluated the calibration and discrimination of these equations for predicting ASCVD risk in a contemporary population-based cohort, the REasons for Geographic And Racial Differences in Stroke (REGARDS) study. Calibration provides assessment of whether a risk prediction model accurately estimates the absolute observed risk level. Discrimination provides assessment of whether a risk prediction model accurately rank orders individuals (i.e., are individuals with higher predicted risk more likely to have events). In addition to analyses of the overall REGARDS population without ASCVD at baseline, we conducted analyses restricted to the population for which the Pooled Cohorts risk equations are intended to inform discussions about initiating statins (participants without clinical ASCVD or diabetes, with LDL-C between 70 - 189 mg/dL, and not taking statins). To maximize event surveillance, we also conducted analyses including ASCVD events identified in Medicare claims data.
METHODS
Study Population
The REGARDS study was designed to investigate reasons underlying the higher stroke mortality in blacks compared to whites and residents of the Southeastern region versus other regions of the US.5 CHD events are being identified and adjudicated in an ancillary study.6 REGARDS participants were selected from a commercially available nationwide list purchased through Genesys Inc. Criteria for inclusion in the sample include having a name, telephone number and address in the Genesys database. A letter and study brochure were sent to potential participants and telephone contact was attempted approximately 2 weeks later. A total of 30,239 adults ≥45 years of age from all 48 contiguous US states and the District of Columbia were enrolled between January 2003 and October 2007. The inclusion criteria for the current analyses were chosen to match those used in the development of the Pooled Cohort risk equations. We restricted the analysis to participants 45 to 79 years of age and without a history of CHD, stroke, heart failure or atrial fibrillation at baseline. A history of heart failure was not ascertained during the REGARDS baseline study visit. As a proxy for heart failure, we excluded participants taking digoxin, as determined through the baseline pill bottle review. We further excluded participants with missing data on components of the Pooled Cohort risk equations. The REGARDS study protocol was approved by the Institutional Review Boards governing research in human subjects at the participating centers and all participants provided written informed consent.
Data Collected at Baseline
Computer-assisted telephone interviews were administered by trained staff and used to collect information on participants’ age, race, sex, smoking status, prior diagnosed co-morbid conditions, and use of antihypertensive and antidiabetes medications. Race was self-reported. Following the interview, trained health professionals conducted in-home examinations that included blood pressure measurements, an electrocardiogram, collection of a blood sample, plus a review of prescription and over the counter medications used during the 2 week period prior to the study visit. Blood pressure was measured two times following a standardized protocol and averaged for analysis. Total and high-density lipoprotein (HDL) cholesterol, serum triglycerides, and glucose were measured using the Ortho Vitros Clinical Chemistry System 950IRC instrument (Johnson & Johnson Clinical Diagnostics, New Brunswick, NJ). For participants who fasted prior to their study visit with serum triglycerides <400 mg/dL, LDL-C was calculated using the Friedewald equation.7 For others, non-HDL-C was calculated. Diabetes was defined as glucose ≥126 mg/dL for participants who had fasted prior to their blood draw, glucose ≥200 mg/dL for those non-fasting participants, or self-report of a prior diagnosis of diabetes with current use of insulin or oral hypoglycemic medications. Through the pill bottle review, we identified the use of statins at baseline. Digoxin use was identified and served as a marker for prevalent heart failure.8 Atrial fibrillation was defined based on the study electrocardiogram or self-report. History of CHD at baseline was defined by self-report of MI or revascularization procedure or evidence on the study ECG of MI. History of stroke was defined by self-report.
Data Collected During Follow-up
Living participants or their proxies were contacted every 6 months via telephone to assess new-onset stroke and CHD events. Consistent with the definition used to derive the Pooled Cohort risk equations, the outcome for our primary analyses was defined as the first ASCVD event, which included a non-fatal or fatal stroke or a non-fatal MI or CHD death. Adjudication of stroke and CHD events are currently available through December 31, 2010.
Stroke
Stroke events were identified via self-report or proxy report of stroke/transient ischemic attack or stroke symptoms.9,10 For reported events, hospital charts and physician office records were retrieved for adjudication. Stroke events were confirmed by a panel of experts according to the World Health Organization (WHO) definition.11 Events not meeting the WHO definition but characterized by symptoms lasting <24 hours with neuroimaging consistent with acute infarct or hemorrhage were classified as clinical strokes. The present analysis included WHO–defined as well as clinical stroke.12
Coronary Heart Disease
CHD events (nonfatal MI or CHD death), were detected during the follow-up telephone interviews. Medical records were retrieved and events were adjudicated by trained clinicians following published guidelines.13,14 Records were examined for the presence of signs or symptoms suggestive of ischemia, a rising and/or falling pattern in cardiac troponin or creatine phosphokinase-MB over 6 or more hours with a peak value greater than or equal to twice the upper limit of normal, and electrocardiogram changes consistent with ischemia or MI, guided by the Minnesota code and classified as evolving diagnostic, positive, nonspecific or not consistent with ischemia.15,16 For deceased participants, interviews with next-of-kin or proxies, medical records from the last year of life, death certificates and autopsy reports were reviewed to determine if stroke or CHD were main underlying causes of death. Only definite or probable events were included.
Events Identified in Medicare Claims
The cohorts used to develop the Pooled Cohort risk equations had surveillance components (e.g., review of hospital discharges and obituaries in local newspapers) to detect ASCVD events not reported by participants. REGARDS did not have this active surveillance and, therefore, some ASCVD events were possibly missed. To address this limitation, we used linked Medicare claims data to identify ASCVD events not detected through routine cohort follow-up. Medicare provides health insurance to adults age ≥ 65 years, those with end-stage renal disease, or disability. REGARDS participants were linked to Medicare enrollment and claims data from 1999 through 2010 by social security number, sex and date of birth. MIs were defined by an overnight hospitalization in an acute care facility with a discharge diagnosis ICD9 code of 410.×× (except 410.×2 which indicates a subsequent episode of care) in any position and stroke events were defined by a discharge diagnosis ICD9 code of 430.××, 431.××, 433.××, 434.×× or 436.× in the primary position. These definitions have positive predictive values >90%.17-19
Statistical Analysis
Analyses were performed for the overall population and, separately, in the subgroup for whom the 2013 cholesterol treatment guidelines recommend using 10-year ASCVD risk from the Pooled Cohort risk equations to inform discussions about initiating statins for primary prevention of ASCVD (i.e., participants without ASCVD or diabetes, with an LDL-C between 70 and 189 mg/dL or, if LDL-C was not available (n=1,255), non-HDL-C between 100 and 219 mg/dL; and not already taking statins).2 We calculated each participant's predicted 10-year ASCVD risk using the Pooled Cohort risk equations. Participants were categorized into four groups according to their 10-year predicted ASCVD risk: <5%, 5% to <7.5%, 7.5% to <10%, and ≥10%. Participant characteristics, including age, race, sex, current smoking, diabetes, systolic blood pressure, use of antihypertensive medication, total cholesterol, HDL-C, LDL-C and use of statins, were calculated within each 10-year predicted ASCVD risk group.
Because REGARDS has not yet completed 10 years of follow-up, we calculated observed and predicted ASCVD incidence rates at 5 years within the four ASCVD risk groups described above. Observed rates were calculated using adjudicated events. Participants were censored at the time the first of the following events occurred: (1) an ASCVD event; (2) death; (3) their last REGARDS follow-up interview; (4) 5 years of follow-up or (5) December 31, 2010. Overall, 53.6% of REGARDS participants were censored at 5 years of follow-up free of ASCVD events. Predicted ASCVD incidence at 5 years of follow-up was calculated using the Pooled Cohort risk equations and (S0(t)) at 5 years (Supplemental Table 1). The observed number of ASCVD events at 5 years was adjusted for variable follow-up time using the Kaplan-Meier estimate.20 The predicted number of events was calculated based on the mean predicted ASCVD incidence at 5 years. Next, participants were grouped into deciles of predicted ASCVD risk. The calibration of the Pooled Cohort risk equations was determined using the observed and predicted number of ASCVD events at 5-years of follow-up in each decile and a modified Hosmer-Lemeshow chi-square statistic.21 A chi-square >20 or p-value <0.05 indicates poor calibration. We calculated the c-index to estimate discrimination of the ASCVD risk equation.22,23 Although no thresholds exist, a c-index between 0.70 and 0.80 is considered moderate to good and ≥0.80 is considered excellent.24 The above analyses were performed for the overall population, and separately for men and women and for whites and blacks. Analyses were also repeated for participants residing in the stroke belt (North Carolina, South Carolina, Georgia, Alabama, Mississippi, Louisiana, Tennessee and Arkansas) and non-belt regions of the continental US. Analyses were repeated limited to participants age ≥ 65 years at baseline with Medicare Part A coverage and including adjudicated ASCVD events plus events identified in the Medicare claims. Two sided p-values<0.05 were considered statistically significant. Analyses were conducted using SAS Version 9.3 (SAS Institute, Cary, NC).
RESULTS
Study Population
We restricted the analyses to REGARDS participants 45-79 years of age (n=28,044, Supplemental Figure 1). After excluding 7,326 participants with a history of CHD, stroke, atrial fibrillation, or heart failure, 1,929 missing data on Pooled Cohort risk equations components, and 291 participants without follow-up data, there were 18,498 participants available for analysis. After further excluding participants with diabetes, LDL-C <70 mg dL or ≥190 mg/dL, or taking statins at baseline, there were 10,997 participants available in the subgroup for whom the 2013 cholesterol treatment guidelines recommend consideration of statin initiation based on their estimated ASCVD risk.
At baseline, 25.0% of REGARDS participants were taking statins. Levels of Pooled Cohort risk equations components are provided by 10-year predicted ASCVD risk stratum in the top panel of Table 1, with levels for those considered for statin initiation based on ASCVD risk provided in the bottom panel of Table 1. Additional characteristics of REGARDS study participants are presented in Supplemental Table 2. In both populations, participants with higher 10-year predicted ASCVD risk were older and a higher percentage was black, men, current smokers and taking antihypertensive medication. Mean systolic blood pressure and LDL-C were higher and mean HDL-C was lower for those with higher 10-year predicted ASCVD risk. In the overall population, a higher prevalence of diabetes and statin use at baseline was present and total cholesterol was lower at higher 10-year predicted ASCVD risk. Total cholesterol was similar across ASCVD risk groups among participants considered for statin initiation based on their ASCVD risk.
Table 1.
Baseline characteristics of the overall REGARDS population included in this analysis (n=18,498, top panel) and those without diabetes, with LDL-C between 70 and 189 mg/dL and not taking statins (n=10,997, bottom panel) according to 10-year ASCVD predicted risk.
| Overall population 10-year predicted ASCVD risk | p-trend | ||||
|---|---|---|---|---|---|
| <5% | 5% to <7.5% | 7.5% to <10% | ≥10% | ||
| Participants, n (%) | 4,579 (24.8) | 2,343 (12.7) | 2,112 (11.4) | 9,464 (51.2) | - |
| Age (years), mean (SD) | 55.1 (5.4) | 59.7 (5.6) | 61.7 (5.9) | 67.3 (7.0) | <0.001 |
| Blacks, n (%) | 1,288 (28.1) | 948 (40.5) | 931 (44.1) | 4,538 (48.0) | <0.001 |
| Men, n (%) | 613 (13.4) | 837 (35.7) | 913 (43.2) | 5,361 (56.7) | <0.001 |
| Current smoking, n (%) | 331 (7.2) | 287 (12.3) | 301 (14.3) | 1,751 (18.5) | <0.001 |
| Diabetes, n (%) | 140 (3.1) | 157 (6.7) | 208 (9.9) | 2,791 (29.5) | <0.001 |
| SBP (mmHg), mean (SD) | 115.9 (12.5) | 121.7 (12.2) | 124.9 (12.9) | 132.6 (16.0) | <0.001 |
| Antihypertensive medication, n (%) | 1,102 (24.1) | 875 (37.4) | 962 (45.6) | 5,821 (61.5) | <0.001 |
| Total cholesterol (mg/dL), mean (SD) | 197.2 (36.2) | 197.0 (38.7) | 196.3 (38.5) | 194.6 (40.1) | <0.001 |
| HDL cholesterol (mg/dL), mean (SD) | 59.5 (16.4) | 53.7 (15.4) | 51.9 (15.1) | 49.7 (15.5) | <0.001 |
| LDL cholesterol (mg/dL),† mean (SD) | 115.9 (32.2) | 119.3 (33.4) | 119.7 (34.4) | 118.4 (35.0) | 0.008 |
| Statin use, n (%) | 749 (16.4) | 550 (23.5) | 535 (25.3) | 2,793 (29.5) | <0.001 |
| Participants without diabetes, LDL-C 70 to 189 mg/dL,‡ not taking statins 10-year predicted ASCVD risk | |||||
|---|---|---|---|---|---|
| <5% | 5% to <7.5% | 7.5% to <10% | ≥10% | ||
| Participants, n (%) | 3,453 (31.4) | 1,578 (14.4) | 1,332 (12.1) | 4,634 (42.1) | - |
| Age (years), mean (SD) | 54.8 (5.4) | 59.5 (5.6) | 62.0 (5.8) | 67.9 (6.8) | <0.001 |
| Blacks, n (%) | 980 (28.4) | 661 (41.9) | 593 (44.5) | 1,898 (41.0) | <0.001 |
| Men, n (%) | 455 (13.2) | 613 (38.9) | 617 (46.3) | 2,795 (60.3) | <0.001 |
| Current smoking, n (%) | 255 (7.4) | 214 (13.6) | 210 (15.8) | 947 (20.4) | <0.001 |
| SBP (mmHg), mean (SD) | 115.7 (12.5) | 122.0 (12.2) | 125.2 (13.3) | 132.5 (16.2) | <0.001 |
| Antihypertensive medication, n (%) | 692 (20.0) | 520 (33.0) | 508 (38.1) | 2,414 (52.1) | <0.001 |
| Total cholesterol (mg/dL), mean (SD) | 202.2 (30.8) | 202.6 (31.5) | 203.2 (31.4) | 202.7 (30.9) | 0.52 |
| HDL cholesterol (mg/dL), mean (SD) | 59.8 (16.4) | 53.9 (15.5) | 51.9 (15.1) | 50.6 (16.2) | <0.001 |
| LDL cholesterol (mg/dL),† mean (SD) | 121.0 (26.9) | 125.5 (26.6) | 126.6 (26.7) | 126.4 (26.8) | <0.001 |
ASCVD: atherosclerotic cardiovascular disease; HDL: high density lipoprotein; LDL-C: low density lipoprotein; REGARDS: REasons for Geographic And Racial Differences in Stroke; SBP: systolic blood pressure; SD: standard deviation.
LDL-C values were available for participants who fasted prior to their REGARDS study visit.
Non-HDL cholesterol between 100 and 219 for the 1,255 who had not fasted prior to their REGARDS study visit.
Pooled Cohorts Risk Equations in the Overall Population
There were 674 adjudicated ASCVD events (382 CHD and 292 strokes) over 79,321 person-years of follow-up in the overall cohort. For participants with a 10-year predicted ASCVD risk <5%, observed and predicted 5-year incidence rates were 2.2 (95%CI: 1.7 – 3.0) and 2.0 per 1,000 person-years (Table 2). In higher 10-year predicted ASCVD risk strata, 5-year observed risk was lower than predicted risk. For example, for those with predicted risk ≥10%, the observed and predicted risk were 12.6 (95% CI: 12.0 – 14.2) and 17.8, respectively. Calibration for the overall population was poor (Hosmer-Lemeshow χ2 84.2, p-value<0.001; Table 2, Figure 1, and Supplemental Table 3). The Pooled Cohort risk equations over-estimated risk for men and women and whites and blacks. The c-index for the overall population was 0.71 (95%CI: 0.69 – 0.72) and was higher in women compared to men and whites compared to blacks.
Table 2.
Observed and predicted incidence rates of atherosclerotic cardiovascular disease in the REasons for Geographic And Racial Differences in Stroke (REGARDS) study by 10-year predicted risk.
| Events / person-years | Events in 5-years | 5-year incidence rate† | Calibration | Discrimination | |||
|---|---|---|---|---|---|---|---|
| Observed KM-adjusted | Predicted | Observed KM-adjusted (95% CI) | Predicted | Chi-squared (p-value) | C-index (95% CI) | ||
| All participants | |||||||
| 10-year predicted risk | 84.2 (<0.001) | 0.71 (0.69-0.72) | |||||
| <5% | 44 / 19,631 | 51.4 | 44.7 | 2.2 (1.7-3.0) | 2.0 | ||
| 5% to <7.5% | 42 / 10,224 | 49.0 | 56.0 | 4.2 (3.1-5.7) | 4.8 | ||
| 7.5% to <10% | 48 / 9,202 | 57.1 | 72.2 | 5.0 (4.1-7.2) | 6.8 | ||
| ≥10% | 540 / 40,264 | 611.8 | 840.6 | 12.6 (12.0-14.2) | 17.8 | ||
| Women | 27.9 (<0.001) | 0.74 (0.71-0.76) | |||||
| 10-year predicted risk | |||||||
| <5% | 32 / 17,059 | 38.6 | 35.5 | 1.9 (1.4-2.8) | 1.8 | ||
| 5% to <7.5% | 23 / 6,586 | 27.0 | 33.6 | 3.6 (2.4-5.4) | 4.5 | ||
| 7.5% to <10% | 28 / 5,199 | 34.3 | 38.1 | 5.7 (3.9-8.3) | 6.4 | ||
| ≥10% | 215 / 17,247 | 246.7 | 319.2 | 12.0 (10.6-13.8) | 15.6 | ||
| Men | 62.8 (<0.001) | 0.65 (0.62-0.68) | |||||
| 10-year predicted risk | |||||||
| <5% | 12 / 2,572 | 12.7 | 9.1 | 4.2 (2.4-7.3) | 3.0 | ||
| 5% to <7.5% | 19 / 3,638 | 22.0 | 22.4 | 5.3 (3.4-8.2) | 5.3 | ||
| 7.5% to <10% | 20 / 4,004 | 23.1 | 34.1 | 5.1 (3.3-7.8) | 7.5 | ||
| ≥10% | 325 / 23,017 | 365.6 | 521.4 | 13.7 (12.3-15.3) | 19.5 | ||
| Black | 41.9 (<0.001) | 0.68 (0.65-0.71) | |||||
| 10-year predicted risk | |||||||
| <5% | 13 / 5,349 | 15.7 | 14.3 | 2.4 (1.4-4.2) | 2.2 | ||
| 5% to <7.5% | 16 / 4,027 | 19.4 | 23.6 | 4.1 (2.5-6.7) | 5.0 | ||
| 7.5% to <10% | 23 / 3,976 | 28.4 | 32.2 | 6.1 (4.0-9.2) | 6.9 | ||
| ≥10% | 256 / 18,968 | 293.9 | 404.0 | 13.0 (11.5-14.7) | 17.8 | ||
| White | 44.4 (<0.001) | 0.72 (0.70-0.75) | |||||
| 10-year predicted risk | |||||||
| <5% | 31 / 14,282 | 35.7 | 30.3 | 2.2 (1.5-3.1) | 1.8 | ||
| 5% to <7.5% | 26 / 6,197 | 29.6 | 32.3 | 4.2 (2.9-6.2) | 4.6 | ||
| 7.5% to <10% | 25 / 5,227 | 29.0 | 40.0 | 4.9 (3.3-7.3) | 6.8 | ||
| ≥10% | 284 / 21,296 | 319.8 | 437.6 | 13.0 (11.6-14.6) | 17.7 | ||
Predicted risk determined using the Pooled Cohort risk equations.
95% CI: 95% confidence interval. KM: Kaplan-Meier.
Per 1,000 person-years.
Figure 1.
Observed and predicted atherosclerotic cardiovascular disease risk among REGARDS participants. Top panel contains the overall REGARDS population and the bottom panel includes REGARDS participants ≥ 65 years of age with Medicare coverage including events identified through Medicare claims.
Suppressed – Medicare data are not presented in these cells due to a small sample size.
Predicted risk determined using the Pooled Cohort equations.
LDL-C: low density lipoprotein cholesterol; REGARDS: REasons for Geographic And Racial Differences in Stroke.
† The range of predicted risk for each decile is provided in Supplemental Table 3.
‡ The range of predicted risk for each decile in the REGARDS population with Medicare insurance coverage is provided in Supplemental Table 6.
Pooled Cohorts Risk Equations in the Population Considered for Statin Initiation Based on Estimated ASCVD Risk
Among the subgroup for whom statin treatment should be considered based on ASCVD risk, there were 338 adjudicated events (192 CHD and 146 strokes) over 47,481 person-years. Calibration was better in this clinically relevant population, with less overestimation of risk (Hosmer-Lemeshow χ2 19.9, p-value=0.01; Table 3, Figure 1, and Supplemental Table 3). Most of the over-estimation of risk occurred in deciles 7 through 10 of risk, where participants had a 10-year predicted ASCVD risk ≥10%. Additionally, the Hosmer-Lemeshow χ2 indicated good calibration among women (χ2 8.3, p-value=0.41), blacks (χ2 11.8, p-value=0.16) and whites (χ2 14.0, p-value=0.08). The c-index was 0.72 (95%CI: 0.70 – 0.75) and, similar to the overall population, was higher in women and whites compared to men and blacks. The Pooled Cohort risk equations performed similarly in the stroke belt and the remainder of the continental US (Supplemental Table 4).
Table 3.
Observed and predicted incidence rates of atherosclerotic cardiovascular disease among REasons for Geographic And Racial Differences in Stroke (REGARDS) study participants without diabetes, with low density lipoprotein cholesterol 70 to 189 mg/dL† and who were not taking statins by 10-year predicted atherosclerotic cardiovascular disease risk.
| Events / person-years | Events in 5-years | 5-year incidence rate‡ | Calibration | Discrimination | |||
|---|---|---|---|---|---|---|---|
| Observed KM-adjusted | Predicted | Observed KM-adjusted (95% CI) | Predicted | Chi-squared (p-value) | C-index (95% CI) | ||
| All participants | |||||||
| 10-year predicted risk | 19.9 (0.01) | 0.72 (0.70-0.75) | |||||
| <5% | 28 / 14,816 | 32.2 | 32.8 | 1.9 (1.3-2.7) | 1.9 | ||
| 5% to <7.5% | 32 / 6,866 | 37.6 | 37.8 | 4.8 (3.4-6.7) | 4.8 | ||
| 7.5% to <10% | 34 / 5,853 | 40.8 | 45.7 | 6.1 (4.4-8.6) | 6.9 | ||
| ≥10% | 244 / 19,946 | 277.6 | 350.3 | 12.0 (10.6-13.6) | 15.1 | ||
| Women | |||||||
| 10-year predicted risk | 8.3 (0.41) | 0.75 (0.71-0.79) | |||||
| <5% | 20 / 12,907 | 23.9 | 25.9 | 1.6 (1.0-2.5) | 1.7 | ||
| 5% to <7.5% | 17 / 4,231 | 19.9 | 21.5 | 4.1 (2.5-6.6) | 4.5 | ||
| 7.5% to <10% | 19 / 3,121 | 23.4 | 22.7 | 6.5 (4.2-10.2) | 6.3 | ||
| ≥10% | 85 / 7,877 | 98.4 | 118.6 | 10.7 (8.7-13.2) | 12.9 | ||
| Men | |||||||
| 10-year predicted risk | 16.5 (0.04) | 0.66 (0.62-0.70) | |||||
| <5% | 8 / 1,908 | 8.2 | 6.8 | 3.6 (1.8-7.1) | 3.0 | ||
| 5% to <7.5% | 15 / 2,635 | 17.8 | 16.3 | 5.8 (3.5-9.6) | 5.3 | ||
| 7.5% to <10% | 15 / 2,732 | 17.6 | 23.0 | 5.7 (3.4-9.4) | 7.5 | ||
| ≥10% | 159 / 12,069 | 179.5 | 231.7 | 12.9 (11.1-15.0) | 16.6 | ||
| Black | |||||||
| 10-year predicted risk | 11.8 (0.16) | 0.69 (0.65-0.74) | |||||
| <5% | 7 / 4,110 | 8.7 | 10.6 | 1.8 (0.8-3.8) | 2.2 | ||
| 5% to <7.5% | 14 / 2,792 | 17.0 | 16.4 | 5.1 (3.0-8.7) | 5.0 | ||
| 7.5% to <10% | 16 / 2,577 | 19.7 | 20.6 | 6.6 (4.1-10.8) | 6.9 | ||
| ≥10% | 91 / 8,071 | 103.5 | 133.7 | 10.9 (8.9-13.3) | 14.1 | ||
| White | |||||||
| 10-year predicted risk | 14.0 (0.08) | 0.74 (0.71-0.77) | |||||
| <5% | 21 / 10,706 | 23.6 | 22.2 | 1.9 (1.2-2.9) | 1.8 | ||
| 5% to <7.5% | 18 / 4,074 | 20.8 | 21.4 | 4.5 (2.8-7.2) | 4.7 | ||
| 7.5% to <10% | 18 / 3,276 | 21.2 | 25.1 | 5.7 (3.6-9.1) | 6.8 | ||
| ≥10% | 153 / 11,875 | 174.0 | 216.6 | 12.8 (10.9-14.9) | 15.8 | ||
Predicted risk determined using the Pooled Cohort risk equations. 95% CI: 95% confidence interval. KM: Kaplan-Meier
Non-HDL cholesterol between 100 and 219 for the 1,255 who did not have valid low density lipoprotein cholesterol measurements.
Per 1,000 person-years.
REGARDS Participants with Medicare Linked Data
Among the subset of 6,121 REGARDS participants with linked Medicare data, 3,333 participants were in the subgroup for which consideration of statin treatment should be based on ASCVD risk. Due to the limited number of participants with 10-year predicted ASCVD risk <5%, we pooled participants with <7.5% for these analyses. Characteristics of participants by 10-year predicted risk are provided in Supplemental Table 5. There were 457 ASCVD events (225 CHD and 232 strokes with 112 [24.5%] events identified in Medicare claims) during 27,524 person-years in the overall REGARDS-Medicare linked population and 234 ASCVD events (120 CHD and 114 strokes with 57 [24.4%] events identified in Medicare claims) during 15,094 person-years in the REGARDS-Medicare linked population considered for statin treatment based on ASCVD risk. For both of these populations, the Pooled Cohort risk equations were well-calibrated (Table 4, Figure 1, and Supplemental Table 6). With more complete ascertainment of events in this subgroup, there tended to be modest under-prediction of event rates by the Pooled Cohort equations. For example, among the REGARDS-Medicare linked population who could be considered for statin treatment based on ASCVD risk, the observed and predicted risk for participants with a 10-year predicted ASCVD risk <7.5% was 5.3 (95%CI 2.8-10.1) and 4.0, risk 7.5% to <10% was 7.9 (95%CI: 4.6-13.5) and 6.4, and risk ≥10% was 17.4 (95%CI: 15.3-19.8) and 16.4, respectively (χ2=5.4, p-value=0.71).
Table 4.
Observed and predicted incidence rates of atherosclerotic cardiovascular disease among REasons for Geographic And Racial Differences in Stroke (REGARDS) study – Medicare linked participants by 10-year predicted atherosclerotic cardiovascular disease risk.
| Events / person-years | Events in 5-years | 5-years incidence rate† | Calibration | Discrimination | |||
|---|---|---|---|---|---|---|---|
| Observed KM-adjusted | Predicted | Observed KM-adjusted (95% CI) | Predicted | Chi-squared (p-value) | C-index (95% CI) | ||
| 10-year predicted risk | All Medicare linked participants (n=6,121) | 11.4 (0.18) | 0.65 (0.62-0.67) | ||||
| <7.5% | 14 / 2,601 | 14.7 | 11.3 | 5.3 (3.1-8.9) | 4.1 | ||
| 7.5% to <10% | 19 / 2,582 | 21.6 | 18.0 | 7.7 (4.9-12.0) | 6.5 | ||
| ≥10% | 424 / 22,341 | 450.2 | 484.1 | 18.2 (16.6-19.9) | 19.3 | ||
| 10-year predicted risk | Medicare linked participants without diabetes, with LDL-C 70 to 189 mg/dL‡ who were not taking statins (n=3,333) | 5.4 (0.71) | 0.67 (0.64-0.71) | ||||
| <7.5% | N< 11 (Suppressed) | 9.3 | 7.1 | 5.3 (2.8-10.1) | 4.0 | ||
| 7.5% to <10% | (Suppressed) | 14.5 | 11.8 | 7.9 (4.6-13.5) | 6.4 | ||
| ≥10% | 212 / 11,754 | 225.9 | 214.9 | 17.4 (15.3-19.8) | 16.4 | ||
Suppressed – Medicare data are not presented in these cells due to a small sample size.
95% CI: 95% confidence interval. HDL: high density lipoprotein; KM: Kaplan-Meier; LDL-LC: low density lipoprotein cholesterol.
Per 1,000 person-years.
Non-HDL cholesterol between 100 and 219 for the 1,255 who had not fasted prior to their REGARDS study visit.
DISCUSSION
In this large contemporary population-based cohort of black and white US adults, the recently published ACC/AHA Pooled Cohort risk equations appeared to over-estimate ASCVD risk. However, differences in the observed and predicted ASCVD risk were small when limited to participants without diabetes, with LDL-C between 70 and 189 mg/dL, and who were not already taking statins. Calibration in this group is particularly important as it represents the population for whom high predicted risk is intended to trigger a discussion about statin initiation. Furthermore, the observed and predicted ASCVD risks were much more similar when evaluated in participants with Medicare insurance coverage and including ASCVD events identified in Medicare claims. In addition to demonstrating good calibration, the Pooled Cohort risk equations had good discrimination.
For risk equations to be useful in clinical practice, they should be well calibrated so that predicted risk estimates are similar to observed disease incidence. In the full working group report accompanying the publication of the Pooled Cohort risk equations, some over-prediction of ASCVD risk was noted in short-term follow up of the REGARDS study and the Multi-Ethnic Study of Atherosclerosis.1 Following the publication of the working group report, predicted ASCVD risk using the Pooled Cohort risk equations was reported to be systematically higher than observed risk in the Women's Health Study, Physician's Health Study and Women's Health Initiative Observational Study.25 Lack of active surveillance in these studies may have led to the appearance of over-prediction by the Pooled Cohort risk equations because of under-ascertainment of events. As reported recently, 3.6% (1,345/37,397) of women age ≥ 65 years with Medicare Part A coverage in the Women's Health Initiative Clinical Trial had a MI when defined by study adjudication versus 4.8% (1,784/37,397) when including events identified by Medicare claims, a 33% higher number of events similar to the 25% observed in REGARDS.26 The primary reasons for these events not being adjudicated were participants not reporting an event or the inability to retrieve hospital records for adjudication. As we demonstrated in the current analysis, ASCVD risk may appear to be over-estimated by the Pooled Cohort risk equations if all events are not identified. In fact, the predicted and observed risks were remarkably similar when incorporating a surveillance component (i.e., Medicare claims) to provide more complete capture of all clinically relevant events.
A second potential reason for the reported over-estimation of ASCVD risk using the Pooled Cohort risk equations is the high prevalence of statin use in contemporary cohorts.27 At baseline, 29.5% of REGARDS participants with a 10-year ASCVD risk ≥ 10%, the group where the over-estimation of ASCVD risk has been most pronounced, were taking statins. However, the Pooled Cohort risk equations were well calibrated in the subgroup for which these equations were designed to be used, providing assurance of their clinical utility.
The Pooled Cohort risk equations performed well in discriminating between low and high risk participants. The c-index of 0.71 for the Pooled Cohort risk equations in the overall REGARDS study population is similar to the c-index observed for the external validation of the Framingham 10-year CHD risk score used in the Adult Treatment Panel III guidelines.28 Additionally, the c-index in the current analysis was 0.72 for the population for which the 2013 cholesterol treatment guidelines recommend consideration of statin initiation based on ASCVD risk. The c-index was substantially lower in our analyses of REGARDS participants 65 years and older with Medicare coverage. This was not surprising as discrimination is expected to be lower when risk prediction models are applied in narrowly defined populations. Also, as noted in the 2013 ACC/AHA Guideline on the Assessment of Cardiovascular Risk, there are ASCVD risk factors that might improve the discrimination of the Pooled Cohort risk equations. Adding risk factors (e.g., coronary artery calcium score) to the Pooled Cohort risk equations should be examined in future analyses.
A strength of the current analyses includes the large number of REGARDS study participants residing in the continental US. REGARDS enrolled community dwelling adults and provides high generalizability to white and black US adults. REGARDS cohort participant data are linked to Medicare claims providing surveillance for a large subcohort. The results of our study should be interpreted in the context of its limitations. Although follow-up of REGARDS participants is ongoing, data were only available to calculate observed ASCVD risk at 5 years. As the Pooled Cohort risk equations were designed to estimate 10-year ASCVD risk, studies are needed to ensure its accurate calibration over a longer duration. While high positive predictive values have been reported for CHD and stroke events identified using claims-based algorithms17-19, these algorithms have not been validated in the REGARDS study. Therefore, it is possible that the observed ASCVD risk when including Medicare events may be over-estimated, which could affect the good calibration of the Pooled Cohort that we report. We were not able to assess the impact of statin initiation after baseline on the calibration of the risk equations. Although risk prediction is a useful tool for guiding preventive approaches, counseling and treatment decisions should be individualized, as suggested by the new cholesterol guidelines.
Conclusions
In this cohort of US adults for whom statin initiation is considered based on the ACC/AHA pooled risk equations, observed and predicted 5-year ASCVD risks were similar, indicating that these risk equations were well calibrated in the population for which they were designed to be used, and demonstrated moderate to good discrimination. The current study supports the validity of the Pooled Cohort risk equations to inform clinical management decisions.
Supplementary Material
Acknowledgements
This research project is supported by a cooperative agreement U01 NS041588 from the National Institute of Neurological Disorders and Stroke, National Institutes of Health, Department of Health and Human Service. Additional support was provided by grants R01 HL08477 and K24 HL111154 from the National Heart, Lung, and Blood Institute. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Neurological Disorders and Stroke or the National Institutes of Health. The authors thank the other investigators, the staff, and the participants of the REGARDS study for their valuable contributions. A full list of participating REGARDS investigators and institutions and further information about the study can be found at http://www.regardsstudy.org
Role of the Sponsors
Representatives of the funding agency have been involved in the review of the manuscript but not directly involved in the collection, management, analysis or interpretation of the data.
Footnotes
JAMA Access to Data Statement:
Drs. Paul Muntner and Lisandro Colantonio had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
References
- 1.Goff DC, Jr., Lloyd-Jones DM, Bennett G, et al. 2013 ACC/AHA Guideline on the Assessment of Cardiovascular Risk: A Report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2013 Nov 7; doi: 10.1016/j.jacc.2013.11.005. doi: 10.1016/j.jacc.2013.1011.1005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Stone NJ, Robinson J, Lichtenstein AH, et al. 2013 ACC/AHA Guideline on the Treatment of Blood Cholesterol to Reduce Atherosclerotic Cardiovascular Risk in Adults: A Report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2013 Nov 12; doi: 10.1161/1101.cir.0000437738.0000463853.0000437737a. [Google Scholar]
- 3.Rosamond WD, Chambless LE, Heiss G, et al. Twenty-Two-Year Trends in Incidence of Myocardial Infarction, Coronary Heart Disease Mortality, and Case Fatality in 4 US Communities, 1987-2008. Circulation. 2012 Apr 17;125(15):1848–1857. doi: 10.1161/CIRCULATIONAHA.111.047480. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Kleindorfer DO, Khoury J, Moomaw CJ, et al. Stroke incidence is decreasing in whites but not in blacks: a population-based estimate of temporal trends in stroke incidence from the Greater Cincinnati/Northern Kentucky Stroke Study. Stroke. 2010 Jul;41(7):1326–1331. doi: 10.1161/STROKEAHA.109.575043. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Howard VJ, Cushman M, Pulley L, et al. The reasons for geographic and racial differences in stroke study: objectives and design. Neuroepidemiology. 2005;25(3):135–143. doi: 10.1159/000086678. 2005. [DOI] [PubMed] [Google Scholar]
- 6.Safford MM, Brown TM, Muntner P, et al. Asociation of race and sex with risk of incident acute coronary heart disease events in the REasons for Geographic and Racial Differences in Stroke (REGARDS) study. JAMA. 2012;308(17):1768–1774. doi: 10.1001/jama.2012.14306. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Friedewald WT, Levy RI, Fredrickson DS. Estimation of the concentration of low-density lipoprotein cholesterol in plasma, without use of the preparative ultracentrifuge. Clin.Chem. 1972;18(6):499–502. 6/1972. [PubMed] [Google Scholar]
- 8.Fonseca C, Oliveira AG, Mota T, et al. Evaluation of the performance and concordance of clinical questionnaires for the diagnosis of heart failure in primary care. Eur J Heart Fail. 2004 Oct;6(6):813–820, 821-812. doi: 10.1016/j.ejheart.2004.08.003. [DOI] [PubMed] [Google Scholar]
- 9.Meschia JF, Brott TG, Chukwudelunzu FE, et al. Verifying the stroke-free phenotype by structured telephone interview. Stroke. 2000;31(5):1076–1080. doi: 10.1161/01.str.31.5.1076. 5/2000. [DOI] [PubMed] [Google Scholar]
- 10.Howard VJ, Kleindorfer DO, Judd SE, et al. Disparities in stroke incidence contributing to disparities in stroke mortality. Ann.Neurol. 2011;69(4):619–627. doi: 10.1002/ana.22385. 4/2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Stroke--1989. Recommendations on stroke prevention, diagnosis, and therapy. Report of the WHO Task Force on Stroke and other Cerebrovascular Disorders. Stroke. 1989 Oct;20(10):1407–1431. doi: 10.1161/01.str.20.10.1407. [DOI] [PubMed] [Google Scholar]
- 12.Soliman EZ, Howard G, Cushman M, et al. Prolongation of QTc and risk of stroke: The REGARDS (REasons for Geographic and Racial Differences in Stroke) study. J Am Coll Cardiol. 2012 Apr 17;59(16):1460–1467. doi: 10.1016/j.jacc.2012.01.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Thygesen K, Alpert JS, White HD, et al. Universal definition of myocardial infarction. Circulation. 2007 Nov 27;116(22):2634–2653. doi: 10.1161/CIRCULATIONAHA.107.187397. [DOI] [PubMed] [Google Scholar]
- 14.Luepker RV, Apple FS, Christenson RH, et al. Case definitions for acute coronary heart disease in epidemiology and clinical research studies: a statement from the AHA Council on Epidemiology and Prevention; AHA Statistics Committee; World Heart Federation Council on Epidemiology and Prevention; the European Society of Cardiology Working Group on Epidemiology and Prevention; Centers for Disease Control and Prevention; and the National Heart, Lung, and Blood Institute. Circulation. 2003 Nov 18;108(20):2543–2549. doi: 10.1161/01.CIR.0000100560.46946.EA. [DOI] [PubMed] [Google Scholar]
- 15.Prineas RJ, Crow RS, Blackburn H. The Minnesota code manual of electrocardiographic findings: Standards and procedures for measurement and classification. Wright-OSG; Boston, MA: 1982. [Google Scholar]
- 16.Prineas RJ, Crow RS, Zhang ZM. Minnesota Code Manual of Electrocardiographic Findings. 2nd edition ed. Springer-Verlag; London, England: 2010. [Google Scholar]
- 17.Kiyota Y, Schneeweiss S, Glynn RJ, Cannuscio CC, Avorn J, Solomon DH. Accuracy of Medicare claims-based diagnosis of acute myocardial infarction: estimating positive predictive value on the basis of review of hospital records. Am Heart J. 2004 Jul;148(1):99–104. doi: 10.1016/j.ahj.2004.02.013. [DOI] [PubMed] [Google Scholar]
- 18.Kokotailo RA, Hill MD. Coding of stroke and stroke risk factors using international classification of diseases, revisions 9 and 10. Stroke. 2005 Aug;36(8):1776–1781. doi: 10.1161/01.STR.0000174293.17959.a1. [DOI] [PubMed] [Google Scholar]
- 19.Tirschwell DL, Longstreth WT., Jr. Validating administrative data in stroke research. Stroke. 2002 Oct;33(10):2465–2470. doi: 10.1161/01.str.0000032240.28636.bd. [DOI] [PubMed] [Google Scholar]
- 20.Colette D. Modeling Survival Data in Medical Research. Chapman & Hall; London, England: 1994. [Google Scholar]
- 21.Lemeshow S, Hosmer DW., Jr. A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol. 1982 Jan;115(1):92–106. doi: 10.1093/oxfordjournals.aje.a113284. [DOI] [PubMed] [Google Scholar]
- 22.Harrell FE, Jr., Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat.Med. 1996;15(4):361–387. doi: 10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4. 2/28/1996. [DOI] [PubMed] [Google Scholar]
- 23.Pencina MJ, D'Agostino RB. Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Stat Med. 2004 Jul 15;23(13):2109–2123. doi: 10.1002/sim.1802. [DOI] [PubMed] [Google Scholar]
- 24.Hosmer DW, Jr., Lemeshow S. Applied Logistic Regression. Second Edition ed. John Wiley & Sons; New York, NY: 2000. [Google Scholar]
- 25.Ridker PM, Cook NR. Statins: new American guidelines for prevention of cardiovascular disease. Lancet. 2013 Nov 30;382(9907):1762–1765. doi: 10.1016/S0140-6736(13)62388-0. [DOI] [PubMed] [Google Scholar]
- 26.Hlatky MA, Ray RM, Burwen DR, et al. Use of Medicare Data to Identify Coronary Heart Disease Outcomes in the Women's Health Initiative. Circulation. Cardiovascular quality and outcomes. 2014 Jan 1;7(1):157–162. doi: 10.1161/CIRCOUTCOMES.113.000373. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Muntner P, Levitan EB, Brown TM, et al. Trends in the prevalence, awareness, treatment and control of high low density lipoprotein-cholesterol among United States adults from 1999-2000 through 2009-2010. The American journal of cardiology. 2013 Sep 1;112(5):664–670. doi: 10.1016/j.amjcard.2013.04.041. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.D'Agostino RB, Sr., Grundy S, Sullivan LM, Wilson P. Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA. 2001;286(2):180–187. doi: 10.1001/jama.286.2.180. 7/11/2001. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.