Headed in the Right Direction But at Risk for Miscalculation

Abstract

The newly released 2013 ACC/AHA Guidelines for Assessing Cardiovascular Risk makes progress compared with previous cardiovascular risk assessment algorithms. For example, the new focus on total atherosclerotic cardiovascular diseases (ASCVD) is now inclusive of stroke in addition to hard coronary events, and there are now separate equations to facilitate estimation of risk in non-Hispanic white and black individuals and separate equations for women. Physicians may now estimate lifetime risk in addition to 10-year risk. Despite this progress, the new risk equations do not appear to lead to significantly better discrimination than older models. Because the exact same risk factors are incorporated, using the new risk estimators may lead to inaccurate assessment of atherosclerotic cardiovascular risk in special groups such as younger individuals with unique ASCVD risk factors. In general, there appears to be an overestimation of risk when applied to modern populations with greater use of preventive therapy, although the magnitude of overestimation remains unclear. Because absolute risk estimates are directly used for treatment decisions in the new cholesterol guidelines, these issues could result in overuse of pharmacologic management. The guidelines could provide clearer direction on which individuals would benefit from additional testing, such as coronary calcium scores, for more personalized preventive therapies. We applaud the advances of these new guidelines, and we aim to critically appraise the applicability of the risk assessment tools so that future iterations of the estimators can be improved to more accurately assess risk in individual patients.

In November 2013, the American College of Cardiology/American Heart Association released new guidelines for assessing cardiovascular risk (1). These risk assessment guidelines (RAG) are an evolution from previous recommendations that asymptomatic adults undergo a global risk assessment as the first step in the evaluation of atherosclerotic cardiovascular disease (ASCVD) risk (2). Previous guidelines, such as the Adult Treatment Panel III (ATP-III), used a modified Framingham risk score (FRS) (3) to set thresholds for lipid treatment based on one’s predicted 10-year risk for myocardial infarction or coronary heart disease (CHD) death. However, the FRS was limited by both underestimation (4–6) and overestimation (7) of risk in different populations. Newer risk models emerged, including the Reynolds Risk Score (RRS) for ASCVD (8) and variations of the FRS developed for total ASCVD, lifetime risk, and risk estimation without laboratory results (9). With so many risk-scoring systems, clinicians often ignored formal global risk assessment in clinical practice (10) because it was challenging to know which to use for each particular patient.

The RAG also endorse global risk assessment as a first step in asymptomatic adults age 40 to 79 years. New pooled cohort equations (2013 Risk Models) estimate 10-year risk for a first hard ASCVD event, including CHD and stroke, but not congestive heart failure or revascularization. These risk equations were derived using pooled data from National Heart, Lung, and Blood Institute (NHLBI)–funded cohorts beyond the Framingham original and offspring cohorts, including ARIC (Atherosclerosis Risk in Communities), CHS (Cardiovascular Health Study), and CARDIA (Coronary Artery Risk Development in Young Adults) cohorts. Importantly, RAG support separate risk assessment equations by gender and for African-American populations. Former guidelines had recommended the use of single risk stratification algorithms for all individuals regardless of race/ethnicity, thereby potentially overestimating or underestimating risk in nonwhite populations.

We applaud these incremental advances of RAG, but we also recognize limitations for its widespread applicability and use. This paper critically appraises RAG and provides our view about what could be improved in future iterations of the guidelines.

Performance in More Diverse and Modern Populations

Despite having a separate score by sex and race, the RAG caution against applying this algorithm to groups that are neither white nor black. This may result in overestimation of ASCVD risk in groups such as Chinese/East-Asian Americans and underestimation in American-Indians and Americans of South Asian descent (11). One notable absence from the pooled cohorts was MESA (Multi-Ethnic Study of Atherosclerosis), which had not reached 10 years of follow-up time at the time the 2013 risk models were developed. This is disappointing because, by design, MESA is the most diverse of the NHLBI-funded cohorts. Furthermore, MESA’s important findings regarding the added prognostic value for risk prediction conferred by subclinical vascular disease testing was not considered for inclusion in the model.

By design, the populations included were from a generation ago and were less diverse. Many population changes have occurred since that time, with secular trends involving smoking habits, trans-fat intake, and preventive medications. While the “outcome” of the risk score has evolved to include ASCVD events, the “input” variables for the risk prediction model remain identical. If future iterations of the guidelines continue to rely on older cohorts, prioritizing them as more representative of the natural history of ASCVD, then we will become more and more disconnected from the modern patients in our clinics, and have less ability to incorporate new inputs in risk estimation simply because they were not measured generations ago.

The RAG also acknowledge overestimation of risk when the 2013 risk models were applied to 2 external validation cohorts (MESA and REGARDS [Reasons for Geographic and Racial Differences in Stroke]) and to contemporary data from 2 of the derivation cohorts (ARIC/Framingham) (11). However, overestimation was more pronounced at higher-risk (a group above the risk-based treatment threshold) than for lower-risk groups where there would be clinical concern for over-treatment. Discrimination was also suboptimal, with C statistics of 0.66 to 0.77 in women and 0.56 to 0.71 in men compared with 0.81 and 0.71 to 0.75 in the respective derivation cohorts (11).

When the 2013 risk models were applied to 3 large-scale primary prevention cohorts—WHS (Women’s Health Study), PHS (Physicians’ Health Study), and WHI (Women’s Health Initiative observational study), there was also overestimation of risk (12). Discrepancies may exist in these particular cohorts because these were largely lower risk white cohorts and because of different methods for risk factor measurement. A flurry of attention was immediately drawn to this new estimator and the implication that more patients would be considered for statin therapy. As a counterpoint, the guideline writers noted that most of the discordance was in those with >10% estimated 10-year risk, and that the risk estimator does not prescribe statin therapy, but rather serves as a starting point for a “risk discussion.” Less discordance occurred in the middle risk groups of 5.0% to 7.5% and 7.5% to 10.0%. Moreover, the older ATP-III version of the FRS for hard CHD events was also previously shown to overestimate risk (7).

A subsequent re-analysis of REGARDS (13) suggested that calibration was improved if 5-year modifications of 2013 risk models were applied only in a “clinically relevant population”—those age 40 to 75 years, without diabetes, not taking statins, and with LDL-C levels of 70 to 189 mg/dl. This was especially the case if Medicare claims files were used as a form of active ascertainment to identify more events. Unfortunately, these claims events were not adjudicated, and data available for calibration with observed ASCVD events in REGARDS was only through 5 years (i.e., not 10 years as per risk estimator). Moreover, discrimination remained suboptimal, with C-statistics of ~0.7 (13).

Similarly, in a more recent comparison of the 2013 Risk Models vs. ATP III versus European Society of Cardiology guidelines among participants of the Rotterdam Study, a European cohort (14), all 3 models showed poor calibration (overestimation of risk compared to observed incidence of events) and just moderate discrimination (C-statistics 0.67 to 0.77). However, using the 2013 risk models with the new thresholds set in the treatment guidelines, more people would be eligible for a patient-clinician discussion to consider statin therapy.

Collectively, these studies seem to call for a continued examination of the proper population from which to derive risk equations, and critical evaluation of the continued use of population-equations for estimating risk in individual patients (15).

Exclusion of Family History in the Model

Family history, which is asked by virtually every clinician conducting clinical risk assessments, was considered for inclusion in the 2013 risk models. However, it did not adequately improve model prediction, likely because it was not distinguished from a family history of premature CHD, a well-established predictor of subsequent ASCVD events (16). The RAG limit the use of family history to those patients in whom there is uncertainty in the risk-based treatment decision. This is a departure from ATP-III, which considered a family history of premature CHD a major risk factor. It is similarly endorsed as a significant risk factor by experts in the 2011 American Heart Association’s Effectiveness-Based Guidelines for the Prevention of Cardiovascular Disease in Women (17). Additionally, Canadian Cardiovascular Society guidelines recommended that a person’s estimated risk be doubled with a family history of premature ASCVD (18). Integrating family history into a risk estimator could facilitate its universal adoption, such as is done with the Reynolds Risk Score (8). Preventive cardiovascular practice dictates that family history be included in each patient’s clinical assessment of risk, and we anticipate that future iterations of the RAG will need to more fully integrate family history into risk assessment.

No Mention of Special Populations at Risk

Systemic autoimmune collagen vascular diseases, such as lupus erythematosus and rheumatoid arthritis, are more prevalent in women and have been shown to increase one’s relative risk for ASCVD (19). Pre-eclampsia is independently associated with an increased risk for ASCVD (20). The 2011 AHA Women’s Guidelines consider these disorders to be significant risk factors for ASCVD, on par with traditional risk factors such as smoking and hypertension (17). Further, chronic kidney disease is considered to be a risk equivalent on par with clinically manifest ASCVD and diabetes.

It would be cumbersome and impractical to add these and other unique risk factors to a universal risk prediction model. However, physicians require further direction on how best to categorize individuals with unique risk factors. It is especially in this setting that further risk stratification with tests that inherently clarify individual risk (such as by using imaging tests, for example coronary artery calcium [CAC] scoring), may better delineate those who would most likely benefit from preventive therapies (15).

Inclusion of Stroke in Risk Score Outcome

Atherosclerotic stroke and CHD share many risk factors, and it is commendable that the RAG also highlighted stroke among its important ASCVD outcomes. However, only ~40% of strokes are from large vessel (i.e., carotid) atherosclerotic disease; therefore, only this fraction is most amenable to primary prevention with statin therapy. The remainder of strokes are cardioembolic, lacunar, or hemorrhagic and unlikely to benefit from statins (21,22). We caution against extrapolating the ASCVD risk estimate to determine statin eligibility given the diverse etiology of “stroke.” It remains to be seen how well this risk threshold will perform in discriminating who would best and least benefit from preventive therapy for stroke prevention, which is a separate question than predicting global ASCVD risk.

The inclusion of stroke also makes the new risk estimator much more sensitive to age, which was already perceived as a possible weakness of the FRS. Because strokes are very rare in young patients, and nearly exclusively occur in older patients, risk with the new estimator rises much faster with age compared to the FRS.

Lowering the High-Risk Threshold to ≥7.5%

The RAG now adopts a threshold of ≥7.5% 10-year ASCVD risk to indicate “high-risk” status, and reference is made to the 2013 Cholesterol Guidelines, which endorse instituting statin treatment at this threshold after patient–clinician discussion (1). In contrast, ATP-III categorized 10% to 20% as “intermediate risk” and ≥20% as high risk (3). These risk percentages are not directly comparable, but the newer measures will significantly increase the number of individuals potentially eligible for statin therapy. This is especially true because persons with low-density lipoprotein cholesterol levels previously characterized as normal (70 to 100 mg/dl) might now be considered for pharmacotherapy.

Many people will cross the 7.5% threshold on the basis of chronologic age alone, a nonmodifiable risk factor. Nearly all African-American men over age 62 years with ideal parameters would cross over the 7.5% threshold and be considered for statin treatment. Risk equations apply to populations: no single individual has a 7.5% 10-year risk. Therefore, without personalized attention to the individual, patients may be overtreated or undertreated (15). The estimators do not take into account unique risk factors or “vascular age.” For example, elderly individuals with no CAC actually have a lower mortality than younger individuals with high CAC scores (23).

Previous guidelines have supported the use of additional screening tests such as CAC in the “intermediate-risk” group (10% to 20% 10-year risk) with a Class IIa indication and with a Class IIb indication for 6% to 10% 10-year risk (2). However, in the RAG, the new de facto “intermediate-risk” category of 5.0% to 7.5% is now rather narrow (24). This further minimizes the potential gains to be added by testing for subclinical atherosclerosis. Furthermore, by putting more people into low and high risk groups, the RAG gives the appearance of greater certainty of risk, when in fact the 2013 risk models use the risk modeling techniques and same risk factors as prior estimators (24).

Subclinical Atherosclerosis Assessment

The most significant limitation of the RAG is a tepid endorsement for the selective use of screening tests for subclinical atherosclerosis. The new guidelines only give a Class IIb indication for optional screening tests when risk-based decisions are “uncertain.” Risk status may be upgraded to a higher score if one of the following is present: family history of premature ASCVD, high-sensitivity C-reactive protein score ≥2.0 mg/l, CAC score ≥300 or ≥75th percentile, or an ankle-brachial index <0.9. Carotid intima-medial thickness alone was not recommended as a screening tool. No guidance was given on how to use these tests to downgrade risk estimates, although this may be one of best applications of CAC testing (25). The guidelines do not clarify in which patients risk is “uncertain.” Some analyses have suggested that nearly all risk estimates based just on one-time assessments of traditional risk factors are in some ways “uncertain” (15). Using the new risk guidelines, providers and payers will likely find it difficult to discern and defend which patients would most benefit from additional screening for subclinical atherosclerosis.

Patients with a family history of premature ASCVD, metabolic syndrome, inflammatory disorders, or complicated pregnancy history (i.e., pre-eclampsia, low birth weight infants) might be good candidates for additional evaluation for subclinical atherosclerosis, regardless of their RAG-estimated risk. It is important to remember that not everyone with a family history of premature ASCVD is at a relatively high risk for a future ASCVD event, and some parental history of ASCVD may have been confounded by prior smoking, which was much more common in prior generations. Additionally, patients with higher risk driven solely or primarily by age may be good candidates for additional evaluation that could lead to downgrading the risk estimate (15).

Compared with traditional risk factors, CAC clearly improves risk discrimination. For comparison, in MESA, the area under the ROC curve for the prediction of CHD events was 0.76 (95% confidence interval: 0.72 to 0.79) using a model with only traditional risk factors similar to those factors in the 2013 risk models, but increased to 0.81 (95% confidence interval: 0.78 to 0.84) with the addition of CAC (26). Individuals without risk factors but elevated CAC have substantially higher event rates than those who have multiple risk factors but no CAC (27,28). In the MESA, individuals with 0 risk factors and CAC >300 had an event rate 3.5 times higher than individuals with ≥3 risk factors and CAC of 0 (28). Similar findings were seen when CAC was compared across FRS categories (28). When compared head-to-head in the same population, CAC was a much more potent risk discriminator than high-sensitivity C-reactive protein (29), with far superior net reclassification (26).

Individuals with CAC of 0 but elevated global risk score (above the 7.5% threshold) may be prone to overtreatment with pharmacologic therapies. Individuals with CAC of 0 have very low event rates (25,29,30); thus, the number needed to treat to prevent 1 event with statin therapy may be prohibitively high in this group. This raises the need for a paradigm shift away from a predominantly risk factors–based prevention approach for determining high risk, towards an approach guided by the detection of subclinical atherosclerosis for additional identification of patients unlikely to gain a new benefit of treatment (31).

The threshold of a CAC score of >300 is also quite high, consistent with advanced plaque. High event rates similar to rates in secondary prevention populations have been noted for CAC scores >100 (29,32), and that alternate threshold may also be considered for determining eligibility for statin therapy. Furthermore, a CAC >75th percentile score does not confer the same very high risk status as a score >300 or >100, although these are viewed as equivalent options in the RAG. For predicting 5-year event rates, absolute CAC scores are superior to percentile scores (33).

The RAG did acknowledge that among additional screening tests, CAC would be the most useful for improving risk prediction. Their decision against incorporating CAC was based on a systematic review of subclinical atherosclerosis markers (34) where the outcomes were CHD events but not total ASCVD. However, in the Heinz Nixdorf Recall Study, CAC also predicted the risk of stroke (35). In MESA, CAC predicts both CHD and total ASCVD, although it was stronger for CHD (29).

The RAG further cite issues with cost and radiation as reasons against a stronger endorsement for CAC. The cost of CAC screening is now commonly $75 to $100, and less expensive than certain diagnostic tests or years of treatment with generic statin therapy. The radiation dose is down to ~1 mSv, comparable to a bilateral mammogram, which is already an accepted screening tool (36). The EISNER (Early Identification of Subclinical Atherosclerosis by Noninvasive Imaging Research) study showed that downstream costs were not increased with CAC screening, and subsequent testing and treatment costs were reduced for those found to have CAC scores of 0 (36).

10-Year Risk Versus Lifetime Risk

The RAG give a modest Class IIb indication for assessing a 30-year or lifetime ASCVD risk based on traditional risk factors for adults age 20 to 59 years not at increased 10-year risk. This will help in additional risk assessment for younger individuals who may face unique risks. Individuals with low 10-year but high lifetime risk have a greater subclinical disease burden and greater incidence of atherosclerotic progression compared with individuals with low 10-year and low lifetime risk, even at younger ages (37). Differences in risk factor burden in middle age translate to substantial differences in longevity and lifetime risk of ASCVD (38). Assessment of lifetime risk may influence physician behavior in recommending pharmacologic preventive therapies (39).

Conclusions

The new 10-year pooled ASCVD risk score with sex- and race-specific equations is an incremental improvement from the ATP-III version using the 10-year FRS for hard CHD. However, it will only be considered an overall improvement if it leads to more personalized risk estimates, rather than less personalized risk estimates. The RAG still need further refinement in their application for ethnically diverse populations and younger individuals with unique cardiovascular risk factors. In the next iterations, we would encourage a thorough rethinking of the traditional approach to risk assessment, with careful consideration of the interplay between a risk-factor based model and an atherosclerosis imaging-based model, with greater clarity on who would most benefit from screening for subclinical atherosclerosis. As with any risk model, the discussion and use of the information should be individualized. We would like to see more explicit guidance on how to best conduct a “risk discussion” in busy clinical practice. We commend the advances made with the most recent 2013 risk models, and we intend our constructive criticism to serve as a starting point for further refinement of the RAG.

Abbreviations and Acronyms

ATP-III

Adult Treatment Panel III

ASCVD

atherosclerotic cardiovascular disease

CAC

coronary artery calcium

CHD

coronary heart disease

FRS

Framingham Risk Score

RAG

risk assessment guideline