Abstract
Background
Limited attention has been paid to negative cardiovascular disease (CVD) risk markers despite their potential to improve medical decision-making. We compared thirteen negative risk markers using diagnostic likelihood ratios (DLR), which model the change in risk for an individual after the result of an additional test.
Methods and Results
We examined 6,814 participants from the Multi-Ethnic Study of Atherosclerosis. Coronary artery calcium (CAC) =0, carotid intima-media thickness (CIMT) <25th percentile, absence of carotid plaque, brachial flow-mediated dilation >5% change, ankle brachial index (ABI) >0.9 and <1.3, high sensitivity C-reactive protein (hsCRP) <2 mg/L, homocysteine <10 µmol/L, NTpro-BNP <100 pg/mL, no microalbuminuria, no family history of coronary heart disease (CHD) (any/premature), absence of metabolic syndrome, and healthy lifestyle were compared for all, hard CHD and all CVD events over 10-year follow-up. Models were adjusted for traditional CVD risk factors. Among all negative risk markers CAC=0 was the strongest, with adjusted mean DLR (SD) of 0.41 (0.12) for all CHD and 0.54 (0.12) for CVD, followed by CIMT <25th percentile (DLRs 0.65 [0.04] and 0.75 [0.04], respectively). HsCRP <2 mg/L and normal ABI had DLRs >0.80. Among clinical features, absence of any family history of CHD was the strongest (DLRs 0.76 [0.07] and 0.81 [0.06], respectively). Net Reclassification Improvement (NRI) analyses yielded similar findings, with CAC=0 resulting in the largest, most accurate downward risk reclassification.
Conclusions
Negative results of atherosclerosis-imaging tests, particularly CAC=0, resulted in the greatest downward shift in estimated CVD risk. These results may help guide discussions regarding identification of individuals less likely to receive net benefit from lifelong preventive pharmacotherapy.
Keywords: Cardiovascular disease, marker, risk prediction, calcium score
Clinical risk assessment and stratification are the primary means for selecting individuals most likely to benefit from preventive medications, such as statin and aspirin therapy.1 To date, attention has been focused on the identification of individuals at high risk of cardiovascular disease (CVD) who may require pharmacotherapy. However, in the current context of population aging, increasing rates of chronic conditions and polypharmacy,2,3 the accurate identification of individuals at very low risk who might justifiably avoid lifelong pharmacotherapies is worthy of consideration.
The current risk factor-based paradigm in CVD risk prediction is designed to identify individuals at increased risk of events. Given their strong dependence on chronological age and gender,4 risk scores may systematically consider certain groups like middle-age and older men as high risk,5 failing to identify low-risk individuals. The traditional focus on identifying increased risk individuals is most useful in the setting of a high threshold for preventive pharmacotherapy. However, recent American College of Cardiology / American Heart Association (ACC/AHA) guidelines6 have significantly lowered the threshold for considering statin treatment, which coupled with concerns for risk overestimation7 may lead to a potential for overtreatment.
In the current setting of near universal treatment qualification for certain subgroups, negative risk markers (i.e., the absence of a clinical risk marker, or a negative result of a prognostic test) may have potential importance.8 A strong negative risk marker may allow identification of truly low-risk individuals who are less likely to receive a net benefit from preventive therapy, aiding decision making in the context of the physician-patient discussion. Due to their high sensitivity for detecting disease and, therefore, higher negative predictive values, there is increasing interest in the value of subclinical atherosclerosis-imaging tests such as a coronary artery calcium (CAC) score of zero as potential negative risk markers.8,9 However, little is known regarding the performance of other negative tests and clinical features for identifying low absolute risk.
We sought to compare the relative value of various negative risk markers in a contemporary, multi-ethnic cohort. For this aim we adapted a methodology for calculating risk-adjusted diagnostic likelihood ratios (DLR), a technique which emulates Bayesian clinical decision making, by describing how risk estimates are adjusted after new knowledge is available from the result of a test.10,11 In addition, to evaluate the accuracy of change in risk classification we calculated the net reclassification improvement (NRI) for each of the 13 negative risk markers.
Methods
Study Design and Participants
The Multi-Ethnic Study of Atherosclerosis (MESA) is a NIH/NHLBI-funded, population-based, prospective cohort study. Full details of the study design and methods have been reported previously.12 MESA enrolled 6,814 men and women aged 45 to 84 years free from clinical CVD at the time of recruitment. Study participants were enrolled between July 2000 and September 2002 at six US field centers and provided written informed consent. The study protocol was approved by the institutional review board of each study site.
Traditional CVD Risk Factors and 10-Year CVD Risk Assessment
Information on demographics, cardiovascular risk factors and medication use was collected at the baseline study examination. The metabolic syndrome was defined according to the NCEP–ATP III definition.13 Family history of coronary heart disease (CHD) was assessed using standardized questionnaires, with information on family history of premature CHD collected at the second study visit. A positive family history of CHD was defined as any first-degree relative with a history of fatal/non-fatal myocardial infarction and/or having undergone coronary revascularization. Family history of premature CHD was defined as any of the previous occurring before 55 and 65 years of age for male and female family members, respectively. Diet was assessed with a food frequency questionnaire, and physical activity was determined using a validated survey tool.14 The 10-year atherosclerotic CVD (ASCVD) risk of each participant was calculated using the ACC/AHA 2013 Pooled Cohort Equations.15
Non-Traditional CVD Risk Marker Assessment
At baseline, participants underwent measurements of CAC, ankle-brachial index (ABI), carotid intima-media thickness (CIMT), carotid plaque, and brachial flow-mediated dilation (FMD).
Non-contrast cardiac-gated chest tomography was performed following a standardized protocol and CAC was quantified using the method of Agatston. CAC scans were performed twice, and the average of the two scans was considered the baseline CAC score.
High-resolution B-mode ultrasound of the left and right common and internal carotid arteries was used for assessing CIMT and carotid plaque. Regarding CIMT, the mean of the maximum IMT of the common carotid arteries was used for the analyses. Carotid plaque presence was defined as a focal absolute wall thickness (IMT >1.5 mm) or a relative focal thickening of >50% of the adjacent IMT.
For measurement of FMD, high-resolution ultrasound was used for measuring the diameter of the right brachial artery at baseline, and 60 seconds after inducing ischemia for five minutes. Reactivity was assessed as the percentage difference between diameters. FMD measurements were available in 3,027 participants and we used sample weights to account for the case-cohort selection of these data.
Systolic blood pressure determinations were performed with the participant in the supine position after resting for five minutes using a Doppler probe in the bilateral brachial, dorsalis pedis, and posterior tibial arteries. The ABI was calculated for each leg as the ratio between the highest of the two pressures obtained in the same ankle, and the highest of the two brachial artery pressures. The lowest of the two ABIs (left leg vs right leg) was used for the analyses.
Levels of high sensitivity C-reactive protein (hsCRP) were quantified using a BNII nephelometer (N-High Sensitivity CRP; Dade Behring Inc., Deerfield, IL). Homocysteine was measured by fluorescence polarization immunoassay using an IMx Analyzer (IMx Homocysteine Assay, Axis Biochemicals ASA, Oslo, Norway). Levels of NT-ProBNP were measured using an ElecSys 2010 analyzer (Roche Diagnostics, Indianapolis, IN). Urine albumin was measured by nephelometry. Detailed data on reliability of measurements can be found in the Supplementary Methods Appendix.
Definition of Negative Risk Markers
To maximize clinical relevance, the definitions of negative risk markers matched the cutpoints used in clinical practice and/or widely cited in the literature.
“CAC=0” was defined as a CAC score of zero Agatston Units.8,9 “Low CIMT” was defined as a CIMT ≤25th percentile16 of the study population. “Normal FMD” was defined as an ischemia-induced diameter change ≥5%.17 “Normal ABI” was defined as an index between 0.9 and 1.3.18
For serum biomarkers the following definitions were used for a negative result: hsCRP <2mg/L,19 homocysteine <10µmol/L,20 and NT-ProBNP <100pg/mL.21
“No microalbuminuria” was defined as a daily urine excretion of albumin <30mg.22 “Healthy lifestyle”, as defined by Ahmed et al23, was characterized by the presence of at least two of the following three lifestyle-related traits: regular physical activity, Mediterranean-style diet, and body mass index ≥18.5 and <25 kg/m2. Absence of smoking was not used in this definition, as tobacco use was already included in the models used for calculating multivariable-adjusted DLRs.
In sensitivity analyses, we also examined the results for a CIMT standardized z-score <0, and for the combinations of CIMT ≤25th and ≤50th percentiles and the absence of carotid plaque as joint negative risk markers. Also, alternative cut-off points for hsCRP (<1mg/L and <3mg/L) as well as the sample median for each continuous test were also tested.
Follow Up
Follow-up extended for a mean of 10.3±2.3 years. Incident CHD and CVD events were adjudicated by the MESA mortality and morbidity review committee.12 Hard CHD events included non-fatal myocardial infarction, death from CHD, and resuscitated cardiac arrest. All CHD events included hard CHD events plus definite angina and probable angina followed by revascularization. CVD events included all CHD events, fatal/non-fatal stroke, other atherosclerotic death, and other CVD death.
Statistical Analysis
The DLR is also known as a “Bayes factor” as it quantifies the change in risk obtained with knowledge of a test result (post-test risk) versus not knowing the result (pre-test risk).10,11 The DLR can range from 0 to ∞, with values >1 indicating that the test result is more likely to be seen in those who become cases (in our study, develop CHD/CVD events), therefore useful for upgrading disease risk; whereas values <1 indicate that the test result is less likely to be seen in those who become cases, therefore more useful for downgrading disease risk.
We constructed equations for calculating the DLR of each negative risk marker by formally comparing coefficients from multivariable logistic regression models before and after the addition of each negative test, according to the methods described by Janssens and others10 and Gu and Pepe.11
Given a set of baseline predictors X (e.g., the CVD risk factors included in the ACC/AHA 2013 10-year ASCVD risk estimator) to predict a binary outcome D (e.g., not having vs. having a CVD event at 10 years), with an additional predictor Y representing a negative risk marker (a test with a given result, e.g. CAC=0), the pre-test and post-test risks can be expressed as follows:
Pre-Test Risk logit P (D = 1 | X) = β0* + βX*X
Post-Test Risk logit P (D = 1 | X, Y) = β0 + βXX + βYY + βXYXY
As described by Gu and Pepe,11 the covariate-adjusted DLR is calculated by subtracting the pre-test risk model from the post-test risk model:
log DLRX (Y) = (β0 - β0*) + (βX - βX*) + βYY + βXYXY.
The DLR is covariate-specific, that is, the DLRX will vary per individual based on the individual set of risk factors that are present. For illustration, in addition to mean DLRs we have provided the DLRX for several patient scenarios of varying age, gender, race/ethnicity and risk factor burden. When the patient-specific DLRX is known, the post-test risk for an individual case can be computed through the relationship:
logit P (D = 1| Y, X) = logit P (D = 1| X) + log DLRX (Y).
As opposed to the traditionally reported covariate-adjusted odds ratio, which is not specific to an individual set of covariates, the DLR models the anticipated result of a new test contingent on a known set of risk factors for an individual patient. Importantly, as demonstrated by Gu and Pepe11 and others24 this implies that DLR for the individual may be small even when the covariate-adjusted odds ratio for a given test may be large. The average DLR is proportional to the average decrease in risk after knowing the result a specific negative risk marker.
Following the above methods, multivariable adjusted-DLR were calculated after adjusting for 1) traditional cardiovascular risk factors (age, gender, smoking status, diabetes, total and HDL cholesterol, systolic blood pressure, and hypertension medication use), and race/ethnicity; and for 2) predicted 10-year ASCVD risk using the ACC/AHA 2013 Pooled Cohort Equations.15 These methods were repeated for the outcomes of CHD (any), hard CHD, and CVD.
As summary statistics, we provide the mean DLRs (standard deviation [SD]) for the thirteen negative risk markers for the entire MESA population as well as for important subgroups. In addition, we also graphically examined the mean effect of a single covariate (such as age or the 10-year ASCVD risk estimate) on the DLR. The DLRs for each test for other common clinical case scenarios are shown in the Supplementary Appendix.
As a measure of risk reclassification, the NRI was calculated for each of the negative risk markers using different risk thresholds. First, we calculated the NRI for all individuals with a pre-test 10-year CVD risk of >7.5%, testing the accuracy of reclassification to below 7.5% risk. We repeated our analysis in the population of participants with 10-year pre-test risk of >5%, testing the accuracy of reclassification to below such threshold. Finally, we also modeled the ability of each test to accurately move individuals from >7.5% risk to below 5% risk, which would be a highly clinically significant risk shift. The NRIs were calculated as follows:
[Prob (being correctly reclassified to a lower-risk category|nonevent) − Prob (being incorrectly classified to a lower-risk category|event)]
Which is the traditional calculation of the NRI, accounting for the fact that it is not possible for negative risk markers tp upwardly reclassify risk.
Analyses were performed using Stata version 13 (StataCorp, College Station, TX).
Results
The mean age of the 6,814 study participants was 62±10 years and 53% were women. At baseline, 13% were current smokers, 13% had diabetes, 37% used antihypertensive medications and 16% lipid-lowering treatments. The mean estimated 10-year ASCVD risk was 16.4% among men and 11.2% among women (Table 1).
Table 1.
Baseline characteristics of the 6,814 study participants, the Multi-Ethnic Study of Atherosclerosis.*
| All (n = 6,814) |
Men (n = 3,213) |
Women (n = 3,601) |
P value | |
|---|---|---|---|---|
| Age, years | 62.2 (10.2) | 62.2 (10.2) | 62.1 (10.3) | - |
| Race/ethnicity | 0.063 | |||
| White | 2,622 (38.5) | 1,259 (39.2) | 1,363 (37.9) | |
| African American | 1,893 (27.8) | 843 (26.2) | 1,050 (29.2) | |
| Hispanic | 1,496 (22.0) | 721 (22.4) | 775 (21.5) | |
| Chinese American | 803 (11.8) | 390 (12.1) | 413 (11.5) | |
| Body mass index, kg/m2 | 28.3 (5.5) | 27.9 (4.5) | 28.8 (6.2) | 0.001 |
| Current smoker | 887 (13.1) | 467 (14.6) | 420 (11.7) | <0.001 |
| Physical activity†, MET-min/week | 5,749 (5,896) | 6,445 (6,751) | 5,127 (4,930) | <0.001 |
| Diabetes | 859 (12.7) | 449 (14.0) | 410 (11.4) | 0.001 |
| Systolic blood pressure, mmHg | 127 (21) | 126 (19) | 127 (23) | 0.635 |
| Diastolic blood pressure, mmHg | 72 (10) | 75 (9) | 69 (10) | <0.001 |
| Total cholesterol, mg/dL | 194 (36) | 188 (35) | 200 (36) | <0.001 |
| HDL cholesterol, mg/dL | 51 (15) | 45 (12) | 56 (15) | <0.001 |
| Hypertension treatment use | 2,536 (37.2) | 1,151 (35.8) | 1,385 (38.5) | 0.023 |
| Lipid-lowering treatment use | 1,100 (16.2) | 521 (16.2) | 579 (16.1) | 0.890 |
| 10-year CHD Risk‡ | 10.95 (8.52) | 15.24 (9.66) | 7.10 (4.80) | <0.001 |
| 10-year ASCVD Risk§ | 13.7 (13.3) | 16.4 (13.1) | 11.2 (12.9) | <0.001 |
Variables presented as mean (SD) or n (%).
Defined as moderate and vigorous physical activity in MET-min/week
Calculated using the Framingham Risk Score.
Calculated using the ACC/AHA 2013 Pooled Cohort Equations.
HDL indicates high density lipoprotein; CHD, coronary heart disease; and ASCVD, atherosclerotic cardiovascular disease.
The baseline prevalence of each negative risk marker is shown in Table 2 (distribution of continuous risk markers shown in Table S1). Normal ABI and no microalbuminuria were the most prevalent negative risk markers (93% and 90%, respectively), whereas normal FMD and a low CIMT were present in 36% and 25% of the study participants, respectively.
Table 2.
Prevalence of baseline negative risk markers, and number of incident coronary heart disease and cardiovascular disease events at follow-up among participants with each negative risk marker.*
| Events at follow-up |
||||
|---|---|---|---|---|
| Prevalence at baseline |
All CHD | Hard CHD | All CVD | |
| Coronary Artery Calcium Score = 0 | 3,416 (50.1) | 74 (2.2) | 57 (1.7) | 137 (4.0) |
| Carotid Intima-Media Thickness ≤ 25th Percentile | 1,685 (25.1) | 52 (3.1) | 39 (2.3) | 80 (4.7) |
| No Carotid Plaque | 3,927 (58.5) | 191 (4.9) | 124 (3.2) | 280 (7.1) |
| Flow-Mediated Dilation ≥ 5% Change | 1,079 (35.7) | 69 (6.4) | 38 (3.5) | 89 (8.2) |
| Ankle-Brachial Index > 0.9 – < 1.3 | 6,243 (92.7) | 429 (6.9) | 266 (4.3) | 620 (9.9) |
| HsCRP < 2 mg/L | 3,496 (51.7) | 234 (6.7) | 155 (4.4) | 324 (9.3) |
| Homocysteine < 10µmol/L | 4,660 (68.6) | 279 (6.0) | 171 (3.7) | 402 (8.6) |
| NT-ProBNP < 100pg/mL | 3,988 (71.3) | 235 (5.9) | 133 (3.3) | 326 (8.2) |
| No Microalbuminuria | 6,127 (90.4) | 403 (6.6) | 258 (4.2) | 572 (9.3) |
| No Family History of CHD | 3,661 (57.3) | 204 (5.6) | 130 (3.6) | 307 (8.4) |
| No Family History of Premature CHD | 4,303 (80.5) | 301 (7.0) | 190 (4.4) | 418 (9.7) |
| No Metabolic Syndrome | 4,346 (64.0) | 245 (5.6) | 155 (3.6) | 347 (8.0) |
| Healthy Lifestyle | 1,397 (28.8) | 84 (6.0) | 56 (4.0) | 116 (8.3) |
Variables presented as n (%).
CHD indicates coronary heart disease; CVD, cardiovascular disease; HsCRP, high sensitivity C-reactive protein; and NT-ProBNP, N-terminal pro-brain natriuretic peptide.
After a mean follow-up of 10.3±2.3 years, 710 CVD events occurred (10.4%), including 498 CHD events (7.3%) and 321 hard CHD events (4.7%). The lowest proportion of CVD events was observed in participants with CAC=0 (4.0%), whereas the highest occurred in those with a normal ABI (9.9%). Participants with CAC=0 also had the lowest CHD and hard CHD event proportions (Table 2).
Based on the mean multivariable-adjusted DLR for the total population, among all negative risk markers CAC=0 showed the best performance, resulting in the greatest pre-test – post-test risk shift (Figure 1). CAC=0 had stable risk-factor adjusted DLRs across diverse clinical situations: 0.36 (0.09) in men (i.e., the pre-test logit function will be multiplied by 0.36, which represents a 64% pre-test – post-test relative reduction) and 0.46 (0.12) in women for CHD events, 0.49 (0.10) in men and 0.59 (0.12) in women for CVD events, and ranged from 0.47 (0.08) in high CVD risk participants to 0.67 (0.09) in low risk participants for CVD events (Tables 3 and S2). CAC=0 was particularly informative in older ages as well as in those with higher pre-test predicted 10-year ASCVD risk (Figures 2 and 3).
Figure 1.
Relationship between pre-test and post-test CVD risk after the knowledge of the negative result of each risk marker. The regression lines display the relationship between the pre-test predicted 10-year ASCVD risk (x axis), and the post-test risk (y axis) after the knowledge of the negative result of each risk marker. A broken back line is displayed as reference (risk shift with no additional testing). Results were obtained by plotting the pre-test and post-test risk based on the DLR of each MESA participant, and then applying a linear fit. CAC indicates coronary artery calcium; CIMT, carotid intima-media thickness; FMD, flow-mediated dilation; ABI, ankle-brachial index; HsCRP, high sensitivity C-reactive protein; BNP, brain natriuretic peptide; and CHD, coronary heart disease.
Table 3.
Mean diagnostic likelihood ratios (SD) of CAC=0 for coronary heart disease and cardiovascular disease events, adjusted for traditional risk factors and 10-year atherosclerotic cardiovascular disease predicted risk.
| ASCVD Risk*Categories |
||||||
|---|---|---|---|---|---|---|
| All | Male | Female | 0 – 5 % | 5 – < 7.5 % | ≥ 7.5 % | |
| All CHD Events | ||||||
| Risk Factors† | 0.41 (0.12) | 0.36 (0.09) | 0.46 (0.12) | 0.54 (0.09) | 0.44 (0.06) | 0.34 (0.07) |
| ASCVD Risk* | 0.35 (0.06) | 0.31 (0.06) | 0.35 (0.06) | 0.39 (0.04) | 0.37 (0.04) | 0.31 (0.06) |
| Hard CHD Events | ||||||
| Risk Factors† | 0.51 (0.12) | 0.46 (0.09) | 0.56 (0.12) | 0.64 (0.09) | 0.54 (0.06) | 0.44 (0.07) |
| ASCVD Risk* | 0.43 (0.07) | 0.38 (0.06) | 0.44 (0.06) | 0.48 (0.04) | 0.46 (0.04) | 0.40 (0.06) |
| All CVD Events | ||||||
| Risk Factors† | 0.54 (0.12) | 0.49 (0.10) | 0.59 (0.12) | 0.67 (0.09) | 0.57 (0.06) | 0.47 (0.08) |
| ASCVD Risk* | 0.48 (0.07) | 0.42 (0.07) | 0.48 (0.07) | 0.52 (0.04) | 0.50 (0.04) | 0.43 (0.07) |
10-year ASCVD risk, calculated using the ACC/AHA 2013 Pooled Cohort Equations.
Age, gender, smoking status, diabetes, total and HDL cholesterol, systolic blood pressure, hypertension medication use and race/ethnicity.
ASCVD indicates atherosclerotic cardiovascular disease; CHD, coronary heart disease; and CVD, cardiovascular disease.
Figure 2.
DLR of CAC=0 for cardiovascular disease events by age and pre-test 10-year atherosclerotic cardiovascular disease risk. Scatterplots of DLR of CAC=0 for CVD events by age (Figure 2A) and pre-test predicted 10-year ASCVD risk (Figure 2B). Best-fit curves (in red) were used to plot the mean DLR for each value of age and pre-test predicted risk. CVD indicates cardiovascular disease; and CAC, coronary artery calcium.
Figure 3.
Risk factor-adjusted DLR of negative risk markers for CVD events in four sample patients. Patient A: 50-year-old Chinese American man, non-smoker, SBP 130mmHg (no medication), non-diabetic, HDL 40mg/dL, total cholesterol 170mg/dL. Patient B: 60-year-old White man, smoker, SBP 135mmHg on medication, non-diabetic, HDL 40mg/dL, total cholesterol 200mg/dL. Patient C: 65-year-old Hispanic woman, current smoker, SBP 160mmHg on medication, non-diabetic, HDL 45mg/dL, total cholesterol 220mg/dL. Patient D: 75-year-old African American man, non-smoker, SBP 145mmHg on medication, non-diabetic, HDL 40mg/dL, total cholesterol 230 mg/dL. CAC indicates coronary artery calcium; CIMT, carotid intima-media thickness; FMD, flow-mediated dilation; ABI, ankle-brachial index; HsCRP, high sensitivity C-reactive protein; BNP, brain natriuretic peptide; and CHD, coronary heart disease.
Other negative tests and clinical features did not revise post-test risk as much as was seen for CAC (Tables 4, S3 and S4). Among them, low CIMT showed the best performance after CAC=0. The pre-test prediction was only slightly changed after the finding of a normal ABI, and the performance of serum biomarkers as negative risk markers was modest. Among clinical features, the absence of any family history of CHD was the most informative negative risk marker (mean risk factor-adjusted DLR for CVD events 0.81 [0.06], for CHD events 0.76 [0.07]).
Table 4.
Mean diagnostic likelihood ratios (SD) of other negative risk markers for coronary heart disease and cardiovascular disease events, adjusted for traditional risk factors and 10-year atherosclerotic cardiovascular disease predicted risk.
| All CHD Events | Hard CHD Events | All CVD Events | ||||
|---|---|---|---|---|---|---|
| Risk Factors* | ASCVD Risk† | Risk Factors* | ASCVD Risk† | Risk Factors* | ASCVD Risk† | |
| Carotid Intima-Media Thickness ≤ 25th Percentile |
0.65 (0.04) | 0.53 (0.03) | 0.78 (0.04) | 0.64 (0.03) | 0.75 (0.04) | 0.61 (0.03) |
| No Carotid Plaque | 0.84 (0.07) | 0.77 (0.06) | 0.88 (0.06) | 0.81 (0.05) | 0.88 (0.06) | 0.81 (0.05) |
| Flow-Mediated Dilation ≥ 5% Change | 0.94 (0.10) | 0.90 (0.08) | 0.86 (0.16) | 0.81 (0.04) | 0.91 (0.15) | 0.87 (0.10) |
| Ankle-Brachial Index > 0.9 – < 1.3 | 0.98 (0.02) | 0.98 (0.01) | 0.97 (0.03) | 0.95 (0.02) | 1.00 (0.02) | 0.99 (0.01) |
| HsCRP < 2mg/L | 0.90 (0.03) | 0.91 (0.02) | 0.98 (0.01) | 0.96 (0.01) | 0.89 (0.04) | 0.87 (0.03) |
| Homocysteine < 10µmol/L | 0.96 (0.02) | 0.90 (0.03) | 0.94 (0.03) | 0.89 (0.03) | 0.96 (0.02) | 0.91 (0.03) |
| NT-ProBNP < 100pg/mL | 0.86 (0.10) | 0.92 (0.04) | 0.79 (0.12) | 0.83 (0.05) | 0.88 (0.09) | 0.90 (0.04) |
| No Microalbuminura | 0.96 (0.05) | 0.95 (0.05) | 0.97 (0.05) | 0.96 (0.04) | 0.97 (0.05) | 0.95 (0.05) |
| No Family History of CHD (any) | 0.76 (0.07) | 0.76 (0.06) | 0.78 (0.08) | 0.77 (0.07) | 0.81 (0.06) | 0.80 (0.05) |
| No Family History of Premature CHD | 0.97 (0.08) | 0.96 (0.06) | 0.99 (0.01) | 0.96 (0.05) | 0.96 (0.07) | 0.95 (0.05) |
| No Metabolic Syndrome | 0.90 (0.08) | 0.85 (0.04) | 0.91 (0.08) | 0.87 (0.03) | 0.91 (0.08) | 0.85 (0.04) |
| Healthy Lifestyle | 0.90 (0.13) | 0.84 (0.04) | 0.87 (0.08) | 0.81 (0.01) | 0.98 (0.16) | 0.90 (0.05) |
Age, gender, smoking status, diabetes, total and HDL cholesterol, systolic blood pressure, hypertension medication use and race/ethnicity.
10-year atherosclerotic ASCVD risk, calculated using the ACC/AHA 2013 Pooled Cohort Equations.
CHD indicates coronary heart disease; CVD, cardiovascular disease; ASCVD, atherosclerotic cardiovascular disease; HsCRP, high sensitivity C-reactive protein; and NT-ProBNP, N-terminal pro-brain natriuretic peptide.
See Figures S1–9 and Tables S5–S6 for examples applying the DLRs for each negative risk marker to additional individual common clinical scenarios. See Tables S7–9 for complete information on all coefficients used to derive the equations for DLR calculations for each outcome in this study.
According to the NRI analyses, CAC=0 also resulted in the largest, most accurate downward risk reclassification. This was consistent regardless of the definition of “reclassification” used (Tables 5, S10 and S11).
Table 5.
Net reclassification improvement for CVD events with the 13 negative risk markers in MESA participants with a pre-test 10-year CVD risk ≥7.5% (N=3,833, including 3,227 participants without events and 606 with events). Reclassification was defined as a post-test risk <7.5%.
| Prevalence of risk marker |
Correct reclassification (non-events) |
Incorrect reclassification (events) |
|||||
|---|---|---|---|---|---|---|---|
| N | % | N | % | N | % | NRI | |
| CAC Score = 0 | 1,291 | 33.6 | 710 | 22.0 | 50 | 8.3 | 13.75 |
| HsCRP < 2 mg/dL | 1,923 | 50.4 | 118 | 3.7 | 12 | 2.0 | 1.68 |
| Homocysteine < 10µmol/L | 2,252 | 58.7 | 87 | 2.7 | 8 | 1.3 | 1.38 |
| NT-ProBNP < 100pg/mL | 1,992 | 62.5 | 93 | 2.9 | 13 | 2.2 | 0.74 |
| No Microalbuminuria | 3,294 | 86.1 | 52 | 1.6 | 4 | 0.7 | 0.95 |
| Ankle-Brachial Index > 0.9 – < 1.3 | 3,079 | 81.4 | 100 | 3.1 | 10 | 1.7 | 1.45 |
| CIMT ≤ 25th Percentile | 482 | 12.7 | 216 | 6.7 | 11 | 1.8 | 4.88 |
| No Carotid Plaque | 1,747 | 46.2 | 211 | 6.5 | 13 | 2.2 | 4.39 |
| FMD ≥ 5% Change | 379 | 24.0 | 41 | 1.3 | 3 | 0.5 | 0.78 |
| No Family History of CHD | 1,970 | 55.4 | 207 | 6.4 | 17 | 2.8 | 3.61 |
| No Family History of Premature CHD | 2,341 | 82.0 | 74 | 2.3 | 9 | 1.5 | 0.81 |
| No Metabolic Syndrome | 2,094 | 54.6 | 52 | 1.6 | 4 | 0.7 | 0.95 |
| Healthy Lifestyle | 738 | 26.6 | 87 | 2.7 | 9 | 1.45 | 1.21 |
CVD indicates cardiovascular disease; NRI, net reclassification improvement; CAC, coronary artery calcium; HsCRP, high sensitivity C-reactive protein; NT-ProBNP, N-terminal pro-brain natriuretic peptide; CIMT, carotid intima-media thickness; FMD, flow mediated dilation; CHD, coronary heart disease.
Discussion
Negative risk markers represent a novel approach to augmenting conventional CVD risk assessment, with implications for identifying individual patients who may be less suitable candidates for preventive pharmacotherapy. Among a wide range of negative risk markers including atherosclerosis imaging techniques, serum biomarkers clinical features, and other tests, CAC=0 resulted in the greatest reduction in post-test risk. The conclusions were consistent across gender and 10-year ASCVD risk categories, and using different baseline multivariable models. Carotid ultrasound imaging with a normal result showed the best performance after CAC=0, whereas the performance of the other negative risk markers was minimal or modest. CAC=0 also yielded the largest, most accurate reclassification of risk to below commonly accepted treatment thresholds. These results may have implications for future risk assessment guidelines, providing evidence for the consideration of tests such as CAC and carotid imaging to determine if a patient actually has low estimated absolute risk despite a higher conventional risk factor score-based risk estimate (i.e. using the Pooled Cohort Equations).
Notwithstanding the notable reductions in CVD events attained with preventive treatments such as statins, the costs, side-effects, and potential for disutility25 of these lifelong therapies when used in patients unlikely to have events has resulted in the development of absolute risk-based algorithms, aimed at allocating preventive pharmacotherapies to those most likely to benefit.1,15 Nevertheless, the marked weight of chronological age in current risk scores has resulted in the consideration of almost every elderly individual as candidate for statin treatment, resembling a “treat-all” strategy for specific demographic groups.
Atherosclerosis imaging techniques have a high sensitivity for detecting the presence of disease. This therefore translates into a very high negative predictive value, making these tools particularly appealing for ruling out prevalent disease.26 The presence of atherosclerosis in vascular beds such as the coronary and carotid arteries is a strong predictor of CVD events,27,28 and accordingly, the absence of disease is associated with a markedly reduced risk.8,9
In this study, CAC=0 resulted in the greatest reduction in post-test risk, and in the most accurate downward risk reclassification, followed by a low CIMT. Among presently available advanced prognostic tests, CAC is considered the best tool for CVD risk assessment,15,29 improving risk predictions across age and score-based risk categories.30,31 Even though much attention has focused on the ability of CAC to improve detection of high risk beyond traditional risk factors, in the last few years population-based studies and large clinical cohorts have expanded our understanding regarding the excellent prognosis of asymptomatic individuals with a negative CAC result.9 Our results highlight the fact that most individuals with a pre-rest risk estimate of up to 15% will be expected to have a post-test risk estimate of approximately <7.5% after a CAC=0 result. These findings have distinct implications for future attempts to conceptualize an “intermediate risk” group guiding appropriate use of negative risk markers in clinical practice.
Regarding low CIMT, a recent study suggested that one of the key uses of carotid ultrasound imaging may be for down-classifying risk in subjects unlikely to have events.32 Our findings support the use of these imaging tests as tools for detecting low-risk individuals among subgroups who would otherwise qualify for preventive pharmacotherapies according to a traditional risk-score approach. Use of additional tests may be particularly helpful when the high predicted risk is exclusively or predominantly driven by chronological age, in individuals at increased odds of drug-drug interactions, and in patients who express reservations about taking statins long-term.
The performance of some negative risk markers in this study was minimal or modest. We noted a limited post-test risk change/reclassification for hsCRP, which was consistent regardless of the cut-point chosen for a negative result. HsCRP is supported by current guidelines as a tool for further cardiovascular risk assessment in individuals in which management is uncertain,15 even though its value beyond traditional risk factors for detecting high risk is considered limited.33 Our findings suggest that the performance of hsCRP as a tool for reassuring the patient and the clinician, when the test result is normal or low, is also limited.
The absence of any family history of CHD was the strongest among a number of “negative” clinical markers. Thus, even though any family history of CHD is often considered in clinical practice as a “positive” risk marker, our findings suggest that its absence may be helpful for detecting lower CVD risk. This may be particularly informative in older individuals and in those with large families,34 as more “person-years” free of CHD within a family will be captured. On the other hand, absence of family history of premature CHD performed poorly as a negative risk marker, and may be more helpful for detecting increased CVD risk when present.15
The potential role of negative results of a number of risk markers commonly used in clinical practice had not yet been studied in a single, large, population-based study. Previous studies assessed the performance of some of these markers.35,36 However, comparisons were focused on the ability for detecting high risk, or on the overall performance of the tests throughout their range of values. Moreover, by comparing exclusively areas under the receiver operating characteristic (ROC) curve or NRIs, those studies could not address the actual ability of the tests for impacting individual-level risk estimates.24,37 The DLR approach that we used provides an opportunity for quantifying the magnitude of change in predicted risk before and after knowing the result of a diagnostic test - an approach that emulates Bayesian analysis, providing more intuitive estimates for the clinician decision-maker. By accounting for the values of other risk factors, the DLRs provide insight on the information that the negative result of a test provides in specific clinical scenarios for individual patients.
Study limitations
Some limitations of our study must be acknowledged. First, 10-year ASCVD risk was calculated using the original ACC/AHA 2013 Pooled Cohort Equations.15 However, for precise risk assessment in MESA the equations would need to be recalibrated, otherwise ASCVD risks are likely to be overestimated. Thus for our primary analysis we adjusted for individual risk factors and not the Pooled Cohort Equations. Second, FMD measurements were available in only 44% of the study participants. Nevertheless, when missing FMD values were imputed, the relative strength of FMD as a negative risk marker remained the same. Finally, the calculation of DLRs, although well described in the statistical literature and ideally suited for a binary exposures such as negative risk markers, is relatively new to the cardiovascular literature. Formal comparison testing of DLRs, which are covariate-specific, is less well described. However, the observed DLRs for CAC=0 were consistent across gender and ASCVD risk categories, and were consistently different from those of the other negative risk markers, reinforcing our conclusions. Moreover, the relative strength of the negative risk markers was consistent with that observed using NRIs.
Conclusion
Among a wide range of tests and clinical features, CAC=0, and to a lesser extent a normal carotid ultrasound, were associated with the lowest DLR and, therefore, resulted in the greatest post-test downward shift risk. CAC=0 also resulted in the largest, most accurate downward reclassification of estimated risk. Although this specific use of prognostic tests for down-grading risk estimates (i.e. negative risk markers) was not discussed in the latest ACC/AHA risk assessment guidelines, in the current context of population aging and concern for overtreatment, the clinical and public health implications of these findings could be substantial.
Supplementary Material
Clinical Perspectives.
Clinical cardiovascular disease (CVD) risk assessment has predominantly focused on the identification of individuals at high absolute CVD risk. This holds true for non-traditional risk markers including novel risk assessment technologies, where the literature mostly describes their use to detect additional patients who may benefit from preventive pharmacotherapies. However, in the context of the new ACC/AHA prevention guidelines – where population aging, possible risk overestimation, potential overtreatment of older individuals with few risk factors, and overmedicalization are concerns – the accurate identification of individuals that are truly low-risk may have even greater clinical, economic, and public health implications. For this purpose, we compared the ability of 13 “negative risk markers” widely used in clinical practice to modify 10-year coronary heart disease and CVD risk estimates, and tested their accuracy for reclassifying individuals below the treatment threshold, in a large, multi-ethnic, contemporary cohort. Among these markers, negative results of atherosclerosis imaging tests, particularly a coronary artery calcium score of zero, resulted in the greatest downward shift in risk estimation, and in the largest net risk reclassification. These findings have direct implications for identification of individuals less likely to receive net benefit from statins and other preventive pharmacotherapies. Moreover, given the lack of guidance in current guidelines for downwardly reclassifying risk, our findings could be used to inform guideline updates. The next research steps should be the definition of which “high-risk” patients should be tested for presence of negative risk markers, and the evaluation of the cost-effectiveness of different “de-risking” strategies.
Acknowledgments
The authors thank the other investigators, the staff, and the participants of the MESA study for their valuable contributions. A full list of participating MESA investigators and institutions can be found at http://www.mesa-nhlbi.org.
Funding Sources: This research was supported by contracts N01-HC-95159, N01-HC-95160, N01-HC-95161, N01-HC-95162, N01-HC-95163, N01- HC-95164, N01-HC-95165, N01-HC-95166, N01-HC-95167, N01- HC-95168 and N01-HC-95169 from the NHLBI and by grants UL1-TR-000040 and UL1-TR-001079 from NCRR. MC-A was funded by a research grant from the Spanish Society of Cardiology.
Footnotes
Disclosures: None.
References
- 1.27th Bethesda Conference. Matching the Intensity of Risk Factor Management with the Hazard for Coronary Disease Events. September 14–15, 1995. J Am Coll Cardiol. 1996;27:957–1047. [PubMed] [Google Scholar]
- 2.World Health Organization. Global Status Report on Non-communicable Diseases 2010. Geneva, Switzerland: World Health Organization; 2011. [Google Scholar]
- 3.Kildemoes HW, Andersen M, Støvring H. The impact of ageing and changing utilization patterns on future cardiovascular drug expenditure: a pharmacoepidemiological projection approach. Pharmacoepidemiol Drug Saf. 2010;19:1276–1286. doi: 10.1002/pds.2039. [DOI] [PubMed] [Google Scholar]
- 4.Karmali KN, Goff DC, Jr, Ning H, Lloyd-Jones DM. A systematic examination of the 2013 ACC/AHA pooled cohort risk assessment tool for atherosclerotic cardiovascular disease. J Am Coll Cardiol. 2014;64:959–968. doi: 10.1016/j.jacc.2014.06.1186. [DOI] [PubMed] [Google Scholar]
- 5.Pencina MJ, Navar-Boggan AM, D’Agostino RB, Sr, Williams K, Neely B, Sniderman AD, Peterson ED. Application of new cholesterol guidelines to a population-based sample. N Engl J Med. 2014;370:1422–1431. doi: 10.1056/NEJMoa1315665. [DOI] [PubMed] [Google Scholar]
- 6.Stone NJ, Robinson JG, Lichtenstein AH, Bairey Merz CN, Blum CB, Eckel RH, Goldberg AC, Gordon D, Levy D, Lloyd-Jones DM, McBride P, Schwartz JS, Shero ST, Smith SC, Jr, Watson K, Wilson PW ACC/AHA Task Force on Practice Guidelines. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults. J Am Coll Cardiol. 2014;63:2889–2934. doi: 10.1016/j.jacc.2013.11.002. [DOI] [PubMed] [Google Scholar]
- 7.Ridker PM, Cook NR. Statins: new American guidelines for prevention of cardiovascular disease. Lancet. 2013;382:1762–1765. doi: 10.1016/S0140-6736(13)62388-0. [DOI] [PubMed] [Google Scholar]
- 8.Hecht HS. A zero coronary artery calcium score: priceless. J Am Coll Cardiol. 2010;55:1118–1120. doi: 10.1016/j.jacc.2009.09.064. [DOI] [PubMed] [Google Scholar]
- 9.Sarwar A, Shaw LJ, Shapiro MD, Blankstein R, Hoffmann U, Cury RC, Abbara S, Brady TJ, Budoff MJ, Blumenthal RS, Nasir K. Diagnostic and prognostic value of absence of coronary artery calcification. JACC Cardiovasc Imaging. 2009;2:675–688. doi: 10.1016/j.jcmg.2008.12.031. [DOI] [PubMed] [Google Scholar]
- 10.Janssens AC, Deng Y, Borsboom GJ, Eijkemans MJ, Habbema JD, Steyerberg EW. A new logistic regression approach for the evaluation of diagnostic test results. Med Decis Making. 2005;25:168–177. doi: 10.1177/0272989X05275154. [DOI] [PubMed] [Google Scholar]
- 11.Gu W, Pepe MS. Estimating the capacity for improvement in risk prediction with a marker. Biostatistics. 2009;10:172–186. doi: 10.1093/biostatistics/kxn025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Bild DE, Bluemke DA, Burke GL, Detrano R, Diez Roux AV, Folsom AR, Greenland P, Jacob DR, Jr, Kronmal R, Liu K, Nelson JC, O’Leary D, Saad MF, Shea S, Szklo M, Tracy RP. Multi-ethnic study of atherosclerosis: objectives and design. Am J Epidemiol. 2002;156:871–881. doi: 10.1093/aje/kwf113. [DOI] [PubMed] [Google Scholar]
- 13.Grundy SM, Brewer HB, Jr, Cleeman JI, Smith SC, Jr, Lenfant C American Heart Association; National Heart, Lung, Blood Institute. Definition of metabolic syndrome. Circulation. 2004;109:433–438. doi: 10.1161/01.CIR.0000111245.75752.C6. [DOI] [PubMed] [Google Scholar]
- 14.Bertoni AG, Whitt-Glover MC, Chung H, Le KY, Barr RG, Mahesh M, Jenny NS, Burke GL, Jacobs DR. The association between physical activity and subclinical atherosclerosis: the Multi-Ethnic Study of Atherosclerosis. Am J Epidemiol. 2009;169:444–454. doi: 10.1093/aje/kwn350. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Goff DC, Jr, Lloyd-Jones DM, Bennett G, Coady S, D’Agostino RB, Gibbons R, Greenland P, Lackland DT, Levy D, O’Donnell CJ, Robinson JG, Schwartz JS, Shero ST, Smith SC, Jr, Sorlie P, Stone NJ, Wilson PW, Jordan HS, Nevo L, Wnek J, Anderson JL, Halperin JL, Albert NM, Bozkurt B, Brindis RG, Curtis LH, DeMets D, Hochman JS, Kovacs RJ, Ohman EM, Pressler SJ, Sellke FW, Shen WK, Smith SC, Jr, Tomaselli GF ACC/AHA Task Force on Practice Guidelines. 2013 ACC/AHA guideline on the assessment of cardiovascular risk. Circulation. 2014;129(25 Suppl 2):S49–S73. doi: 10.1161/01.cir.0000437741.48606.98. [DOI] [PubMed] [Google Scholar]
- 16.Sharma K, Blaha MJ, Blumenthal RS, Musunuru K. Clinical and research applications of carotid intima-media thickness. Am J Cardiol. 2009;103:1316–1320. doi: 10.1016/j.amjcard.2009.01.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Bots ML, Westerink J, Rabelink TJ, de Koning EJ. Assessment of flow-mediated vasodilatation (FMD) of the brachial artery: effects of technical aspects of the FMD measurement on the FMD response. Eur Heart J. 2005;26:363–368. doi: 10.1093/eurheartj/ehi017. [DOI] [PubMed] [Google Scholar]
- 18.Tison GH, Ndumele CE, Gerstenblith G, Allison MA, Polak JF, Szklo M. Usefulness of baseline obesity to predict development of a high ankle brachial index (from the Multi-Ethnic Study of Atherosclerosis) Am J Cardiol. 2011;107:1386–1391. doi: 10.1016/j.amjcard.2010.12.050. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Ridker PM, Danielson E, Fonseca FA, Genest J, Gotto AM, Jr, Kastelein JJ, Koenig W, Libby P, Lorenzatti AJ, MacFadyen JG, Nordestgaard BG, Shepherd J, Willerson JT, Glynn RJ JUPITER Study Group. Rosuvastatin to prevent vascular events in men and women with elevated C-reactive protein. N Engl J Med. 2008;359:2195–2207. doi: 10.1056/NEJMoa0807646. [DOI] [PubMed] [Google Scholar]
- 20.Veeranna V, Zalawadiya SK, Niraj A, Pradhan J, Ference B, Burack RC, Jacob S, Afonso L. Homocysteine and reclassification of cardiovascular disease risk. J Am Coll Cardiol. 2011;58:1025–1033. doi: 10.1016/j.jacc.2011.05.028. [DOI] [PubMed] [Google Scholar]
- 21.Sanchez OA, Duprez DA, Daniels LB, Maisel AS, Otvos JD, Peralta CA, Lima JA, Bahrami H, Jacobs DR., Jr The association between N-terminal pro B-type natriuretic peptide and lipoprotein particle concentration plateaus at higher N-terminal pro B-type natriuretic peptide values: Multi-Ethnic Study on Atherosclerosis. Metabolism. 2015;64:857–861. doi: 10.1016/j.metabol.2015.04.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Toto RD. Microalbuminuria: definition, detection, and clinical significance. J Clin Hypertens (Greenwich) 2004;6(11 Suppl 3):2–7. doi: 10.1111/j.1524-6175.2004.4064.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Ahmed HM, Blaha MJ, Nasir K, Jones SR, Rivera JJ, Agatston A, Blankstein R, Wong ND, Lakoski S, Budoff MJ, Burke GL, Sibley CT, Ouyang P, Blumenthal RS. Low-risk lifestyle, coronary calcium, cardiovascular events, and mortality: results from MESA. Am J Epidemiol. 2013;178:12–21. doi: 10.1093/aje/kws453. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Cook NR. Use and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction. Circulation. 2007;115:928–935. doi: 10.1161/CIRCULATIONAHA.106.672402. [DOI] [PubMed] [Google Scholar]
- 25.Fontana M, Asaria P, Moraldo M, Finegold J, Hassanally K, Manisty CH, Francis DP. Patient-accessible tool for shared decision making in cardiovascular primary prevention: balancing longevity benefits against medication disutility. Circulation. 2014;129:2539–2546. doi: 10.1161/CIRCULATIONAHA.113.007595. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Blaha MJ, Silverman MG, Budoff MJ. Is there a role for coronary artery calcium scoring for management of asymptomatic patients at risk for coronary artery disease? Circ Cardiovasc Imaging. 2014;7:398–408. doi: 10.1161/CIRCIMAGING.113.000341. [DOI] [PubMed] [Google Scholar]
- 27.Detrano R, Guerci AD, Carr JJ, Bild DE, Burke G, Folsom AR, Liu K, Shea S, Szklo M, Bluemke DA, O’Leary DH, Tracy R, Watson K, Wong ND, Kronmal RA. Coronary calcium as a predictor of coronary events in four racial or ethnic groups. N Engl J Med. 2008;358:1336–1345. doi: 10.1056/NEJMoa072100. [DOI] [PubMed] [Google Scholar]
- 28.Lorenz MW, Markus HS, Bots ML, Rosvall M, Sitzer M. Prediction of clinical cardiovascular events with carotid intima-media thickness: a systematic review and meta-analysis. Circulation. 2007;115:459–467. doi: 10.1161/CIRCULATIONAHA.106.628875. [DOI] [PubMed] [Google Scholar]
- 29.Peters SAE, den Ruijter HM, Bots ML, Moons KG. Improvements in risk stratification for the occurrence of cardiovascular disease by imaging subclinical atherosclerosis: a systematic review. Heart. 2012;98:177–184. doi: 10.1136/heartjnl-2011-300747. [DOI] [PubMed] [Google Scholar]
- 30.Tota-Maharaj R, Blaha MJ, McEvoy JW, Blumenthal RS, Muse ED, Budoff MJ, Shaw LJ, Berman DS, Rana JS, Rumberger J, Callister T, Rivera J, Agatston A, Nasir K. Coronary artery calcium for the prediction of mortality in young adults <45 years old and elderly adults >75 years old. Eur Heart J. 2012;33:2955–2962. doi: 10.1093/eurheartj/ehs230. [DOI] [PubMed] [Google Scholar]
- 31.Silverman MG, Blaha MJ, Krumholz HM, Budoff MJ, Blankstein R, Sibley CT, Agatston A, Blumenthal RS, Nasir K. Impact of coronary artery calcium on coronary heart disease events in individuals at the extremes of traditional risk factor burden: the Multi-Ethnic Study of Atherosclerosis. Eur Heart J. 2014;35:2232–2241. doi: 10.1093/eurheartj/eht508. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Gardin JM, Bartz TM, Polak JF, O’Leary DH, Wong ND. What do carotid intima-media thickness and plaque add to the prediction of stroke and cardiovascular disease risk in older adults? The cardiovascular health study. J Am Soc Echocardiogr. 2014;27:998–1005. doi: 10.1016/j.echo.2014.06.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Yousuf O, Mohanty BD, Martin SS, Joshi PH, Blaha MJ, Nasir K, Blumenthal RS, Budoff MJ. High-sensitivity C-reactive protein and cardiovascular disease: a resolute belief or an elusive link? J Am Coll Cardiol. 2013;62:397–408. doi: 10.1016/j.jacc.2013.05.016. [DOI] [PubMed] [Google Scholar]
- 34.Ciampi A, Courteau J, Niyonsenga T, Xhignesse M, Lussier-Cacan S, Roy M. Family history and the risk of coronary heart disease: comparing predictive models. Eur J Epidemiol. 2001;17:609–620. doi: 10.1023/a:1015587428172. [DOI] [PubMed] [Google Scholar]
- 35.Kavousi M, Elias-Smale S, Rutten JH, Leening MJ, Vliegenthart R, Verwoert GC, Krestin GP, Oudkerk M, de Maat MP, Leebeek FW, Mattace-Raso FU, Lindemans J, Hofman A, Steyerberg EW, van der Lugt A, van den Meiracker AH, Witteman JC. Evaluation of newer risk markers for coronary heart disease risk classification: a cohort study. Ann Intern Med. 2012;156:438–444. doi: 10.7326/0003-4819-156-6-201203200-00006. [DOI] [PubMed] [Google Scholar]
- 36.Simon A, Chironi G, Levenson J. Comparative performance of subclinical atherosclerosis tests in predicting coronary heart disease in asymptomatic individuals. Eur Heart J. 2007;28:2967–2971. doi: 10.1093/eurheartj/ehm487. [DOI] [PubMed] [Google Scholar]
- 37.Pepe MS, Feng Z, Gu JW. Comments on ‘Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification, beyond’ by M.J. Pencina et al., Statistics in Medicine. Stat Med. 2008;27:173–181. doi: 10.1002/sim.2991. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.