Improved coronary risk assessment among intermediate risk patients using a clinical and biomarker based algorithm developed and validated in two population cohorts

Abstract

Background

Many coronary heart disease (CHD) events occur in individuals classified as intermediate risk by commonly used assessment tools. Over half the individuals presenting with a severe cardiac event, such as Myocardial Infarction (MI), have at most one risk factor as included in the widely used Framingham risk assessment. Individuals classified as intermediate risk, who are actually at high risk, may not receive guideline recommended treatments. A clinically useful method for accurately predicting 5-year CHD risk among intermediate risk patients remains an unmet medical need.

Objective

This study sought to develop a CHD Risk Assessment (CHDRA) model that improves 5-year risk stratification among intermediate risk individuals.

Methods

Assay panels for biomarkers associated with atherosclerosis biology (inflammation, angiogenesis, apoptosis, chemotaxis, etc.) were optimized for measuring baseline serum samples from 1084 initially CHD-free Marshfield Clinic Personalized Medicine Research Project (PMRP) individuals. A multivariable Cox regression model was fit using the most powerful risk predictors within the clinical and protein variables identified by repeated cross-validation. The resulting CHDRA algorithm was validated in a Multiple-Ethnic Study of Atherosclerosis (MESA) case-cohort sample.

Results

A CHDRA algorithm of age, sex, diabetes, and family history of MI, combined with serum levels of seven biomarkers (CTACK, Eotaxin, Fas Ligand, HGF, IL-16, MCP-3, and sFas) yielded a clinical net reclassification index of 42.7% (p<0.001) for MESA patients with a recalibrated Framingham 5-year intermediate risk level. Across all patients, the model predicted acute coronary events (hazard ratio=2.17, p<0.001), and remained an independent predictor after Framingham risk factor adjustments.

Limitations

These include the slightly different event definition with the MESA samples and inability to include PMRP fatal CHD events.

Conclusions

A novel risk score of serum protein levels plus clinical risk factors, developed and validated in independent cohorts, demonstrated clinical utility for assessing the true risk of CHD events in intermediate risk patients. Improved accuracy in cardiovascular risk classification could lead to improved preventive care and fewer deaths.

Introduction

Coronary heart disease (CHD) remains a major cause of morbidity and mortality despite recent improvements in disease management^{1, 2}. A critical requirement for reducing CHD is to accurately identify individuals with subclinical disease who are at risk of experiencing a coronary event, and for whom early intervention can help. Guidelines recommend formal risk stratification based on clinical characteristics such as the Framingham risk score to calculate 10-year risk for patients ^{3, 4}. However, models using established risk factors do not fully estimate the incidence and prevalence of cardiovascular disease in the general population ^5–7. Furthermore, fewer than 20% of surveyed physicians report using a risk calculator, and most physicians misclassify patient risk; nearly two-thirds underestimate risk ^{8, 9}.

Common risk assessment tools place many patients into an intermediate risk category where treatment guidelines are unclear and further workup is required ³. Efforts to refine risk assessment for intermediate patients typically have added biomarkers to existing risk algorithms, yet individual biomarkers have correlated poorly with relevant clinical events, and have seldom been validated in independent cohorts^10–12. Inadequate biomarker analytical performance may partially explain the failures¹³. Even established biomarkers such as low density lipoprotein (LDL) fail to fully explain CHD risk with nearly half of myocardial infarctions (MI) occurring in patients with normal lipids¹⁴. Attempts at combining multiple biomarkers have shown variable success, perhaps because such studies have addressed all-comers including those classified at low or high 10-year risk by conventional assessments^{15, 16}. Focusing on the intermediate risk population, and considering a 5-year risk horizon may be more useful and also help motivate therapy compliance as suggested in a study of patient and physician preferences for risk timeframes.¹⁷

We developed and optimized quantitative assays for serum proteins related to inflammation, apoptosis, thrombosis, vascular remodeling, and other processes previously shown to underlie CHD development ^{18, 19}. We used these customized assays to measure the biomarker levels in a population-based cohort initially CHD-free to identify the optimum 5-year CHD Risk Assessment (CHDRA) model for patients classified as intermediate risk by current risk algorithms. A 5-year time frame was selected to balance the need for sufficient events to occur in the population being studied with the desire for a more immediate risk interval, as suggested in a study of patient and physician preferences.¹⁷ Validation of the model was assessed by the correct classification of intermediate risk individuals in the independent, prospective, Multiple-Ethnic Study of Atherosclerosis (MESA).

Methods

Study population for model derivation: PMRP

Study participants for model derivation were from the Marshfield Clinic Personalized Medicine Research Project (PMRP), a population-based sample repository collected in 2002 to 2004 from Marshfield, Wisconsin residents to study genetic epidemiology and pharmacogenomics ²⁰. The PMRP serum bio-bank contained pristine samples collected under controlled preanalytical processes and stored at −80C. The PMRP represents a relatively homogeneous population of Northern European ancestry. Participants provided broad written informed consent for research use of samples and medical records. The Marshfield Clinic Research Foundation Institutional Review Board approved the protocol. Clinical data came from adjudicated medical records and enrollment questionnaires that were geared toward a genetic study focus for which the project was initially designed. Self-reported family history of CHD was defined as an MI or angina in two or more first-degree relatives without reference to the relative’s age at occurrence. Repository blood samples were selected from 40 to 80 year old individuals with no history of cardiovascular disease at enrollment when the blood samples were collected. The eligible cohort included 10,623 individuals. Using a modified case-cohort type design, baseline samples from all individuals who subsequently had an event (n=385) within the 5-years after enrollment were selected, along with baseline samples from 838 individuals randomly selected from those without events during follow-up (n=10238), resulting in a sampling fraction of 8.19% (838/10238). A case-cohort type design was used, rather than a case-control study matching age and sex, so that the CHDRA model could be applied to all individuals using their age and sex as input variables. An event was defined as a first non-fatal, acute MI identified in the medical records as the primary diagnosis ICD-9 codes 410.xx and 412.xx (n=164) or hospitalization for unstable angina, code 411.1 (n=221), up to five years after enrollment. All fatal events were excluded because the cause of death could not be unambiguously confirmed. Baseline lipid levels were measured in the sample or obtained from medical records if already measured within 1 year of the sample collection.

Study population for model validation: MESA

Model validation study participants were from the Multi-Ethnic Study of Atherosclerosis (MESA), a prospective study of subclinical cardiovascular disease prevalence and progression ²¹. The MESA population had 6,814 participants 45–84 years old, self-identified as White, African-American, Hispanic, or Chinese. All participants were free of cardiovascular disease at study entry (2000–2002) and gave informed consent, although 1,083 denied use of their samples for commercial purposes. The serum samples were collected at baseline under controlled preanalytical processes and stored at −70C until tested. Weighted average follow-up time for the cohort was 5.4 years (minimum 5 years). All incident CHD events (n=222) from adjudicated medical records were used. These included definite or probable MI (either abnormal cardiac biomarkers, evolving Q waves, or a combination of chest pain with ST-T evolution or new left bundle branch block ECG changes), resuscitated cardiac arrest, fatal CHD (MI within the prior 28 days, chest pain within 72 hr, or a history of CHD, and no known non-cardiac cause of death), definite angina (chest pain symptoms with 70% or greater obstruction on coronary angiography, or evidence of ischemia by stress test or resting ECG), and probable angina (physician diagnosis) if accompanied by revascularization. The case-cohort sample of 623 participants was selected at random giving a sampling fraction of 11.1%. This random selection included 21 of the 222 who had an event during follow-up, so the remaining 201 events were added to result in 824 participants: all 222 with events and 602 without events.

Protein Biomarker Assays

The protein biomarkers tested with the PMRP samples (Figure 1) were putative CHD-related proteins identified in prior work and literature reviews ¹⁹. The list was refined based on antibody availability and what could be tested reliably in xMAP® (Luminex Corp, Austin, TX) and MULTI-SPOT® (MSD, Gaithersburg, MD) multiplexed panels (See Appendix Supplementary methods for details). Thirty-two biomarker assays remained after dropping those exhibiting poor analytical performance (sensitivity, specificity, and reproducibility). The MESA samples were tested and analyzed with no knowledge of their event/non-event status that was only revealed by the MESA committee following submission of the CHDRA scores.

Figure 1.

List of 32 biomarkers tested for model derivation and their associated biological pathways.

Statistical Analysis

The CHDRA model was derived with 1084 PMRP samples (362 events) for which all clinical data and protein biomarker values could be obtained out of the 1,223 samples (385 events) tested. The number of samples from individuals without events was based on power calculations for observing Hazard Ratios (HR) >1.3 for a CHD event in 5 years ²². Statistical analyses were conducted with R ²³. A Cox proportional hazard model determined the univariate associations of the log10 transformed protein biomarker levels with 5-year cardiovascular events in the PMRP population ²⁴. Coefficients were calculated for 1-SD increment of the biomarker distributions in the individuals without events. Models were evaluated both unadjusted and adjusted for age, sex, blood pressure, total cholesterol, high–density lipoprotein cholesterol (HDL), hypertension, antihypertensive medication use, hyperlipidemia diagnosis, diabetes, and smoking status. Those variables were selected from the Framingham and ATP-III models to maintain a consistency with the variables in the more familiar risk assessment models used by clinicians ^{4, 25}. The function survfit (R package: survival) was used to compute the predicted survivor function for the Cox proportional hazards model and was modified to account for the event/non-event weights required in the case-cohort analysis and to estimate the baseline survivor function for the population and not the sample ²³.

The number of protein biomarkers in the final model was determined with a forward selection methodology while the CHDRA model was obtained by fitting a weighted Cox proportional hazard model to all data, using a forward variable selection method to choose from the measured protein biomarkers and clinical risk factors ²⁶.

Algorithm pre-validation

Before validating the CHDRA with the MESA population, a pre-validation was performed within the PMRP samples to prevent over fitting ²⁷. The pre-validation process was repeated 10 times to obtain a CHDRA performance estimate for comparison against two reference models. The first reference model was a Framingham risk model for CHD events (angina, MI, coronary insufficiency, and CHD death) using the published Framingham risk model coefficients for age, sex, total cholesterol, HDL cholesterol, systolic and diastolic blood pressures, diabetes, and smoking status, calibrated for 5-year risks ⁴. The 5-year calibration transformed the typical Framingham risk model 10-year risk categories of Low: 0–10%, Intermediate: 10–20%, and High: >20% risks, into 5-year equivalents of Low: 0 to <3.5%, Intermediate: 3.5 to <7.5%, and High: >7.5% risk. The risk thresholds of 3.5% for intermediate risk and 7.5% for high risk were the closest pragmatic cut-offs providing concordance of the predicted rates to the PMRP population observed event rates for men and women within each Framingham risk category mapped to a 5-year time frame. Such lower category thresholds are consistent with lower baseline event rates observed in populations more modern than the Framingham cohort ⁴. The second reference was a nine clinical risk factors model containing the Framingham risk factors plus antihypertensive medication use. The Framingham risk model represents a familiar gold standard, while the 9-clinical risk factor model captures a variable often considered in risk assessments ²⁵. The same risk thresholds of 3.5% for intermediate risk and 7.5% for high risk were used for all models to maintain consistency.

Algorithm performance comparisons

The Net Reclassification Index (NRI) and the clinical Net Reclassification Index (clinical NRI) were used for comparing the CHDRA to the Framingham risk and 9-clinical risk factor reference models ²⁸. The NRI indicates the overall improvement in correct classification by the CHDRA model for all risk categories, while the clinical NRI describes the improvement in classification of just those individuals assigned as intermediate risk by the reference model. It indicates the net improvement of correctly reclassifying those individuals who had an event to the high risk category and those who did not experience an event to the low risk category. The 9–clinical risk factor model used coefficients developed with the PMRP cohort, while the Framingham risk model used the published Framingham risk coefficients calibrated for a 5-year follow-up as described. The risk categorization for an individual was defined with the reference model (adjusted for cohort weights and accounting for missing samples). The association of the CHDRA risk score (log-10 transformed) and CHD events (n=179) in MESA was determined with the Cox proportional hazard model as hazard ratio (HR) in units of 1-SD of the score distribution for individuals without events (n=495). Additional models adjusted for 1) the Framingham risk model, 2) the Framingham risk model plus diabetes, 3) the Framingham risk model plus family history, and 4) the 9-clinical risk factor covariates plus race/ethnicity. These adjustments were chosen to identify independent relationships of the CHDRA with common CHD risk assessment variables and to test its incremental benefit in risk prediction. Although the primary focus was on performance of the CHDRA model as improving risk assessment for the intermediate risk individuals, a continuous NRI analysis and an integrated discrimination analysis were completed to assess performance relative to the Framingham risk model^{29, 30}.

The predictive power of only the CHDRA seven proteins was determined by fitting two additional models in the PMRP sample set: 1) the Framingham clinical risk factors plus family history of MI, and 2) those same variables plus the CHDRA seven proteins. The two fitted models were then tested in the MESA set and clinical NRI values calculated.

Finally, the incremental effect of the CHDRA on discrimination of risk for CHD events in Framingham intermediate risk individuals was determined by computing the area under the curve (AUC) of the receiver-operating characteristic curve for both the CHDRA and Framingham risk models for the 3.5 to <7.5% risk group. The CHDRA Harrell C-statistic, accounting for case-cohort design weights, was determined for both the intermediate risk group and all individuals ³¹.

Results

Sample baseline characteristics

The baseline characteristics of the PMRP (training) and MESA (validation) samples were different (Table 1). Individuals in the PMRP cohort were younger (56.4 years) than in MESA (62 years, p<0.001). The PMRP cohort included 99% Caucasian individuals whereas MESA was multi-race and multi-ethnic (42% Caucasian individuals). The PMRP cohort contained more female participants without events compared to MESA (p=0.008). That difference is consistent with reported trends¹. The proportion of current smokers and LDL levels were comparable in the two populations (p=0.85 for individuals without events and p=0.85 for those with events) while the LDL-C levels were comparable in those with events (p=0.178) and higher for those without events in the PMRP set (p=0.013). Total cholesterol levels (p<0.001), statin use (p < 0.05) and proportion of diabetics (p<0.001) were all higher in the PMRP cohort.

Table 1.

Demographics for PMRP and MESA Samples Tested

	PMRP	MESA
Age, mean (SD) [range], years
Without Events	56.4 (10.9) [40–80]	62.0 (10.0) [44–84]
With Events	63.7 (10.6) [40–80]	67.6 (9.2) [48–84]
Men, No. (%)	502 (46)	361 (53)
Without Events	291 (40)	242 (49)
With Events	211 (58)	119 (66)
Race/Ethnicity No. (%)
Caucasian	1076 (99)	286 (42)
Asian	2 (0)	81 (12)
African American	2 (0)	151 (22)
Hispanic	4 (0)	157 (23)
Blood Pressure, mean (SD) [range], mm Hg
Systolic
Without Events	127.3 (16.7) [80–196]	125.7 (21.4) [77–218]
With Events	136.1 (21.8) [76–227]	134.8 (24.0) [81–191]
Diastolic
Without Events	75.7 (9.2) [42–112]	71.7 (10.2) [41–107]
With Events	75.6 (10.8) [42–108]	73.3 (11.4) [49–108]
Cholesterol, mean (SD) [range], mg/dL
Total
Without Events	209.4 (40.7) [87–343]	192.4 (35.0) [85–456]
With Events	214.9 (40.5) [88–339]	201.2 (41.2) [96–408]
LDL–C
Without Events	122.7 (35.1) [29–266]	116.9 (32.1) [25–246]
With Events	120.6 (35.3) [40–268]	124.4 (34.0) [42–315]
HDL–C
Without Events	46.9 (14.6) [13–128	50.1 (13.9) [21–120]
With Events	42.9 (14.3) [18–102]	46.3 (13.4) [24–103]
Triglycerides, mean (SD) [range], mg/dL
Without Events	139.7 (82.9) [21–936]	127.8 (63.8) [27–551]
With Events	172.9 (117.8) [30–1129]	148.8 (77.7) [27–840]
Diabetes, No. (%)
Without Events	141 (20)	57 (11)
With Events	158 (44)	42 (23)
Family History of MI, No. (%)
Without Events	150 (21)	214 (43)
With Events	128 (35)	101 (56)
Current Smoker, No. (%)
Without Events	85 (12)	61 (12)
With Events	47 (13)	25 (14)
Statin Use, No. (%)
Without Events	149 (21)	77 (16)
With Events	130 (36)	37 (21)
Antihypertensive Medication, No. (%)
Without Events	189 (26)	177 (36)
With Events	148 (41)	95 (53)

PMRP = Personalized Medicine Research Project; MESA = Multiple-Ethnic Study of Atherosclerosis.

CHDRA Derivation

In the univariate analysis, 19 of the reliably measured biomarkers had statistically significant association with event risk (HR, 95% confidence intervals above or below 1.0, p<0.05, Figure 2a). When adjusted for clinical risk factors, 11 biomarkers maintained statistical significance (Figure 2b). (See Appendix Supplemental Tables 1 and Supplemental Tables 2 for HR point estimates and 95% confidence intervals.) The forward selection methodology identified seven proteins as the smallest model size that was within 1-standard error of the optimum fit log-likelihood ratio. The Cox proportional hazard analysis, allowed to choose up to 7 proteins from the 32 measured biomarkers along with the most powerful clinical factors, identified the following as the variables for the CHDRA model: age, sex, diabetes, family history of MI (at least two immediate family members, with no reference to their age at occurrence), CTACK (CCL27), Eotaxin (CCL11), IL-16 (Interleukin 16), MCP-3 (CCL7), HGF (hepatocyte growth factor), Fas Ligand (tumor necrosis factor ligand superfamily member 6), and sFas (tumor necrosis factor receptor superfamily member 6, secreted form). The biomarker protein assay limits of detection were in the pg/mL range and coefficients of variation under 15%, indicating high analytical performance (Table 2). (See Appendix Supplemental Table 3 for performance of all assays).

Figure 2.

Unadjusted (a) and adjusted (b) univariate Hazard Ratio (HR, per 1 SD) and 95% CI for PMRP cohort events. Adjustments: age, sex, systolic BP, diastolic BP, cholesterol, HDL, hypertension, hypertension drug use, hyperlipidemia, diabetes, and smoking status.

Table 2.

Performance characteristics of the CHDRA protein assays.

	Limit of Detection pg/mL	Intra-assay %CV	Inter-assay %CV
CTACK	17.0	6	13
Eotaxin	2.0	13	12
Fas Ligand	2.0	11	9
HGF	4.0	6	8
IL-16	2.0	8	9
MCP-3	0.5	10	8
sFas	2.0	7	6

CHDRA = coronary heart disease risk assessment; CV = coefficient of variation; SD = standard deviation.

CTACK= CCL27; Eotaxin = CCL11; IL-16 = Interleukin 16; MCP-3 = CCL7; HGF = hepatocyte growth factor; Fas Ligand = tumor necrosis factor ligand superfamily member 6; sFas = tumor necrosis factor receptor superfamily member 6, secreted form.

CHDRA Performance

The CHDRA pre-validation significantly reclassified to their correct high- or low risk categories those PMRP samples determined to be intermediate risk by the Framingham (clinical NRI = 38.2%) and the 9-clinical risk factor (clinical NRI = 19.7%) reference models (Table 3). These clinical NRI values indicate that a significant number of individuals, who were classified as intermediate risk by the reference model yet they experienced an event during follow-up, were correctly reclassified as being at high risk. Likewise, those who were event free during follow-up were correctly reclassified as low risk. Net reclassification of the Framingham model intermediate risk individuals was somewhat greater for those who had events than those who were event free (23.1% and 15.1%, respectively), while many more individuals with events were reclassified correctly versus the 9-clinical risk factor model than those without events (16.6% and 3.1%, respectively). In the training set as a whole, the CHDRA exhibited a significant NRI of 13.1% versus the Framingham model and an insignificant reclassification vs. the 9-clinical risk factor model. An AUC analysis with the PMRP intermediate risk patients also indicated improved sensitivity and specificity of the CHDRA compared to the Framingham model (Figure 3a).

Table 3.

CHDRA Pre-validation with PMRP samples NRI^* and clinical NRI^† versus the calibrated Framingham risk and 9–clinical risk factor models.

	All individuals		Individuals with events during follow-up		Individuals without events during follow-up
	Clinical NRI (SD) [n]	NRI (SD) [n]	Clinical NRI (SD) [n]	NRI (SD) [n]	Clinical NRI (SD) [n]	NRI (SD) [n]
Framingham Risk Model	0.382 (0.018) [312]	0.131 (0.015) [1084]	0.231 (0.023) [125]	0.208 (0.012) [362]	0.151 (0.019) [187]	−0.076 (0.009) [722]
9-Clinical Risk Factor Model	0.197 (0.041) [265]	0.004 (0.006) [1084]	0.166 (0.023) [106]	0.073 (0.006) [362]	0.031 (0.030) [159]	−0.069 (0.007) [722]

^*For all risk ranges.

^†For the intermediate risk range 3.5% to <7.5%.

CHDRA = coronary heart disease risk assessment; PMRP = Personalized Medicine Research Project; NRI = net reclassification index; Clinical NRI = clinical net reclassification index; Framingham risk model = Framingham risk score calibrated to PMRP follow–up time and baseline risk; 9–clinical risk factor model: age, sex, diabetes, family history of MI, HDL, systolic BP, total cholesterol, smoking status, and antihypertensive medication use.

Figure 3.

PMRP (a) and MESA (b) Framingham risk model intermediate risk sample set receiver operating characteristic curves for prediction of CHD. The sensitivity vs. 1-specificity curves comparing the CHDRA to the Framingham risk model (containing age, sex, total cholesterol, HDL cholesterol, blood pressure, diabetes and smoking status).

In the separate, independent MESA cohort, the CHDRA indicated a strong association (HR 2.17 [1.79 to 2.63], p<0.001) with incident CHD events (Table 4). The association remained significant after adjustment for Framingham risk and other clinical factors, but not when adjusted for all 10 covariates (Table 4). There was no significant difference between the CHDRA HR and the Framingham risk model HR (p=0.48). Interactions between race/ethnicity categories and the CHDRA score were also not significant (p=0.29). The Harrell’s C-statistics, using the entire MESA sample set, were not significantly different for the CHDRA and the Framingham risk models at 0.72 and 0.73, respectively (p=0.70). (See Appendix Supplemental Table 4 for more details.)

Table 4.

CHDRA MESA sample set HR with CHD risk factors adjustment^* compared to the Framingham risk model.

Model	HR†[95% CI]	p value
CHDRA	2.17 [1.79–2.63]	<0.001
Framingham risk‡	2.32 [1.90–2.83]	<0.001
CHDRA adjusted with Framingham risk	1.58 [1.23–2.04]	<0.001
CHDRA adjusted with Framingham risk + diabetes	1.68 [1.28–2.22]	<0.001
CHDRA adjusted with Framingham risk + FamHx	1.51 [1.16–1.97]	0.002
CHDRA adjusted with 10 clinical risk factors	1.34 [0.89–2.04]	0.16

CHDRA = coronary heart disease risk assessment; MESA = Multiple-Ethnic Study of Atherosclerosis; HR = hazard ratio; Framingham risk model = Framingham risk score calibrated to MESA follow–up time and baseline risk; CI = confidence interval; FamHx = family history of myocardial infarction.

^*Adjustment is based on: Framingham risk model, Framingham risk model plus diabetes, Framingham risk model plus family history of MI, and a combination of 10 clinical risk factors (age, sex, diabetes, family history of MI, HDL, systolic BP, total cholesterol, smoking status, antihypertensive medication use, and race/ethnicity).

^†HR are per 1–SD of the log10–transformed absolute risk score distribution of the individuals without events.

^‡The HR for the Framingham risk model was not significantly different from the CHDRA (p=0.48)

Using risk categories defined by the reference model, the CHDRA clinical NRI values were significant in the MESA cohort for all intermediate risk individuals classified by the Framingham risk and 9–clinical risk factor models (Table 5). The CHDRA correctly reclassified 25.7% individuals with events, and 17% without events, all of whom were initially classified as intermediate risk by the Framingham risk model, yielding a clinical NRI of 42.7%. The clinical NRI was 26.8% with the 9-clinical risk factor model. The NRI assessments, which consider all risk categories, were not statistically significant (Table 5).

Table 5.

CHDRA MESA samples NRI^* and clinical NRI^† versus the calibrated Framingham risk and 9–clinical risk factor models.

	All individuals				Individuals with events during follow-up		Individuals without events during follow-up
	Clinical NRI (SD) [n]	p-value‡	NRI (SD) [n]	p-value‡	Clinical NRI (n)	NRI (n)	Clinical NRI (n)	NRI (n)
Framingham risk Model	0.427 (0.117) [237]	<0.001	0.013 (0.063) [675]	0.84	0.257 (67)	0.049 (179)	0.170 (170)	−0.036 (496)
9-clinical risk factors	0.275 (0.130) [171]	0.04	−0.029 (0.058) [675]	0.62	0.166 (52)	−0.043 (179)	0.109 (119)	0.014 (496)

^*For all risk ranges.

^†For the intermediate risk range 3.5% to <7.5%.

CHDRA = coronary heart disease risk assessment; MESA = Multiple-Ethnic Study of Atherosclerosis; NRI = net reclassification index; Clinical NRI = clinical net reclassification index; Framingham risk model = Framingham risk score calibrated to MESA follow–up time and baseline risk; 9–clinical risk factor model: age, sex, diabetes, family history of MI, HDL, systolic BP, total cholesterol, smoking status, and antihypertensive medication use.

^‡p-value is compared with the null clinical NRI of 0.

The CHDRA reclassification of Framingham risk model intermediate risk individuals was balanced between those with events (30 out of 67; 45%) and those without events (72 out of 170; 42%) (Table 6). An AUC analysis with the intermediate risk patients also indicated improved sensitivity and specificity of the CHDRA compared to the Framingham risk (Figure 3b). When adding the seven selected proteins to a model containing classical risk factors (Framingham risk components plus family history of MI) a significant 25.5% clinical NRI (p=0.027) was obtained for the intermediate risk group vs. the fitted model without the added proteins.

Table 6.

CHDRA score MESA sample set reclassification compared to the calibrated Framingham risk model.

	CHDRA				Reclassified as Lower Risk, n (%)	Reclassified as Higher Risk, n (%)
	0% to <3.5%	3.5% to <7.5%	≥7.5%	Total
Framingham Risk Model
0% to <3.5%
No. with event	10	8	15	33	NA	23 (70)
No. without event	163	52	28	243	NA	80 (33)
No. of participants	173	60	43	276	NA	103
Observed Risk (95% CI)	0.01 (0.00–0.01)	0.02 (0.01–0.03)	0.06 (0.04–0.09)
3.5% to <7.5%
No. with event	13	24	30	67	13 (19)	30 (45)
No. without event	72	55	43	170	72 (42)	43 (25)
No. of participants	85	79	73	237	85	73
Observed Risk (95% CI)	0.02 (0.01–0.03)	0.05 (0.03–0.07)	0.07 (0.05–0.1)
≥7.5%
No. with event	8	22	49	79	30 (38)	NA
No. without event	7	26	50	83	33 (40)	NA
No. of participants	15	48	99	162	63	NA
Observed Risk (95% CI)	0.12 (0.06–0.2)	0.09 (0.06–0.12)	0.1 (0.08–0.13)
Total
No. with event	31	54	94	179	43 (24)	53 (30)
No. without event	242	133	121	496	105 (21)	123 (25)
No. of participants	273	187	215	675	148	176

CHDRA = coronary heart disease risk assessment; Framingham risk model = Framingham risk score calibrated to MESA follow–up time and baseline risk.

Considering all individuals, the continuous NRI versus the Framingham risk model indicated a net reclassification of 13.8% (p=0.001), while the integrated discrimination analysis indicated a significant discrimination of 1.89% (p=0.02).

Discussion

Using the PMRP cohort, we identified the combination of clinical risk factors and disease-relevant biomarkers that predict risk of 5-year CHD events. An algorithm containing seven disease-relevant protein biomarkers (CTACK, Eotaxin, FasLigand, HGF, IL-16, MCP-3, and sFas) and four clinical risk factors (age, sex, diabetes, and family history of MI) was identified and successfully validated in the independent MESA cohort. The reclassification by the CHDRA of 25.7% of the MESA individuals with events and 17% of those without events, all initially classified as intermediate risk by the Framingham risk, is an important incremental improvement in risk stratification capable of influencing clinical practice. Indeed, this 42.7% clinical NRI exceeds the clinical NRI calculated for established risk scores like the Reynolds risk score for men (clinical NRI = 14.2%) or the proposed addition of ECG abnormalities to traditional risk factors (clinical NRI= 13.6%) ^{6, 32}.

The CHDRA is the result of extensive protein assay optimization to identify the most informative atherosclerotic biomarkers, which when combined with clinical factors, provided significant risk reclassification in an independent cohort. This meets the proposed requirements for prognostic model validation ³³. This study also fulfills several recommendations advocated in the AHA scientific statement on novel cardiovascular risk marker evaluations, including prospective validation in an independent population and incremental information when added to standard risk markers ¹³. Although the PMRP population used to develop the CHDRA was homogeneous (99% Caucasian), and did not include fatal CHD, the model performed well in the more diverse MESA population that included fatal CHD endpoints and significant differences in baseline characteristics.

The proteins were from a list of 32 putative biomarkers involved in the cellular processes underlying CHD ^{19, 34}. Some of the biomarkers have not been previously identified as important in CHD risk assessment, yet all were tested because of their associations with atherosclerotic plaque biology, such as inflammation, chemotaxis, angiogenesis, cell proliferation, cell adhesion, and apoptosis ^35–37. Including biomarkers that represent different aspects of the pathophysiology underlying atherosclerosis may explain why this combination improved cardiovascular risk estimation. Indeed, the protein biomarkers themselves demonstrated significant reclassification (clinical NRI = 25.5%) when added to the Framingham risk plus family history of MI.

C-reactive protein (CRP) and N-terminal-pro-B-type natriuretic peptide (NT-proBNP) correlate with CHD risk in other studies, but were absent in the CHDRA. Forcing those proteins into the CHDRA slightly reduced the performance (data not shown). CRP is a marker of general inflammation and NT-proBNP is a marker of “pan-cardiac” damage; neither appears specific for atherosclerosis ^{10, 38}. As others have suggested, risk prediction is improved by including biomarkers more directly relevant to the pathophysiology underlying CHD ¹¹. Recent studies suggest that high-sensitivity cardiac troponin assays may provide CHD risk information, although data show cardiac troponin has more utility in near-term MI diagnosis and death, with a weaker association to atherosclerosis ^{39, 40}. Availability of a high-sensitivity cardiac troponin assay will be needed for future work to examine the CHDRA relationship with troponin levels.

Measuring the ability of biomarkers to assess risk across all categories may reject covariates that accurately assign risk within a clinically important category ⁴¹. And significant improvements can be missed when gauging performance with standard measures of model accuracy such as the C-statistic and AUC, as well as with NRI and integrated discrimination analysis, which look at how the whole sample set is reclassified ^{29, 42, 43}. A more clinically important performance measure is the clinical NRI for the intermediate risk subgroup that we are specifically targeting ^{28, 29}.

We used both NRI and clinical NRI to compare the CHDRA performance to the Framingham risk and 9–clinical risk factor models. While the CHDRA did not improve prediction over traditional measures in the test cohort as a whole (NRI=1.3%, p=0.84), classification was significantly improved in the intermediate risk group (clinical NRI=42.7%, p<0.001). The reclassification was significant for intermediate risk individuals with and without events, and it was more successful at upward reclassification, which is clinically more meaningful and could result in more appropriate aggressive risk factor modifications ^{44, 45}. Modest overall reclassification and discrimination improvements were seen in the continuous NRI and integrated discrimination analysis measurements, yet our focus was the intermediate risk population.

Improved CHD risk algorithms that target the intermediate risk population have value in clinical practice. Over half (58%) of individuals experiencing a CHD event had less than two conventional CHD risk factors ⁴⁶. Using the CHDRA may identify a significant proportion of the intermediate risk population who are eligible for more aggressive intervention and preventive therapy. The clinical utility of identifying individuals at high risk to target aggressive risk factor reductions is well documented ^{47, 48}. Intermediate risk individuals are often given less aggressive goals because more intensive treatments have uncertain risk-benefit ratios ²⁵. The CHDRA can provide physicians a useful clinical tool to further clarify the actual risk of intermediate risk patients, as ACCF/AHA guidelines recommend ³. Indeed, the American College of Preventive Medicine suggested that newer biomarkers may be helpful to reclassify intermediate risk individuals, enabling more favorable risk factor modification, while a greater focus on prevention could substantially reduce cardiovascular deaths ^{45, 48}. In addition, focusing on a 5-year rather than a 10-year horizon may motivate patients to better comply with therapeutic interventions and guideline-based therapies. Patient motivation is a key factor in statin therapy adherence, improvement of which could significantly impact clinical outcomes ^{49, 50}. The CHDRA model can be used in a clinical lab to provide a CHD risk “score” reported on a 100-unit percentage scale to convey the absolute risk of a cardiac event (MI or UA) within the subsequent 5 years⁵¹.

Potential limitations to this study include the use of frozen specimens, slightly different definition of events with the MESA samples, and inability to include fatal CHD as events in the PMRP set. The case-cohort sample size was determined from power calculations that balanced the cost of measuring all cohort samples with the ability to estimate the entire population characteristics through appropriate weights. Although biomarkers relevant to fatal CHD events may have been missed by not being able to include that endpoint during the model development with the PMRP samples, the CHDRA algorithm predicted in the MESA cohort the broader event list including fatal MI, suggesting that the model might be even more powerful if built using fatal CHD events. This possibility is further supported when considering that the PMRP event definition was broader than that used in MESA, yet the model performed well in the more narrowly defined MESA set. The ascertainment of serum lipid values and medication usage from the PMRP medical records or study questionnaire may also have introduced uncertainty into the model. However, the correlation was high (r²>0.87, data not shown) between medical record information and lipid measurements on a sample aliquot group.

Subsequent studies should consider CHDRA impact on patient management and cost effectiveness in a well-powered randomized interventional study. Future research should also examine model refinement for specific subgroups (i.e., men and women), the incorporation of CHD death events, and important risk markers such as cardiac calcium scoring, all of which could further improve cardiac risk assessment for the intermediate risk individual.

Conclusion

This study identified a serum protein biomarker-based prognostic algorithm for assessing the risk of acute cardiovascular events that provided a significant incremental benefit over using clinical risk factors alone for individuals currently assessed as intermediate risk. A test that correctly reclassifies patients to higher and lower risk categories can result in positive changes in physician management, patient behavior, and subsequent clinical outcomes.