Skip to main content
PMC home page
Journal of the American Heart Association: Cardiovascular and Cerebrovascular Disease logoLink to Journal of the American Heart Association: Cardiovascular and Cerebrovascular Disease
. 2015 Jul 6;4(7):e001646. doi: 10.1161/JAHA.114.001646

Traditional Risk Factors Versus Biomarkers for Prediction of Secondary Events in Patients With Stable Coronary Heart Disease: From the Heart and Soul Study

Alexis L Beatty 1,3,4,✉, Ivy A Ku 1,5,6, Kirsten Bibbins-Domingo 1,2,5, Robert H Christenson 7, Christopher R DeFilippi 8, Peter Ganz 1,5, Joachim H Ix 9,10,11, Donald Lloyd-Jones 12,13, Torbjørn Omland 14, Marc S Sabatine 15,16, Nelson B Schiller 1, Michael G Shlipak 1,2,17, Hicham Skali 16, Madoka Takeuchi 6, Eric Vittinghoff 2, Mary A Whooley 1,2,17
PMCID: PMC4608062  PMID: 26150476

Abstract

Background

Patients with stable coronary heart disease (CHD) have widely varying prognoses and treatment options. Validated models for risk stratification of patients with CHD are needed. We sought to evaluate traditional and novel risk factors as predictors of secondary cardiovascular (CV) events, and to develop a prediction model that could be used to risk stratify patients with stable CHD.

Methods and Results

We used independent derivation (912 participants in the Heart and Soul Study) and validation (2876 participants in the PEACE trial) cohorts of patients with stable CHD to develop a risk prediction model using Cox proportional hazards models. The outcome was CV events, defined as myocardial infarction, stroke, or CV death. The annual rate of CV events was 3.4% in the derivation cohort and 2.2% in the validation cohort. With the exception of smoking, traditional risk factors (including age, sex, body mass index, hypertension, dyslipidemia, and diabetes) did not emerge as the top predictors of secondary CV events. The top 4 predictors of secondary events were the following: N-terminal pro-type brain natriuretic peptide, high-sensitivity cardiac troponin T, urinary albumin:creatinine ratio, and current smoking. The 5-year C-index for this 4-predictor model was 0.73 in the derivation cohort and 0.65 in the validation cohort. As compared with variables in the Framingham secondary events model, the Heart and Soul risk model resulted in net reclassification improvement of 0.47 (95% CI 0.25 to 0.73) in the derivation cohort and 0.18 (95% CI 0.01 to 0.40) in the validation cohort.

Conclusions

Novel risk factors are superior to traditional risk factors for predicting 5-year risk of secondary events in patients with stable CHD.

Keywords: coronary disease, epidemiology, prevention, risk prediction

An estimated 15.5 million adults in the United States live with coronary heart disease (CHD).1 With advances in the treatment of acute coronary syndromes and aggressive risk factor management, patients now live longer with chronic CHD, and secondary prevention has become a major focus.2 To date, predicting risk of secondary events has received little attention because all patients with stable CHD are recommended to receive similar treatment with lipid-lowering medications, antiplatelet agents, β-blockers, angiotensin inhibitors, smoking cessation, and glycemic control, regardless of disease severity. However, given the widely varying prognoses3 and expanding range of therapeutic options for patients with chronic CHD, such as novel antiplatelet agents and revascularization procedures, it has become increasingly necessary to define distinct risk groups for whom different treatment strategies may be preferred.4

Most cardiovascular (CV) risk models have focused on predicting incident CHD5,6 or outcomes after acute coronary syndromes.7 In these models, traditional risk factors, such as age, gender, smoking, hypertension, cholesterol, and diabetes, have remained the cornerstone of risk stratification.8 However, it is increasingly recognized that risk factors for incident CHD may not predict secondary events in patients with prevalent CHD, in part because better risk factor profiles may reflect more aggressive control of sicker patients.9 Once clinical CHD exists, markers of end-organ damage may be more important than risk factors for incident disease. Thus, new approaches for risk prediction in patients with stable CHD are needed.

Many novel risk factors have been shown to provide incremental prognostic information in patients with stable CHD.10–19 Likewise, combining traditional risk factors with symptom severity, ejection fraction, and standard laboratory test results can improve prediction of secondary events.20–27 However, unlike the Framingham risk score for developing incident CHD, the few existing secondary prevention models have not become widely accepted for use in clinical practice,19,20,22,23,27 in part because of model complexity and lack of external validation. Simple integrated prediction models that capture risk of secondary events are lacking.

Our objectives were to evaluate traditional risk factors and novel biomarkers as predictors of 5-year risk of secondary events, and to develop a prediction model that could be used to risk stratify patients with chronic stable CHD.

Methods

Derivation Cohort

The Heart and Soul Study is a prospective cohort study that was originally designed to investigate the effect of psychosocial factors on prognosis in patients with stable CHD. Methods have been previously described.28 Subjects were eligible if they met one of the following criteria: (1) history of myocardial infarction, (2) history of coronary revascularization, (3) ≥50% angiographic stenosis in at least 1 coronary artery, or (4) exercise-induced ischemia by treadmill ECG or nuclear perfusion imaging. Exclusion criteria were the following: (1) history of myocardial infarction within the past 6 months, (2) inability to walk 1 block, or (3) intention to move out of the local area within 3 years. The protocol was approved by the appropriate institutional review boards, and all participants provided written informed consent. Between September 2000 and December 2002, 1024 participants enrolled and completed a baseline study visit that included an interview, questionnaire, 12-hour fasting blood draw, and echocardiogram. Serum was stored at −70°C. The current analysis was restricted to the 912 participants with complete baseline data. The last date of follow-up was June 14, 2011.

Model Covariates

Eighteen candidate predictors were chosen based on clinical judgment, prior literature, and reproducibility of tests (Table1). Age, sex, current smoking, medication use, history of myocardial infarction, and history of heart failure were collected by self-report questionnaire. Diabetes was defined by self-report, fasting glucose ≥126 mg/dL, or taking diabetic medications. Hypertension was defined by self-report, systolic blood pressure ≥140 mm Hg, or diastolic blood pressure ≥90 mm Hg. Blood pressure was measured by trained study personnel using calibrated sphygmomanometers. Body mass index was calculated as weight in kilograms divided by height in meters squared. Left ventricular ejection fraction was measured by echocardiography. Physical activity and medication adherence were assessed by self-report.28,29

Table 1.

Characteristics of Patients With or Without Subsequent MI, Stroke, or CV Death

Candidate Predictor Variables Derivation Cohort (n=912) Validation Cohort (n=2876)
Case n=202 (22%) Non-Case n=710 (78%) Case n=304 (11%) Non-Case n=2572 (89%)
Age, y* 70±12 66±11 67±9 64±8
Male 178 (88%) 572 (81%) 250 (82%) 2097 (82%)
BMI, kg/m2* 28.2±5.4 28.5±5.4 28.6±4.8 28.6±4.7
Current smoking 42 (21%) 137 (19%) 62 (20%) 355 (14%)
Diabetes 78 (39%) 212 (30%) 77 (25%) 427 (17%)
Hypertension 168 (83%) 538 (76%) 200 (56%) 1101 (44%)
History of myocardial infarction 123 (61%) 367 (52%) 184 (61%) 1431 (56%)
History of congestive heart failure† 51 (25%) 108 (15%) — —
Medication nonadherence 20 (10%) 52 (7%) — —
Physical inactivity 90 (45%) 241 (34%) — —
High-sensitivity troponin T, pg/mL‡ 16.5 (9.6 to 26.0) 8.6 (5.3 to 14.0) 7.6 (4.9 to 12.0) 6.1 (4.9 to 9.3)
LVEF <50% 40 (20%) 68 (10%) 52 (17%) 375 (15%)
LDL-C, mg/dL* 103±34 105±34 — —
HDL-C, mg/dL* 43±14 46±14 — —
C-reactive protein, mg/L‡ 2.7 (1.4 to 6.3) 2.0 (0.8 to 4.5) 2.1 (1.1 to 4.4) 1.7 (0.8 to 3.4)
NT-proBNP, pg/mL‡ 400 (147 to 1087) 141 (64 to 324) 221 (109 to 419) 131 (69 to 261)
BNP, pg/mL‡,§ 275 (107 to 887) 123 (50 to 296) 65 (33 to 115) 52 (25 to 101)
eGFR, mL/min per 1.73 m2* 62±23 74±20 74±19 78±19
Urine albumin:creatinine ratio, mg/g‡ 14.0 (7.8 to 56.3) 7.9 (4.7 to 14.6) 13.0 (6.3 to 35.8) 8.3 (5.2 to 17.6)
Open in a new tab

BMI indicates body mass index; CV, cardiovascular; eGFR, estimated glomerular filtration rate; HDL-C, high-density lipoprotein cholesterol; LDL-C, low-density lipoprotein cholesterol; LVEF, left ventricular ejection fraction; MI, myocardial infarction; NT-proBNP, N-terminal pro-type brain natriuretic peptide.

*

Mean±standard deviation.

†

History of congestive heart failure was an exclusion criterion in the validation cohort.

‡

Median (interquartile range).

§

BNP was not among the 18 candidate predictors; in a sensitivity analysis, BNP replaced NT-proBNP.

Estimated glomerular filtration rate was calculated using the combined (serum creatinine and cystatin C) CKD-Epi equation.30 Urinary albumin and creatinine were measured from 24-hour urine collections by nephelometry and the rate Jaffe method, respectively, and urinary albumin to creatinine ratio (uACR) was calculated as milligrams of albumin divided by grams of creatinine. Fasting total and high-density lipoprotein (HDL) cholesterol were measured, and low-density lipoprotein cholesterol was calculated by the Friedewald equation. High-sensitivity C-reactive protein was measured by the Roche Integra assay or the Beckman Extended Range assay. Results from these 2 assays were highly correlated (r=0.99). High-sensitivity cardiac troponin T (hs-cTnT) was measured using highly sensitive reagents on an Elecsys 2010 analyzer (Roche Diagnostics).31 N-terminal pro-type brain natriuretic peptide (NT-proBNP) was measured with the Roche Elecsys NT-proBNP assay.

Outcome Variables

The composite outcome variable was time to first nonfatal myocardial infarction, stroke, or CV death. Annual telephone interviews with participants or their proxies were conducted regarding recent emergency room visits, hospitalizations, or death. For any reported event, 2 independent and blinded adjudicators reviewed medical records, ECGs, death certificates, and coroner’s reports. If the adjudicators agreed on the outcome classification, their classification was binding. If they disagreed, a third blinded adjudicator reviewed the event and determined the outcome classification. Nonfatal myocardial infarction and stroke were defined using standard criteria.32,33 CV death was defined as due to myocardial infarction, sudden death, heart failure, coronary revascularization, stroke, or peripheral vascular disease.

External Validation Cohort

We are not aware of any other prospective observational cohort study of patients with stable CHD that has measured all 3 biomarkers (uACR, troponin, and NTproBNP) that were selected in our derivation sample. However, the PEACE trial was a randomized trial of trandolapril versus placebo in 8290 patients with stable CHD and left ventricular ejection fraction >40% who were enrolled from November 1996 to June 2000 and were followed for up to 7 years (median 4.8 years) (last date of follow-up December 31, 2003).34 Of these, 2876 participants had measurements of smoking status, NT-proBNP, hs-cTnT, and uACR, as well as detailed adjudication of CV events (urine samples for uACR were available for only 2977 participants. Additionally, 80 were excluded due to lack of serum biomarker measurements, 3 due to lack of smoking status, and 8 due to lack of follow-up).11,16,35,36

Statistical Analysis

Cox proportional hazards models were developed for risk stratification. At the outset, we omitted the 4 weakest predictors—low-density lipoprotein, high-density lipoprotein (HDL), prior history of myocardial infarction, and hypertension (all P-values >0.5). We then exhaustively screened candidate models with 4 to 11 predictors and potential interactions using 20 repetitions of 10-fold cross-validation of Harrell’s C-index, a measure of discrimination.37 Internal cross-validation was used to minimize overfitting. Non-normal continuous variables were log-transformed to improve linearity.

After the model selection algorithm was run, we selected the top 4-predictor model based on cross-validated C-index (CVCI). Although they had slightly better discrimination (higher C-indices), models incorporating more than 4 predictors included predictors that were not independently associated with CV events in multivariate models. Including additional variables did not meaningfully improve C-index or calibration. Calibration was assessed by visual comparison of observed and model-based risk by decile of model-based risk and informal comparison of the average model-based 5-year event risk with nonparametric Kaplan–Meier estimates. We also assessed model fit with a likelihood ratio test for addition of deciles of model-based risk to the continuous model. In addition, we evaluated the cross-validated calibration slope38 as a measure of overfitting.

The Heart and Soul model was externally validated in 2876 participants from the PEACE trial. We calculated the C-index for the validation cohort and fitted 5-year risks using Cox model coefficients and 5-year baseline survival estimates from the derivation cohort.

To compare the relative strength of individual variables in the top model, we calculated standardized hazard ratios (per standard deviation increase in log-transformed variable), cross-validated C-indices for each variable alone, and net reclassification of each variable added to a base model containing the other 3 variables.

Category-free net reclassification improvement39 was used to compare our model to the Framingham secondary events model for stable CHD.19 To optimize performance of the Framingham model, we used a model in which coefficients for the Framingham predictor variables were refit to the derivation cohort. In the validation cohort, we substituted ln(total cholesterol) for ln(total cholesterol/HDL) because HDL was not measured, and re-estimated the coefficients of both models. We also compared the Framingham secondary events model and the Heart and Soul model C-indices in both derivation and validation cohorts, and estimated bootstrap confidence intervals using 500 repetitions. All statistical analyses were performed using STATA 11.0, 12.1, or 13.1 (Stata Corp, College Station, TX).

Results

The 912 participants in the Heart and Soul derivation cohort contributed 5979 person-years with a mean (SD) of 6.6 (2.8) years follow-up. The 2876 participants in the PEACE validation cohort contributed 13 806 person-years with a mean (SD) of 4.8 (1.4) years follow-up. Compared to the derivation cohort, the validation cohort was significantly healthier at baseline (Table1) and had lower annual rates of CV events (Table2).

Table 2.

Annual Rate of Cardiovascular Events

Event Derivation Cohort (N=912) Validation Cohort (N=2876)
N Annual Rate (%) N Annual Rate (%)
Myocardial infarction 109 1.8 164 1.2
Stroke 37 0.6 58 0.4
Cardiovascular death 106 1.7 98 0.7
Any event 202 3.4 304 2.2
Open in a new tab

Discrimination

After screening for CVCI, the final model selected from the derivation cohort, the Heart and Soul risk model, contained, in order of importance: NT-proBNP, hs-cTnT, uACR, and current smoking. Larger models of up to 7 predictors only marginally improved discrimination, and even larger models beyond 7 predictors actually performed worse than the more parsimonious models. The 4-predictor Heart and Soul model had a CVCI of 0.73 (95% CI 0.69 to 0.76). Notably, traditional CV risk factors such as age, hypertension, low-density lipoprotein, diabetes, and obesity did not appear in the top predictive models. Including interactions between sex and other predictors did not increase the CVCI. Replacing NT-proBNP with BNP resulted in minimally changed CVCI of 0.73 (95% CI 0.69 to 0.76). We also performed a sensitivity analysis adding baseline medications (aspirin, β-blockers, statins, angiotensin-converting enzyme inhibitors, angiotensin receptor blockers, and diuretics) to the model, and did not find any improvement.

Calibration

Observed estimates of 5-year event incidence were similar to predicted risk (Figure1 – Panels A and B) in the derivation cohort. The likelihood-ratio goodness-of-fit P-value was 0.07, and the cross-validated calibration slope was 0.96, indicating acceptable fit.

Figure 1.

Figure 1

Open in a new tab

Observed vs predicted 5-year incidence of secondary events by Heart and Soul risk model in derivation and validation cohorts. Observed 5-year incidence of MI, stroke, or CV death by deciles of predicted risk (A) and by category of predicted risk (B) in the derivation cohort. Observed 5-year incidence of MI, stroke, or CV death by deciles of predicted risk (C) and by category of predicted risk (D) in the validation cohort. CV indicates cardiovascular; MI, myocardial infarction.

External Validation

In the validation cohort, C-index was 0.65 (95% CI 0.61 to 0.68), the goodness-of-fit P-value was 0.13, and the calibration slope was 0.85. Calibration is visually demonstrated in Figure1 – Panels C and D.

Comparison of Individual Variables

Standardized hazard ratios for the 4 individual predictors in the Heart and Soul model showed that NT-proBNP was the strongest biomarker (Table3), followed by hs-cTnT and then uACR. The top model with 5 predictors included the 4 predictors in the Heart and Soul model with the addition of high-sensitivity C-reactive protein, but high-sensitivity C-reactive protein was not significantly associated with CV events after multivariable adjustment for NT-proBNP, hs-cTnT, uACR, and smoking (hazard ratio 1.11 per SD increase, 95% CI 0.97 to 1.27; P=0.15). The hazard ratios of all the biomarkers were weaker in the validation cohort compared to the derivation cohort. C-indices of each individual variable for prediction of CV events and net reclassification of each variable also showed consistent results in terms of relative strength of variables as well as weaker performance of the biomarkers in the validation cohort (Table4).

Table 3.

Multivariable-Adjusted HR for MI, Stroke, or CV Death in the Heart and Soul Risk Model

Variable Derivation Cohort Validation Cohort
Adjusted* HR (95% CI) P Value Adjusted* HR (95% CI) P Value
NT-proBNP† 1.40 (1.24, 1.58) <0.001 1.38 (1.23, 1.55) <0.001
hs-cTnT† 1.65 (1.32, 2.06) <0.001 1.47 (1.18, 1.84) 0.001
uACR† 1.18 (1.08, 1.28) <0.001 1.11 (1.02, 1.21) 0.01
Current smoking 1.57 (1.11, 2.22) 0.01 1.66 (1.25, 2.20) <0.001
Open in a new tab

CV indicates cardiovascular; HR, hazard ratios; hs-cTnT, high-sensitivity cardiac troponin T; MI, myocardial infarction; NT-proBNP, N-terminal pro-type brain natriuretic peptide; uACR, urine albumin to creatinine ratio.

*

Adjusted for other variables in the model.

†

Per standard-deviation increase in log variable.

Table 4.

C-Index and Net Reclassification for Individual Variables Predicting MI, Stroke, or CV Death in Derivation and Validation Cohorts

Variable Derivation Cohort Validation Cohort
C-Index* (95% CI) Net Reclassification Improvement† (95% CI) C-Index* (95% CI) Net Reclassification Improvement† 95% CI
NT-proBNP 0.69 (0.65 to 0.73) 0.32 (0.16 to 0.49) 0.62 (0.58 to 0.65) 0.27 (0.15 to 0.38)
hs-cTnT 0.69 (0.65 to 0.73) 0.21 (0.04 to 0.39) 0.59 (0.55 to 0.62) 0.11 (−0.02 to 0.21)
uACR 0.66 (0.62 to 0.70) 0.14 (−0.08 to 0.28) 0.59 (0.56 to 0.63) 0.16 (0.03 to 0.28)
Current smoking 0.49 (0.49 to 0.50) 0.05 (−0.08 to 0.18) 0.53 (0.51 to 0.56) 0.15 (0.04 to 0.25)
Open in a new tab

CV indicates cardiovascular; hs-cTnT, high-sensitivity cardiac troponin T; MI, myocardial infarction; NT-proBNP, N-terminal pro-type brain natriuretic peptide; uACR, urine albumin to creatinine ratio.

*

C-index calculated for each individual variable alone.

†

Nested category-free net reclassification was calculated by adding each variable to a base model containing the other 3 variables.

Comparison of Heart and Soul Model to Framingham (Traditional) Secondary Events Model

The Framingham secondary events model19 contains age, diabetes, and ln(total cholesterol/HDL) for men, plus an additional 2 variables for women: ln(SBP) and smoking. There was a statistically significant improvement in C-index between the Framingham secondary events model and the Heart and Soul risk model in both cohorts. In the derivation cohort, the CVCI improved from 0.61 (95% CI 0.55 to 0.64) to 0.73 (95% CI 0.69 to 0.76), an absolute improvement of 0.12 (95% CI 0.08 to 0.19; P<0.001). In the validation cohort, the C-index improved from 0.57 (95% CI 0.52 to 0.61) to 0.65 (95% CI 0.61 to 0.68), an absolute improvement of 0.08 (95% CI 0.04 to 0.13; P=0.001). When compared with the Framingham secondary events model, the Heart and Soul model resulted in category-free net reclassification improvement of 0.47 (95% CI 0.25 to 0.73), with reclassification of 0.21 (95% CI 0.12 to 0.34) in cases and 0.26 (95% CI 0.01 to 0.46) in non-cases (Table5). In the validation cohort, the Heart and Soul risk model resulted in category-free net reclassification improvement of 0.18 (95% CI 0.01 to 0.40; cases: 0.10 95% CI 0.01 to 0.34; non-cases: 0.08 95% CI −0.06 to 0.32) over the Framingham secondary events model.

Table 5.

C-Indices and Category-Free Net Reclassification Improvement Compared to a Traditional Secondary Prediction Model in the Derivation and Validation Cohorts

Top Selected Risk Factors Based on Number of Variables Derivation Cohort Validation Cohort
C Index (95% CI) Overall NRI (95% CI) C Index (95% CI) Overall NRI (95% CI)
1 NT-proBNP 0.694 (0.653 to 0.737) 0.25 (0.03 to 0.54) 0.622 (0.586 to 0.653) −0.01 (−0.21 to 0.21)
2 Above+uACR 0.715 (0.676 to 0.752) 0.37 (0.10 to 0.62) 0.633 (0.595 to 0.668) 0.06 (−0.11 to 0.28)
3 Above+hs-cTnT 0.728 (0.697 to 0.763) 0.44 (0.25 to 0.74) 0.636 (0.601 to 0.669) 0.08 (−0.16 to 0.31)
4 Above+current smoking 0.732 (0.692 to 0.763) 0.47 (0.25 to 0.73) 0.646 (0.614 to 0.681) 0.18 (0.01 to 0.40)
5 Above+hs-CRP 0.736 (0.697 to 0.765) 0.45 (0.23 to 0.74) 0.646 (0.614 to 0.682) 0.15 (−0.10 to 0.38)
6 Above+LVEF <50% 0.736 (0.705 to 0.777) 0.46 (0.24 to 0.72) 0.643 (0.617 to 0.684) 0.18 (0.01 to 0.45)
7 Above+male sex 0.738 (0.706 to 0.774) 0.54 (0.34 to 0.77) 0.645 (0.616 to 0.678) 0.14 (−0.10 to 0.34)
8* Above+sex*LVEF interaction 0.738 (0.695 to 0.766) 0.55 (0.38 to 0.86) 0.646 (0.610 to 0.682) 0.17 (−0.04 to 0.42)
Open in a new tab

HDL indicates high-density lipoprotein; hs-CRP, high-sensitivity C-reactive protein; hs-cTnT, high-sensitivity cardiac troponin T; LVEF, left ventricular ejection fraction; NRI, category-free net reclassification improvement vs traditional Framingham secondary events model [age, diabetes, and ln(total cholesterol/HDL) for men, plus an additional 2 variables for women: ln(SBP) and smoking]; NT-proBNP, N-terminal pro-type brain natriuretic peptide; SBP, systolic blood pressure; uACR, urine albumin to creatinine ratio.

*

Cross-validated c-index for the 8-variable model (0.73763) was lower than the 7-variable model (0.73764). All models with 8 or more risk factors had lower c-indices than the 7-variable model.

Discussion

We sought to evaluate traditional and novel risk factors as predictors of 5-year risk of secondary events, and to develop a simple prediction model that could be used to risk stratify patients with chronic stable CHD. The Heart and Soul risk model (NT-proBNP, hs-cTnT, uACR, and smoking) provided robust discrimination and calibration in the derivation cohort (CVCI 0.73) and reasonable performance in the external validation cohort (C-index 0.65). Compared to a risk model based on traditional risk factors, the Heart and Soul risk model resulted in significant reclassification improvement, and categorized patients into a wide range of risk groups, highlighting the heterogeneity of disease course in patients with stable CHD. Traditional risk factors for incident CHD, most notably age, did not emerge as the top predictors of secondary events in patients with prevalent CHD.

Previously developed risk prediction models have incorporated an array of risk factors with varying abilities to discriminate risk. The Framingham secondary events model relies upon traditional risk factors, including age, gender, blood pressure, cholesterol, smoking, and diabetes, but results in poor risk discrimination for CV events.19 Other risk models of up to 16 risk factors, including patient-reported symptoms, left ventricular ejection fraction, and standard laboratory measurements have been developed.20–27 However, most of these models are too complex for routine use, and only 1 has been externally validated.27 Since prediction model creation tends to result in optimistic estimates, external validation provides valuable information about the performance of a model.40 Though we observed a lower C-index in the validation population with our model, an externally validated C-index of 0.65 is well within the range of other commonly used CV prediction models, such as the CHADS-2 model for predicting stroke in patients with atrial fibrillation (validated C-index 0.63 to 0.70)41,42 and the Thrombolysis in Myocardial Infarction risk score for predicting death and ischemic events in patients with unstable angina or non-ST elevation myocardial infarction (validated C-index 0.59 to 0.65).7,43

Furthermore, even when we ran the selection algorithm on the PEACE cohort as if it were the derivation cohort, the highest CVCI was still <0.7. This suggests that the decrease in C-index in the validation cohort is less a reflection of overfitting, but rather that the general health and homogeneity of a clinical trial population makes it harder to risk stratify. Therefore, the decrease in discrimination is largely a reflection of PEACE being a healthier cohort of clinical trial subjects.

The inclusion of multiple novel risk factors in a risk model for secondary CV events represents a significant departure from previous risk models. Using both traditional and novel risk factors as candidate predictors, we found that the top model identified using a prespecified algorithm included only 1 traditional risk factor (smoking) and 3 novel risk factors (NT-proBNP, hs-cTnT, uACR). It has been observed that the risk factors that predict a first event are often not the same factors that predict secondary events, possibly due to more aggressive treatment of traditional risk factors in diseased patients.9 The participants included in this study were individuals with stable CHD, most of whom were receiving treatment for traditional risk factors, which may explain why traditional risk factors did not emerge as strong predictors of subsequent CV events. Biomarkers may provide more information about the burden of disease than traditional risk factors, including age. Notably, the novel risk factors identified as top predictors may reflect risk related to ongoing hemodynamic stress (NT-proBNP), myocardial damage (hs-cTnT), and renal dysfunction (uACR). In addition to improving overall risk estimation, measurement of novel risk factors could also play a role in directing intensification of therapies.

In current practice, measurement of these novel risk factors is not routine, but there is interest in determining whether their use can improve clinical outcomes. Routine measurement of natriuretic peptides is gaining acceptance in the management of heart failure.44 High-sensitivity troponin assays are not commercially available in the United States, but evidence of their utility in the setting of acute chest pain and risk prediction is strong.12,18,45 Measurement of uACR is a recommended method for detecting chronic kidney disease and a Class IIa guideline recommendation in patients with CV disease because of their increased risk of concomitant kidney disease.46,47 Further study is needed to determine whether using a risk model based on multiple novel risk factors can improve clinical outcomes.

Several limitations must be considered in the interpretation of our results. First, both the derivation and validation cohorts included 82% men, so future studies are required to test this model in cohorts with larger numbers of women. Second, HDL was not available in the validation cohort, and is a variable in the Framingham secondary events model; therefore we could not export the published coefficients directly from the Framingham model, and perform a true validation by category-free net reclassification improvement. Nevertheless, by refitting both models to the validation cohort, we were able to make a fair comparison of the performance of the variables themselves, confirming that novel markers were stronger than traditional risk factors in the validation cohort as well. Third, the drop in C-index from 0.73 in the derivation cohort to 0.65 in the validation cohort likely reflects differences in the cohorts, but could also reflect some degree of model overfitting. Fourth, this model was created in the context of the candidate predictors listed in Table1 and does not account for any variation in risk that could occur due to other predictors not included in this analysis. Finally, like most CV risk prediction models,48 this model does not take into account treatment effect during the course of follow-up. However, this model is meant to predict subsequent risk for stable and already well-treated CHD patients.

In summary, we developed and validated a simple 4-predictor model for 5-year risk of CV events in patients with stable treated CHD. A risk model including multiple novel risk factors may be useful for estimating prognosis in patients with stable CHD to inform treatment decisions or for use as a risk stratification tool in future research.

Acknowledgments

The authors wish to acknowledge Dr Marc Pfeffer for providing access to the PEACE trial database and for his insight. Drs Beatty, Ku, Vittinghoff, and Whooley had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Sources of Funding

Dr Beatty was supported by the National Heart, Lung, and Blood Institute (F32 HL110518) and the National Center for Advancing Translational Sciences of the National Institutes of Health (KL2 TR000143). Dr Ku was supported by the National Heart, Lung, and Blood Institute (F32 HL097461) and the Sha Family Fellowship in Cardiovascular Research. The Heart and Soul Study was supported by the Department of Veterans Affairs; the National Heart, Lung, and Blood Institute (R01 HL079235); the American Federation for Aging Research; the Robert Wood Johnson Foundation; and the Ischemia Research and Education Foundation. The PEACE trial was supported by a contract (N01HC65149) from the National Heart, Lung, and Blood Institute and by Knoll Pharmaceuticals and Abbott Laboratories. Dr Sabatine and measurement of hs-CRP in PEACE was supported by the National Heart, Lung, and Blood Institute (R01 HL094390). Roche Diagnostics provided support for NT-proBNP and troponin T testing in both cohorts. The funding organizations had no role in the design and conduct of the study, in the collection, analysis, and interpretation of the data, or in the preparation, review, or approval of the manuscript.

Disclosures

Dr Omland has received speaker’s honoraria from Abbott Diagnostics and Roche Diagnostics, and research support via Akershus University Hospital from Abbott Diagnostics and AstraZeneca. Dr Whooley has received grant support from Roche Diagnostics and Janssen Research and Development, LLC. There are no other conflicts of interest relevant to this research.

References

  1. Mozaffarian D, Benjamin EJ, Go AS, Arnett DK, Blaha MJ, Cushman M, de Ferranti S, Despres JP, Fullerton HJ, Howard VJ, Huffman MD, Judd SE, Kissela BM, Lackland DT, Lichtman JH, Lisabeth LD, Liu S, Mackey RH, Matchar DB, McGuire DK, Mohler ER, III, Moy CS, Muntner P, Mussolino ME, Nasir K, Neumar RW, Nichol G, Palaniappan L, Pandey DK, Reeves MJ, Rodriguez CJ, Sorlie PD, Stein J, Towfighi A, Turan TN, Virani SS, Willey JZ, Woo D, Yeh RW, Turner MB American Heart Association Statistics Committee, Stroke Statistics Subcommittee. Heart disease and stroke statistics—2015 update: a report from the American Heart Association. Circulation. 2015;131:e29–e322. doi: 10.1161/CIR.0000000000000152. [DOI] [PubMed] [Google Scholar]
  2. Morrow DA. Cardiovascular risk prediction in patients with stable and unstable coronary heart disease. Circulation. 2010;121:2681–2691. doi: 10.1161/CIRCULATIONAHA.109.852749. [DOI] [PubMed] [Google Scholar]
  3. Bhatt DL, Eagle KA, Ohman EM, Hirsch AT, Goto S, Mahoney EM, Wilson PW, Alberts MJ, D’Agostino R, Liau CS, Mas JL, Röther J, Smith SC, Salette G, Contant CF, Massaro JM, Steg PG REACH Registry Investigators. Comparative determinants of 4-year cardiovascular event rates in stable outpatients at risk of or with atherothrombosis. JAMA. 2010;304:1350–1357. doi: 10.1001/jama.2010.1322. [DOI] [PubMed] [Google Scholar]
  4. Lloyd-Jones D. Cardiovascular risk prediction: basic concepts, current status, and future directions. Circulation. 2010;121:1768–1777. doi: 10.1161/CIRCULATIONAHA.109.849166. [DOI] [PubMed] [Google Scholar]
  5. Wilson PW, D’Agostino RB, Levy D, Belanger AM, Silbershatz H, Kannel WB. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97:1837–1847. doi: 10.1161/01.cir.97.18.1837. [DOI] [PubMed] [Google Scholar]
  6. D’Agostino RB, Grundy S, Sullivan LM, Wilson P CHD Risk Prediction Group. Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA. 2001;286:180–187. doi: 10.1001/jama.286.2.180. [DOI] [PubMed] [Google Scholar]
  7. Antman EM, Cohen M, Bernink PJ, McCabe CH, Horacek T, Papuchis G, Mautner B, Corbalan R, Radley D, Braunwald E. The TIMI risk score for unstable angina/non-ST elevation MI: a method for prognostication and therapeutic decision making. JAMA. 2000;284:835–842. doi: 10.1001/jama.284.7.835. [DOI] [PubMed] [Google Scholar]
  8. Goff DC, Lloyd-Jones DM, Bennett G, O’Donnell CJ, Coady S, Robinson J, D’Agostino RB, Schwartz JS, Gibbons R, Shero ST, Greenland P, Smith SC, Lackland DT, Sorlie P, Levy D, Stone NJ, Wilson PWF. 2013 ACC/AHA guideline on the assessment of cardiovascular risk. Circulation. 2014;129:S49–S73. doi: 10.1161/01.cir.0000437741.48606.98. [DOI] [PubMed] [Google Scholar]
  9. Dahabreh IJ, Kent DM. Index event bias as an explanation for the paradoxes of recurrence risk research. JAMA. 2011;305:822–823. doi: 10.1001/jama.2011.163. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Kragelund C, Grønning B, Køber L, Hildebrandt P, Steffensen R. N-terminal pro-B-type natriuretic peptide and long-term mortality in stable coronary heart disease. N Engl J Med. 2005;352:666–675. doi: 10.1056/NEJMoa042330. [DOI] [PubMed] [Google Scholar]
  11. Omland T, Sabatine MS, Jablonski KA, Rice MM, Hsia J, Wergeland R, Landaas S, Rouleau JL, Domanski MJ, Hall C, Pfeffer MA, Braunwald E PEACE Investigators. Prognostic value of B-type natriuretic peptides in patients with stable coronary artery disease: the PEACE Trial. J Am Coll Cardiol. 2007;50:205–214. doi: 10.1016/j.jacc.2007.03.038. [DOI] [PubMed] [Google Scholar]
  12. Omland T, de Lemos JA, Sabatine MS, Christophi CA, Rice MM, Jablonski KA, Tjora S, Domanski MJ, Gersh BJ, Rouleau JL, Pfeffer MA, Braunwald E PEACE Trial Investigators. A sensitive cardiac troponin T assay in stable coronary artery disease. N Engl J Med. 2009;361:2538–2547. doi: 10.1056/NEJMoa0805299. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Sabatine MS, Morrow DA, Jablonski KA, Rice MM, Warnica JW, Domanski MJ, Hsia J, Gersh BJ, Rifai N, Ridker PM, Pfeffer MA, Braunwald E PEACE Investigators. Prognostic significance of the Centers for Disease Control/American Heart Association high-sensitivity C-reactive protein cut points for cardiovascular and other outcomes in patients with stable coronary artery disease. Circulation. 2007;115:1528–1536. doi: 10.1161/CIRCULATIONAHA.106.649939. [DOI] [PubMed] [Google Scholar]
  14. Bibbins-Domingo K, Gupta R, Na B, Wu AH, Schiller NB, Whooley MA. N-terminal fragment of the prohormone brain-type natriuretic peptide (NT-proBNP), cardiovascular events, and mortality in patients with stable coronary heart disease. JAMA. 2007;297:169–176. doi: 10.1001/jama.297.2.169. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Blankenberg S, McQueen MJ, Smieja M, Pogue J, Balion C, Lonn E, Rupprecht HJ, Bickel C, Tiret L, Cambien F, Gerstein H, Munzel T, Yusuf S HOPE Study Investigators. Comparative impact of multiple biomarkers and N-Terminal pro-brain natriuretic peptide in the context of conventional risk factors for the prediction of recurrent cardiovascular events in the Heart Outcomes Prevention Evaluation (HOPE) Study. Circulation. 2006;114:201–208. doi: 10.1161/CIRCULATIONAHA.105.590927. [DOI] [PubMed] [Google Scholar]
  16. Solomon SD, Lin J, Solomon CG, Jablonski KA, Rice MM, Steffes M, Domanski M, Hsia J, Gersh BJ, Arnold JMO, Rouleau J, Braunwald E, Pfeffer MA PEACE Investigators. Influence of albuminuria on cardiovascular risk in patients with stable coronary artery disease. Circulation. 2007;116:2687–2693. [Google Scholar]
  17. Sabatine MS, Morrow DA, de Lemos JA, Omland T, Sloan S, Jarolim P, Solomon SD, Pfeffer MA, Braunwald E. Evaluation of multiple biomarkers of cardiovascular stress for risk prediction and guiding medical therapy in patients with stable coronary disease. Circulation. 2012;125:233–240. doi: 10.1161/CIRCULATIONAHA.111.063842. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Beatty AL, Ku IA, Christenson RH, Defilippi CR, Schiller NB, Whooley MA. High-sensitivity cardiac troponin T levels and secondary events in outpatients with coronary heart disease from the Heart and Soul Study. JAMA Intern Med. 2013;173:763–769. doi: 10.1001/jamainternmed.2013.116. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. D’Agostino RB, Russell MW, Huse DM, Ellison RC, Silbershatz H, Wilson PW, Hartz SC. Primary and subsequent coronary risk appraisal: new results from the Framingham study. Am Heart J. 2000;139:272–281. doi: 10.1067/mhj.2000.96469. [DOI] [PubMed] [Google Scholar]
  20. Marschner IC, Colquhoun D, Simes RJ, Glasziou P, Harris P, Singh BB, Friedlander D, White H, Thompson P, Tonkin A LIPID Study Investigators. Long-term risk stratification for survivors of acute coronary syndromes: results from the Long-term Intervention with Pravastatin in Ischemic Disease (LIPID) Study. LIPID Study Investigators. J Am Coll Cardiol. 2001;38:56–63. doi: 10.1016/s0735-1097(01)01360-2. [DOI] [PubMed] [Google Scholar]
  21. Marchioli R, Avanzini F, Barzi F, Chieffo C, Di Castelnuovo A, Franzosi MG, Geraci E, Maggioni AP, Marfisi RM, Mininni N, Nicolosi GL, Santini M, Schweiger C, Tavazzi L, Tognoni G, Valagussa F GISSI-Prevenzione Investigators. Assessment of absolute risk of death after myocardial infarction by use of multiple-risk-factor assessment equations: GISSI-Prevenzione mortality risk chart. Eur Heart J. 2001;22:2085–2103. doi: 10.1053/euhj.2000.2544. [DOI] [PubMed] [Google Scholar]
  22. Clayton TC, Lubsen J, Pocock SJ, Vokó Z, Kirwan BA, Fox KA, Poole-Wilson PA. Risk score for predicting death, myocardial infarction, and stroke in patients with stable angina, based on a large randomised trial cohort of patients. BMJ. 2005;331:869. doi: 10.1136/bmj.38603.656076.63. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Daly CA, De Stavola B, Sendon JL, Tavazzi L, Boersma E, Clemens F, Danchin N, Delahaye F, Gitt A, Julian D, Mulcahy D, Ruzyllo W, Thygesen K, Verheugt F, Fox KM Euro Heart Survey Investigators. Predicting prognosis in stable angina—results from the Euro heart survey of stable angina: prospective observational study. BMJ. 2006;332:262–267. doi: 10.1136/bmj.38695.605440.AE. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Cui J, Forbes A, Kirby A, Simes J, Tonkin A. Laboratory and non-laboratory-based risk prediction models for secondary prevention of cardiovascular disease: the LIPID study. Eur J Cardiovasc Prev Rehabil. 2009;16:660–668. doi: 10.1097/HJR.0b013e32832f3b2b. [DOI] [PubMed] [Google Scholar]
  25. Battes L, Barendse R, Steyerberg EW, Simoons ML, Deckers JW, Nieboer D, Bertrand M, Ferrari R, Remme WJ, Fox K, Takkenberg JJ, Boersma E, Kardys I. Development and validation of a cardiovascular risk assessment model in patients with established coronary artery disease. Am J Cardiol. 2013;112:27–33. doi: 10.1016/j.amjcard.2013.02.049. [DOI] [PubMed] [Google Scholar]
  26. Dorresteijn JA, Visseren FL, Wassink AM, Gondrie MJ, Steyerberg EW, Ridker PM, Cook NR, van der Graaf Y on behalf of the SMART Study Group. Development and validation of a prediction rule for recurrent vascular events based on a cohort study of patients with arterial disease: the SMART risk score. Heart. 2013;99:866–872. doi: 10.1136/heartjnl-2013-303640. [DOI] [PubMed] [Google Scholar]
  27. Rapsomaniki E, Shah A, Perel P, Denaxas S, George J, Nicholas O, Udumyan R, Feder GS, Hingorani AD, Timmis A, Smeeth L, Hemingway H. Prognostic models for stable coronary artery disease based on electronic health record cohort of 102 023 patients. Eur Heart J. 2014;35:844–852. doi: 10.1093/eurheartj/eht533. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Whooley MA, de Jonge P, Vittinghoff E, Otte C, Moos R, Carney RM, Ali S, Dowray S, Na B, Feldman MD, Schiller NB, Browner WS. Depressive symptoms, health behaviors, and risk of cardiovascular events in patients with coronary heart disease. JAMA. 2008;300:2379–2388. doi: 10.1001/jama.2008.711. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Gehi AK, Ali S, Na B, Whooley MA. Self-reported medication adherence and cardiovascular events in patients with stable coronary heart disease: the Heart and Soul Study. Arch Intern Med. 2007;167:1798–1803. doi: 10.1001/archinte.167.16.1798. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Inker LA, Schmid CH, Tighiouart H, Eckfeldt JH, Feldman HI, Greene T, Kusek JW, Manzi J, Van Lente F, Zhang YL, Coresh J, Levey AS CKD-Epi Investigators. Estimating glomerular filtration rate from serum creatinine and cystatin C. N Engl J Med. 2012;367:20–29. doi: 10.1056/NEJMoa1114248. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Giannitsis E, Kurz K, Hallermayer K, Jarausch J, Jaffe AS, Katus HA. Analytical validation of a high-sensitivity cardiac troponin T assay. Clin Chem. 2010;56:254–261. doi: 10.1373/clinchem.2009.132654. [DOI] [PubMed] [Google Scholar]
  32. Luepker RV, Apple FS, Christenson RH, Crow RS, Fortmann SP, Goff D, Goldberg RJ, Hand MM, Jaffe AS, Julian DG, Levy D, Manolio T, Mendis S, Mensah G, Pajak A, Prineas RJ, Reddy KS, Roger VL, Rosamond WD, Shahar E, Sharrett AR, Sorlie P, Tunstall-Pedoe H AHA Council on Epidemiology and Prevention, AHA Statistics Committee, World Heart Federation Council on Epidemiology and Prevention, European Society of Cardiology Working Group on Epidemiology and Prevention, Centers for Disease Control and Prevention, and National Heart, Lung, and Blood Institute. Case definitions for acute coronary heart disease in epidemiology and clinical research studies: a statement from the AHA Council on Epidemiology and Prevention; AHA Statistics Committee; World Heart Federation Council on Epidemiology and Prevention; the European Society of Cardiology Working Group on Epidemiology and Prevention; Centers for Disease Control and Prevention; and the National Heart, Lung, and Blood Institute. Circulation. 2003;108:2543–2549. doi: 10.1161/01.CIR.0000100560.46946.EA. [DOI] [PubMed] [Google Scholar]
  33. McKee PA, Castelli WP, McNamara PM, Kannel WB. The natural history of congestive heart failure: the Framingham study. N Engl J Med. 1971;285:1441–1446. doi: 10.1056/NEJM197112232852601. [DOI] [PubMed] [Google Scholar]
  34. Braunwald E, Domanski MJ, Fowler SE, Geller NL, Gersh BJ, Hsia J, Pfeffer MA, Rice MM, Rosenberg YD, Rouleau JL PEACE Trial Investigators. Angiotensin-converting-enzyme inhibition in stable coronary artery disease. N Engl J Med. 2004;351:2058–2068. doi: 10.1056/NEJMoa042739. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Pfeffer MA, Domanski M, Rosenberg Y, Verter J, Geller N, Albert P, Hsia J, Braunwald E. Prevention of events with angiotensin-converting enzyme inhibition (the PEACE study design). Prevention of events with angiotensin-converting enzyme inhibition. Am J Cardiol. 1998;82:25H–30H. doi: 10.1016/s0002-9149(98)00488-3. [DOI] [PubMed] [Google Scholar]
  36. Lewis EF, Solomon SD, Jablonski KA, Rice MM, Clemenza F, Hsia J, Maggioni AP, Zabalgoitia M, Huynh T, Cuddy TE, Gersh BJ, Rouleau J, Braunwald E, Pfeffer MA PEACE Investigators. Predictors of heart failure in patients with stable coronary artery disease: a PEACE study. Circ Heart Fail. 2009;2:209–216. doi: 10.1161/CIRCHEARTFAILURE.108.820696. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Pencina MJ, D’Agostino RB. Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Stat Med. 2004;23:2109–2123. doi: 10.1002/sim.1802. [DOI] [PubMed] [Google Scholar]
  38. Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer; 2009. [Google Scholar]
  39. Pencina MJ, Agostino RB, Sr, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2010;30:11–21. doi: 10.1002/sim.4085. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med. 2000;19:453–473. doi: 10.1002/(sici)1097-0258(20000229)19:4<453::aid-sim350>3.0.co;2-5. [DOI] [PubMed] [Google Scholar]
  41. Gage BF, Waterman AD, Shannon W, Boechler M, Rich MW, Radford MJ. Validation of clinical classification schemes for predicting stroke: results from the National Registry of Atrial Fibrillation. JAMA. 2001;285:2864–2870. doi: 10.1001/jama.285.22.2864. [DOI] [PubMed] [Google Scholar]
  42. Gage BF, van Walraven C, Pearce L, Hart RG, Koudstaal PJ, Boode BS, Petersen P. Selecting patients with atrial fibrillation for anticoagulation: stroke risk stratification in patients taking aspirin. Circulation. 2004;110:2287–2292. doi: 10.1161/01.CIR.0000145172.55640.93. [DOI] [PubMed] [Google Scholar]
  43. Diverse Populations Collaborative Group. Prediction of mortality from coronary heart disease among diverse populations: is there a common predictive function? Heart. 2002;88:222–228. doi: 10.1136/heart.88.3.222. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Januzzi JL, Troughton R. Are serial BNP measurements useful in heart failure management? Serial natriuretic peptide measurements are useful in heart failure management. Circulation. 2013;127:500–507. doi: 10.1161/CIRCULATIONAHA.112.120485. discussion 508. [DOI] [PubMed] [Google Scholar]
  45. Twerenbold R, Jaffe A, Reichlin T, Reiter M, Mueller C. High-sensitive troponin T measurements: what do we gain and what are the challenges? Eur Heart J. 2012;33:579–586. doi: 10.1093/eurheartj/ehr492. [DOI] [PubMed] [Google Scholar]
  46. Brosius FC, Hostetter TH, Kelepouris E, Mitsnefes MM, Moe SM, Moore MA, Pennathur S, Smith GL, Wilson PWF American Heart Association Kidney and Cardiovascular Disease Council, Councils on High Blood Pressure Research, Cardiovascular Disease in the Young, Epidemiology and Prevention, Quality of Care and Outcomes Research Interdisciplinary Working Group. Detection of chronic kidney disease in patients with or at increased risk of cardiovascular disease: a science advisory from the American Heart Association Kidney And Cardiovascular Disease Council; the Councils on High Blood Pressure Research, Cardiovascular Disease in the Young, and Epidemiology and Prevention; and the Quality of Care and Outcomes Research Interdisciplinary Working Group: developed in collaboration with the National Kidney Foundation. Circulation. 2006;114:1083–1087. doi: 10.1161/CIRCULATIONAHA.106.177321. [DOI] [PubMed] [Google Scholar]
  47. Levey AS, Becker C, Inker LA. Glomerular filtration rate and albuminuria for detection and staging of acute and chronic kidney disease in adults: a systematic review. JAMA. 2015;313:837–846. doi: 10.1001/jama.2015.0602. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Liew SM, Doust J, Glasziou P. Cardiovascular risk scores do not account for the effect of treatment: a review. Heart. 2011;97:689–697. doi: 10.1136/hrt.2010.220442. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Journal of the American Heart Association: Cardiovascular and Cerebrovascular Disease are provided here courtesy of Wiley

RESOURCES