Development and Validation of a Coronary Risk Prediction Model for Older U.S. and European Persons in the Cardiovascular Health Study and the Rotterdam Study

Abstract

Background

Risk scores for prediction of coronary heart disease (CHD) in older adults are needed.

Objective

To develop a sex-specific CHD risk prediction model for older adults that accounts for competing risks for death.

Design

2 observational cohort studies, using data from 4946 participants in the Cardiovascular Health Study (CHS) and 4303 participants in the Rotterdam Study (RS).

Setting

Community settings in the United States (CHS) and Rotterdam, the Netherlands (RS).

Participants

Persons aged 65 years or older who were free of cardiovascular disease.

Measurements

A composite of nonfatal myocardial infarction and coronary death.

Results

During a median follow-up of 16.5 and 14.9 years, 1166 CHS and 698 RS participants had CHD events, respectively. Deaths from noncoronary causes largely exceeded the number of CHD events, complicating accurate CHD risk predictions. The prediction model had moderate ability to discriminate between events and nonevents (c-statistic, 0.63 in both U.S. and European men and 0.67 and 0.68 in U.S. and European women). The model was well-calibrated; predicted risks were in good agreement with observed risks. Compared with the Framingham point scores, the prediction model classified elderly U.S. persons into higher risk categories but elderly European persons into lower risk categories. Differences in classification accuracy were not consistent and depended on cohort and sex. Adding newer cardiovascular risk markers to the model did not substantially improve performance.

Limitation

The model may be less applicable in nonwhite populations, and the comparison Framingham model was not designed for adults older than 79 years.

Conclusion

A CHD risk prediction model that accounts for deaths from noncoronary causes among older adults provided well-calibrated risk estimates but was not substantially more accurate than Framingham point scores. Moreover, adding newer risk markers did not improve accuracy. These findings emphasize the difficulties of predicting CHD risk in elderly persons and the need to improve these predictions.

Primary Funding Source

National Heart, Lung, and Blood Institute; National Institute of Neurological Disorders and Stroke; The Netherlands Organisation for Scientific Research; and the Netherlands Organisation for Health Research and Development.

Specialist societies recommend initiating preventive treatment of cardiovascular disease on the basis of a person’s 10-year risk for coronary heart disease (CHD) (1–3). Well-known prognostic models to estimate this risk originate from the Framingham Heart Study (4–7), the Women’s Health Study (8), the PROCAM (Prospective Cardiovascular Münster) study (9), and the SCORE (Systematic Coronary Risk Evaluation) project (10).

Demographic changes have increasingly led to an extension of primary prevention strategies for CHD to elderly persons (11, 12). However, these persons are under-represented or neglected in well-known CHD prediction models. Several studies show that existing CHD prediction models may extrapolate poorly to persons older than 70 years (13) and that the predictive associations of risk factors for CHD may diminish with increasing age (1, 14–17). A CHD prediction model for elderly persons should also take into account that with growing age and frailty, CHD events may be increasingly precluded by death from competing noncoronary causes. A valid risk prediction approach in this situation must account for competing causes of death to prevent inflated predictions of limited practical use (18–22).

The purpose of this study was to develop and evaluate a population-based algorithm to predict coronary risk in elderly persons on the basis of traditional risk factors. We also examined model performance after the addition of newer risk markers for cardiovascular disease. Our study is the result of a collaboration between 2 large and similarly designed cohort studies on cardiovascular disease in elderly persons, the Cardiovascular Health Study (CHS) in the United States (23) and Rotterdam Study (RS) in the Netherlands (24, 25).

Methods

The CHS is a prospective population-based study in adults aged 65 years or older with the main objective of identifying risk factors related to the onset and course of CHD and stroke. Eligible participants were sampled from Medicare eligibility lists in 4 U.S. communities. The rationale and design of the CHS have been described elsewhere (23). The RS is a prospective population-based cohort study of persons aged 55 years or older living in a suburb of Rotterdam, the Netherlands. This study aims to assess the determinants of cardiovascular and other diseases in elderly persons (24, 25).

For both cohorts, participants gave written informed consent, and the studies were approved by the ethics committees of the CHS sites and the Erasmus Medical Center in Rotterdam.

Study Population

From both cohorts, we selected participants aged 65 years or older who were free of definite CHD and cerebrovascular disease at baseline. After participants with a history of myocardial infarction (MI); electrocardiography (ECG) results consistent with past MI; or a history of percutaneous or surgical coronary revascularization procedures, stroke, or carotid endarterectomy were excluded, 4946 CHS and 4303 RS participants remained in the analyses. The Appendix (available at www.annals.org) and other publications (23, 26, 27) detail procedures for assessing medical history and coronary risk factors at baseline.

End Points

The outcome for this study was time to first CHD event, a composite of nonfatal MI and fatal CHD. The Appendix provides details on the very similar standardized definitions of CHD events for the 2 cohorts (27, 28).

Statistical Analysis

We developed prespecified and sex-specific CHD prediction models using age; systolic blood pressure (BP), with separate effects for treated and untreated participants; presence of diabetes mellitus; levels of total and high-density lipoprotein (HDL) cholesterol; and smoking status as covariates. We used a competing-risk method based on the Fine and Gray model, using the approach by Ruan and Gray (29–31).

We pooled data from both cohorts into 2 sex-specific data sets. Regression models included cohort-stratified baseline hazard functions; that is, we assumed that baseline risk for CHD may differ but that risk factors act equally on U.S. and European persons (10) and assessed this assumption by testing for cohort–risk factor interactions. Appendix Table 1 (available at www.annals.org) provides details about the development steps of the model and the testing of the assumptions.

We assessed prognostic accuracy by evaluating the pooled model on each cohort individually and by cross-validation, in which models were fit to one cohort and evaluated in the other. We quantified the discriminative ability up to 10 years of follow-up with an adaptation of the Harrell c-statistic (17) to the competing-risks setting (21). Calibration was assessed by plotting 10-year predicted risk against observed risk (Appendix Figure, available at www.annals.org) (21).

Next, we computed 10-year risk predictions based on the original Adult Treatment Panel (ATP) III Framingham point score (FPS) (1), which predicts the same composite CHD end point, and compared the accuracy of FPS predictions with predictions from our model by using the c-statistic (21) and risk classification methods with recommended cutoff values (1). Because head-to-head comparisons of different nonnested prediction models, particularly those that have not been fitted on the same population, are hard to interpret with single measures (such as the net reclassification improvement [NRI] [32]), we used summary metrics based on the margins of the reclassification table proposed by Janes and Pepe (33–35). We focused on differences between the 2 models in proportions of events and nonevents classified into the high-risk (>20%) or low-risk (<10%) categories. In these categories, decisions to test or treat are more established than in the intermediate risk category, in which appropriate clinical actions are sometimes less certain. The comparisons are summarized as changes in the true- and false-positive and true- and false-negative rates. Because the FPS is intended for nondiabetic persons, we also refit and reexamined our model in non-diabetic participants.

In a further step, we examined the incremental value of extending the competing-risks model based on traditional risk factors with the single addition of body mass index, C-reactive protein (CRP) levels, carotid intima–media thickness (cIMT), ankle– brachial index (ABI), or the presence of left ventricular hypertrophy on ECG (ECG-LVH) (32). We evaluated the added value of these markers on the basis of statistically significant overall model improvement and the increase in the c-statistic (17). Because we compared nested models, we used the NRI to present reclassification accuracy. We computed the NRIs by adapting the suggestion of Steyerberg and Pencina (32, 36) for survival data to the competing-risks setting. Appendix Table 1 provides details of the evaluation of extended models.

In a sensitivity analysis, we excluded participants aged 80 years or older because the FPS are developed for adults up to age 79 years; there were few meaningful changes in our results so we do not report them.

We report estimates of relative risk and c-statistics with 95% CIs. All hypothesis tests are 2-sided, and the significance level was set to 5%. Statistical analyses were performed using R, version 2.14.2 (R Foundation for Statistical Computing, Vienna, Austria). The Appendix provides more detailed descriptions of our statistical analysis and methods.

Role of the Funding Source

Our research was supported by the National Heart, Lung, and Blood Institute; the National Institute of Neurological Disorders and Stroke; the Netherlands Organisation for Scientific Research; and the Netherlands Organisation for Health Research and Development. None of the funding sources had any role in the design or conduct of the study; the collection, management, analysis, or interpretation of the data; or the preparation, review, or approval of the manuscript.

Results

Table 1 shows baseline characteristics. In the CHS cohort, 15.5% of men and 11.5% of women were aged 80 years or older. In the RS cohort, the corresponding proportions were 19.7% and 33.3%. The median duration of follow-up was 16.5 years (interquartile range, 13.5 to 16.7 years) in the CHS and 14.9 years (interquartile range, 14.1 to 15.7 years) in the RS. We observed 563 CHD events in men and 603 in women in the CHS; in the RS, we observed 283 events in men and 415 in women. This corresponded with 10-year cumulative incidences of 19.9% and 15.8% in men in the CHS and the RS, respectively, and similar incidences in women (11.5% and 10.4%, respectively). In both cohorts, the number of competing non-coronary deaths exceeded the number of CHD events over the entire age range; we observed 2000 and 2244 noncoronary deaths in the CHS and the RS, respectively (Appendix Tables 2 and 3, available at www.annals.org). The incidence of competing noncoronary death increased more rapidly with age than did the incidence of CHD (Figure).

Table 1.

Baseline Characteristics of Men and Women Free of Cardiovascular Disease in the CHS and the RS

Characteristic	CHS		RS
	Men (n = 1917)	Women (n = 3029)	Men (n = 1454)	Women (n = 2849)
Median age (IQR), y	72 (69–77)	71 (68–76)	73 (69–78)	76 (70–83)
Treated hypertension, n (%)	750 (39)	1353 (45)	394 (31)	1095 (45)
Median systolic blood pressure (IQR), mm Hg
Treated for hypertension	137 (123–153)	140 (126–156)	142 (128–158)	147 (132–163)
Not treated for hypertension	132 (119–146)	130 (118–145)	138 (126–154)	142 (128–156)
Smoking, n (%)
Current	221 (12)	377 (12)	414 (30)	342 (13)
Former	1065 (56)	895 (30)	876 (63)	636 (24)
Never	628 (33)	1754 (58)	106 (8)	1639 (63)
Median total cholesterol level (IQR)
mmol/L	5.1 (4.5–5.7)	5.6 (5.0–6.3)	6.1 (5.3–6.8)	6.7 (5.9–7.5)
mg/dL	196.9 (173.7–220.1)	216.2 (193.1–243.2)	235.5 (204.6–262.5)	258.7 (227.8–289.6)
Median HDL cholesterol level (IQR)
mmol/L	1.2 (1.0–1.4)	1.5 (1.2–1.8)	1.2 (1.0–1.4)	1.4 (1.2–1.6)
mg/dL	46.3 (38.6–54.1)	57.9 (46.3–69.5)	46.3 (38.6–54.1)	54.1 (46.3–61.8)
Diabetes mellitus, n (%)	318 (17)	391 (13)	143 (10)	336 (12)
ECG-LVH, n (%)	77 (4)	123 (4)	72 (5)	126 (6)
Median BMI (IQR), kg/m2	26 (24–29)	26 (23–30)	26 (24–27)	27 (24–29)
Median CRP level (IQR), nmol/L	0.23 (0.12–0.42)	0.14 (0.27–0.48)	0.22 (0.10–0.44)	0.21 (0.10–0.38)
Ethnicity, n (%)
White	1628 (85)	2505 (83)	1301 (99)	2444 (99)
Nonwhite	289 (15)	524 (17)	17 (1)	26 (1)
Median ABI (IQR)	1.2 (1.1–1.2)	1.1 (1.0–1.2)	1.2 (1.0–1.3)	1.2 (1.0–1.2)
Median cIMT (IQR)	1.1 (1.0–1.2)	1.0 (0.9–1.1)	1.1 (0.9–1.2)	1.0 (0.9–1.1)

ABI = ankle– brachial index; BMI = body mass index; CHS = Cardiovascular Health Study; cIMT = carotid intima–media thickness; CRP = C-reactive protein; ECG-LVH = electrocardiographic left ventricular hypertrophy; HDL = high-density lipoprotein; IQR = interquartile range; RS = Rotterdam Study.

Figure 1.

Association of age with 10-y actual risk for CHD and the 10-y risk for competing noncoronary death in elderly men and women in the CHS and the RS.

CHD = coronary heart disease.

CHD Prediction

Coronary risk factors were associated with CHD about equally in men and women (Table 2). However, total cholesterol level was predictive of CHD in European but not in U.S. women, so we accounted for cholesterol– cohort interaction in our final prediction model. We also note that systolic BP in men treated for hypertension was not statistically significant and smoking was borderline statistically significant in the multivariate model (Table 2). In women, the positive association of age with CHD was nonlinear and decreased with increasing age. We refer to our model based on established risk factors as the coronary risk in the elderly (CORE) model for the remainder of the text.

Table 2.

Multivariate-Adjusted Risk Factors for CHD in Elderly Participants in the CHS and the RS

Risk Factor	Pooled CHS and RS Data for Men			Pooled CHS and RS Data for Women
	Hazard Ratio (95% CI)*	P Value	P Value for Cohort–Risk Factor Interaction†	Hazard Ratio (95% CI)*	P Value	P Value for Cohort–Risk Factor Interaction†
Age (per 10-y increase)	1.23 (1.10–1.37)	<0.001	0.176	1.68 (1.47–1.91)	<0.001	0.68‡
Treatment vs. no treatment of hypertension (shown for participants with systolic blood pressure of 130 mm Hg)	1.51 (1.29–1.76)	<0.001	0.183	1.33 (1.15–1.55)	<0.001	0.41
Systolic blood pressure
Treated (per 10–mm Hg increase)	1.00 (0.96–1.04)	0.56	0.61	1.08 (1.04–1.12)	<0.001	0.49
Untreated (per 10–mm Hg increase)	1.11 (1.07–1.16)	<0.001	0.106	1.14 (1.09–1.18)	<0.001	0.68
Cholesterol level (per 1-mmol/L [38.6-mg/dL] increase)
Total	1.12 (1.04–1.20)	<0.001	0.068	CHS: 1.04 (0.96–1.13) RS: 1.22 (1.13–1.31)	CHS: 0.31 RS: <0.001	0.004§
HDL	0.69 (0.55–0.86)	<0.001	0.60	0.65 (0.55–0.77)	<0.001	0.74
Current or former smoking vs. never smoking	1.14 (0.97–1.35)	0.114	0.26	1.13 (1.00–1.29)	0.058	0.175
Presence vs. absence of diabetes	1.36 (1.14–1.62)	<0.001	0.69	1.39 (1.18–1.64)	<0.001	0.22

CHD = coronary heart disease; CHS = Cardiovascular Health Study; RS = Rotterdam Study.

^*Hazard ratios for the subdistribution hazards of the Fine and Gray model (29).

^†Test of whether the effect of a risk factor differed between the CHS and the RS.

^‡Because of the nonlinear association of age with CHD in women, the age effect was best modeled with a quadratic polynomial and only the effect of age 79 vs. 69 y is displayed for simplicity. Appendix Table 4 (available at www.annals.org) provides full details and exact regression coefficients. The P value for the cohort–age interaction refers to an overall test of the linear and the quadratic age term.

^§In the final model, a total cholesterol– cohort interaction term was added.

The discriminatory performance of the CORE model was moderate and lower in men than in women (c-statistic, 0.63 in both U.S. and European men and 0.68 and 0.67 in U.S. and European women, respectively) (Table 3). Cross-validation led to loss of discrimination compared with the pooled model (Table 3). Predicted 10-year risks were in good agreement with observed risks in each cohort, indicating good calibration (Appendix Figure). The Appendix includes an example that explains how the CORE model can be applied to derive predicted risks by using the subdistribution hazards and coefficients from Appendix Tables 4 and 5 (available at www.annals.org). A risk calculator developed from our model for the prediction horizons of 3, 5, 7, and 10 years is available online at www.ceb-institute.org/evibox/chd/.

Table 3.

Discriminatory Performance of the CORE Model and FPS in the CHS and the RS

Variable	c-Statistic (95% CI)
	Men	Women
Model derived from pooled CHS and RS data
Evaluated in CHS	0.63 (0.60–0.65)	0.68 (0.65–0.70)
Evaluated in RS	0.63 (0.59–0.66)	0.67 (0.64–0.70)
Cross-validation performance
Model derived in RS and evaluated in CHS	0.60 (0.57–0.63)	0.67 (0.64–0.69)
Model derived in CHS and evaluated in RS	0.62 (0.58–0.65)	0.65 (0.62–0.68)
FPS predictions
Evaluated in CHS	0.60 (0.57–0.63)	0.66 (0.64–0.69)
Evaluated in RS	0.60 (0.56–0.63)	0.65 (0.62–0.69)

CHS = Cardiovascular Health Study; CORE = coronary risk in the elderly; FPS = Framingham point scores; RS = Rotterdam Study.

Comparison With the FPS

We compared the CORE model with the FPS. The c-statistic of the FPS was 0.02 to 0.03 units lower than that of the CORE model in both cohorts (Table 3).

In U.S. men, the CORE model classified many more persons into the high-risk group than did the FPS (47.7% vs. 22.5%) (Table 4). More specifically, compared with the FPS, the CORE model increased classification of events and nonevents as high-risk by 30.6 and 23.8 percentage points, respectively. Similar results were seen for nondiabetic U.S. men (increases in true- and false-positive rates of 24.0 and 16.6 percentage points, respectively) (Appendix Table 6, available at www.annals.org). Of note, observed risks in high-risk nondiabetic men were almost identical in the 2 models (24.6% in the CORE model and 24.8% for the FPS), whereas the number of events classified as high-risk in the CORE model was nearly double that of the FPS (51.4% vs. 27.4%) (Appendix Table 6). Therefore, the increase in the true-positive rate can be ascribed at least in part to better discriminative properties of the CORE model.

Table 4.

Risk Classification According to the CORE Model and FPS

Variable	Men				Women
	<10% Risk	10%–20% Risk	>20% Risk	Total	<10% Risk	10%–20% Risk	>20% Risk	Total
CHS
CORE model
Total, n (%)	35 (1.8)	968 (50.5)	914 (47.7)	1917 (100)	1395 (46.1)	1392 (46.0)	242 (8.0)	3029 (100)
Events, n (%)	2 (0.5)	142 (37.2)	238 (62.3)	382 (100)	78 (22.4)	206 (59.2)	64 (18.4)	348 (100)
Nonevents, n (%)	33 (2.2)	826 (53.8)	676 (44.0)	1535 (100)	1317 (49.1)	1186 (44.2)	178 (6.6)	2681 (100)
Observed risk, %	5.7	14.7	26.0	–	5.6	14.8	26.4	–
FPS
Total, n (%)	230 (12.0)	1255 (65.5)	432 (22.5)	1917 (100)	1976 (65.2)	833 (27.5)	220 (7.3)	3029 (100)
Events, n (%)	24 (6.3)	237 (62.0)	121 (31.7)	382 (100)	150 (43.1)	144 (41.4)	54 (15.5)	348 (100)
Nonevents, n (%)	206 (13.4)	1018 (66.3)	311 (20.3)	1535 (100)	1826 (68.1)	689 (25.7)	166 (6.2)	2681 (100)
Observed risk, %	10.4	18.9	28.0	–	7.6	17.3	24.6	–
	Difference (CORE Model – FPS), percentage points
Change in true-positive rate*		30.6 (62.3 to 31.7)				2.9 (18.4 to 15.5)
Change in false-positive rate†		23.8 (44.0 to 20.3)				0.4 (6.6 to 6.2)
Change in false-negative rate‡		−5.8 (0.5 to 6.3)				−20.7 (22.4 to 43.1)
Change in true-negative rate§		−11.3 (2.2 to 13.4)				−19.0 (49.1 to 68.1)
	<10% Risk	10%–20% Risk	>20% Risk	Total	<10% Risk	10%–20% Risk	>20% Risk	Total
RS
CORE model
Total, n (%)	102 (7.0)	975 (67.1)	377 (25.9)	1454 (100)	1540 (54.1)	1149 (40.3)	160 (5.6)	2849 (100)
Events, n (%)	8 (3.5)	127 (55.5)	94 (41.0)	229 (100)	94 (31.6)	164 (55.2)	39 (13.2)	297 (100)
Nonevents, n (%)	94 (7.7)	848 (69.2)	283 (23.1)	1225 (100)	1445 (56.7)	986 (38.6)	121 (4.7)	2552 (100)
Observed risk, %	7.8	13.0	24.9	–	6.1	14.3	24.6	–
FPS
Total, n (%)	76 (5.2)	803 (55.2)	575 (39.5)	1454 (100)	1155 (40.5)	1159 (40.7)	535 (18.8)	2849 (100)
Events, n (%)	4 (1.8)	107 (46.7)	118 (51.5)	229 (100)	64 (21.5)	128 (43.1)	105 (35.4)	297 (100)
Nonevents, n (%)	72 (5.9)	696 (56.8)	457 (37.3)	1225 (100)	1091 (42.8)	1031 (40.4)	430 (16.8)	2552 (100)
Observed risk, %	5.3	13.3	20.5	–	5.5	11.1	19.7	–
	Difference (CORE Model – FPS), percentage points
Change in true-positive rate*		−10.5 (41.0 to 51.5)				−22.2 (13.2 to 35.4)
Change in false-positive rate†		−14.2 (23.1 to 37.3)				−12.1 (4.7 to 16.8)
Change in false-negative rate‡		1.7 (3.5 to 1.8)				10.1 (31.6 to 21.5)
Change in true-negative rate§		1.8 (7.7 to 5.9)				13.9 (56.7 to 42.8)

CHS = Cardiovascular Health Study; CORE = coronary risk in the elderly; FPS = Framingham point scores; RS = Rotterdam Study.

^*Difference in proportion of events categorized as >20% risk.

^†Difference in proportion of nonevents categorized as >20% risk.

^‡Difference in proportion of events categorized as <10% risk.

^§Difference in proportion of nonevents categorized as <10% risk.

In U.S. women, the CORE model classified fewer events and nonevents as low-risk than the FPS (differences of 20.7 and 19.0 percentage points, respectively).

In Europeans, the CORE model generally classified more persons into lower risk categories than the FPS (Table 4). In European men, the CORE model classified fewer nonevents as high-risk (a difference of 14.2 percentage points) but at the expense of a 10.5–percentage point increase in events misclassified in lower risk strata. The general downward movement of persons with the CORE model was due to systematic overestimation of risk in RS participants with the FPS. For example, observed risks were 11.1% and 19.7% in the intermediate- and high-risk categories using the FPS in European women.

Additional Risk Markers

Of the additional risk markers we evaluated, only the presence of ECG-LVH and low ABI were significantly associated with CHD in both sexes. There was negligible improvement in predictive accuracy when we added ECG-LVH and ABI to the CORE model (an increase in the c-statistic of only 0.019 with an NRI of 3.9% in U.S. men and an increase of only 0.010 with an NRI of 2.5% in European men) (Table 5 and Appendix Tables 7 to 9, available at www.annals.org). In women, adding cIMT, CRP, ABI, and ECG-LVH led to limited improvements in CHD prediction beyond established risk factors. Combining these 4 markers into an extended model resulted in small to moderate predictive improvements in both the U.S. and European populations (the c-statistic increased to 0.69 in both populations with NRIs of 6.8% and 9.6%, respectively) (Table 5 and Appendix Tables 10 to 14, available at www.annals.org).

Table 5.

Reclassification Accuracy of Extended Models Compared With the CORE Model

Additional Risk Marker	Hazard Ratio (95% CI)	Improvement in c-Statistic		NRI, %
		CHS	RS	CHS	RS
Men
ABI (per 0.1-unit increase)	0.58 (0.41–0.81)	0.013	−0.004	3.3	0.3
BMI (per 5-kg/m2 increase)	1.01 (0.99–1.03)	NA*	NA*	NA*	NA*
cIMT (per 1-unit increase on a log-transformed scale)	1.22 (0.91–1.66)	NA*	NA*	NA*	NA*
CRP (per 1-unit increase on a log-transformed scale)	1.00 (1.00–1.01)	NA*	NA*	NA*	NA*
ECG-LVH	2.17 (1.71–2.76)	0.009	0.011	1.7	2.1
ABI plus ECG-LVH	–	0.019	0.010	3.9	2.5
Women
ABI (per 0.1-unit increase)	0.93 (0.90–0.96)	0.001	0.014	0.1	3.6
BMI (per 5-kg/m2 increase)	1.01 (0.99–1.02)	NA*	NA*	NA*	NA*
cIMT (per 1-unit increase on a log-transformed scale)	2.81 (1.96–4.02)	0.010	0.013	2.2	6.1
CRP (per 1-unit increase on a log-transformed scale)	1.08 (1.03–1.13)	0.003	0.004	2.4	0.8
ECG-LVH	1.63 (1.31–2.02)	0.005	0.007	4.1	1.9
ABI plus cIMT plus CRP plus ECG-LVH	–	0.016	0.026	6.8	9.6

ABI = ankle– brachial index; BMI = body mass index; CHS = Cardiovascular Health Study; cIMT = carotid intima–media thickness; CORE = coronary risk in the elderly; CRP = C-reactive protein; ECG-LVH = electrocardiographic left ventricular hypertrophy; NA = not applicable; NRI = net reclassification improvement; RS = Rotterdam Study.

^*Because of a nonsignificant improvement in global model fit.

Discussion

In this study, we report the performance of a sex-specific CHD prediction model tailored for an older population, based on 2 large population-based cohort studies of cardiovascular disease in the elderly. The model accounts for the fact that death from other causes often precludes CHD occurrence. Predicted risks for the presented model were well-calibrated, and risk factors generally showed consistent effects across U.S. and European persons. However, our model had moderate discrimination, its accuracy was not substantially better than the FPS, and adding newer coronary risk markers did not substantially improve risk prediction.

Healthy elderly persons have been promoted as a target for primary prevention of CHD (11, 37). According to the ATP III guideline (1), risk models assessing absolute CHD risk should guide primary preventive measures. However, existing risk-stratification approaches have not considered important characteristics of elderly persons, especially their considerable risk for dying of competing causes rather than CHD. The CORE model addresses this gap, and the 10-year CHD risk it predicts can be used in accordance with the ATP III guideline. In contrast to risk scores based on models that ignore or censor competing events, our model provides real-life and therefore more meaningful estimates of CHD risk for elderly patients and physicians. The 10-year risk for CHD did not exceed approximately 20% in men or 15% in women, and the occurrence of noncoronary death dominated the occurrence of CHD (Figure). This observation refutes the perception that all elderly men are at high risk for CHD (38).

The discriminatory performance of the CORE model was modest compared with those based on younger age ranges; c-statistics of 0.80 or greater have been reported for models based on similar established risk factors (8, 9). However, such comparisons must be interpreted with caution, because inappropriately neglecting a substantial risk for competing noncoronary death leads to apparently high but uninterpretable c-statistics (22). Comparison with the well-known ATP III FPS (1) in the competing-risks setting of our model showed that the FPS had slightly lower accuracy.

Several studies (1, 14–17) have observed that associations of traditional risk factors with CHD diminish with age. For example, smoking is one of the most influential CHD risk factors but had only borderline statistical significance in our cohorts when we used the competing-risk method. The strong association of smoking with death from other causes (such as cancer or chronic obstructive pulmonary disease) competes with the observation and predictability of CHD (39). Considering competing causes of death naturally leads to impairment of nonspecific risk factors (such as age) in predicting CHD (22), in the same way that the benefit of treating a disease may be reduced by other causes of death.

In elderly U.S. persons, improved predictions with the CORE model (such as the 30.6–percentage point increase in true-positive classifications for men) were often paired with risk misclassifications (such as the 23.8–percentage point increase in false-positive classifications for men) (Table 4). In U.S. women, the models primarily differed in classifications of low risk, in which decreases in the false-negative rate were paired with similar decreases in the true-negative rate. However, the decision to use a risk prediction model depends not only on the balance of improved and decreased risk classification but also on the costs and benefits of correct and incorrect classifications, and therefore on such factors as the cost of medication and side effects of treatment. For example, because of the availability of effective treatment for CHD prevention with limited side effects, the increase in the true-positive rate (those who would correctly qualify for treatment) in U.S. men with the CORE model may outweigh the increase in the false-positive rate (those who would receive unnecessary preventive treatment). At the same time, very few U.S. women were classified as high-risk in both models, and therefore only a few women with future CHD events would have received preventive treatment. This raises the question of whether current risk thresholds for treatment allocation need to be reevaluated for elderly women, as has been suggested for younger women (40).

Compared with earlier Framingham risk functions (5, 6), the FPS risk prediction tool is more appropriate for use in older populations because it includes interaction terms for total cholesterol level and smoking with age and has an upper age limit of 79 rather than 74 years (1). Our U.S. study population included a modest proportion of participants aged 80 years or older (15.5% of men and 11.5% of women in the CHS). However, excluding these participants did not lead to meaningful changes in our results. This is consistent with the observation that absolute risk for CHD stabilizes after age 80 years (Figure).

In elderly European persons, the interpretation of model differences was dominated by substantial risk overestimation with the FPS, corresponding with earlier findings of overestimation of Framingham functions in lower-risk European populations (13, 41, 42).

Measures that integrate risk factor information over time, such as measures of abnormal cardiac function, subclinical measures of atherosclerosis, or markers of inflammation, have been suggested to be more promising than traditional risk factors for predicting CHD at older age (43). Our model yielded statistically significant improvements in risk prediction when we added ABI and ECG-LVH for men and ABI, cIMT, CRP, and ECG-LVH for women. However, clinical improvement in risk prediction was small in men and moderate in women (NRIs of up to 9.4% in women in the RS). An improvement of this extent does not outweigh the additional effort required to integrate multiple nontraditional markers into clinical practice. Therefore, we did not include these additional risk markers in our final model. Other markers, such as coronary artery calcification, may perform better in elderly persons, but evaluations in large elderly populations are still lacking.

Our study has limitations. First, the cohorts differed ethnically, particularly in the number of African Americans (who have higher cardiovascular risk). Second, despite the highly similar CHD end point definitions in both cohorts, subtle differences in end point ascertainment may have led to unknown differences in CHD incidence (Appendix Table 3) (27, 28). Third, the CORE model was developed and compared in the same population. However, because the model was largely prespecified by using 2 large cohorts, and the model derived from pooled data showed similar performance in both cohorts, we consider it unlikely that overoptimism has a major effect on the comparison of the CORE model with the FPS. Fourth, our approach to re-classification did not distinguish between persons with competing events and those without an event (both are classified as not having the event of interest). Fifth, the FPS is designed for persons up to age 79 years but was used for older persons in our study as well. Sixth, we examined only a few additional variables in extended models. Finally, we used the recommended cutoff values for risk classification (1), but the appropriateness of these cutoffs in elderly persons is uncertain. Using different values could have affected the results of our reclassification analyses and subsequent clinical implications.

Deaths from noncoronary causes dominate CHD events in elderly persons and therefore pose a challenge to CHD prediction. We developed a model for predicting CHD risk in elderly persons that provides meaningful real-life estimates of absolute CHD risk. Our model showed good generalizability in aging U.S. and European populations, but only moderate discrimination and no consistent improvements in risk classification compared with the FPS. Moreover, adding promising new cardiovascular markers did not substantially improve CHD risk prediction of the model. This emphasizes the need for further work to improve cardiovascular risk prediction in an elderly population.

Context

The Framingham risk score, which predicts 10-year risk for coronary heart disease (CHD), may be less accurate in older populations.

Contribution

These investigators developed a CHD risk prediction model for use in adults older than 65 years that accounts for deaths from non-CHD causes. Although it is methodologically more sophisticated, the new prediction model was about as accurate as the Framingham risk score at predicting CHD risk.

Caution

The findings apply to a mostly white population.

Implication

A new model for predicting CHD in older adults was useful but little better than the existing Framingham risk score in discriminating events and nonevents. Predicting CHD in this population is difficult and hampered by the large number of non-CHD deaths.

—The Editors

Appendix

Baseline Examinations

Of the 5888 participants enrolled in the CHS, we excluded 942 because of definite prevalent cardiovascular disease, usually prevalent MI (60%). Of the 7983 participants enrolled in the RS, all had a home interview and 7153 visited the research center at baseline. Most (67%) RS participants excluded from our analysis were aged 55 to 64 years, and the remainder had prevalent cardiovascular disease. Thus, our final sample comprised 4946 CHS and 4303 RS participants.

Baseline data included information on current health status, history of cardiovascular disease, medication use, and cardiovascular risk factors. Participants were categorized in groups of current, former, or never smokers. Former smoking was defined as having abstained from smoking for at least 2 years. Blood pressure was measured by using a random-zero sphygmomanometer at the right brachial artery in sitting position after a 5-minute rest. The average of 2 consecutive BP measurements was used. In the CHS, participants were asked to fast for 12 hours before coming to their clinical appointments and fasting plasma lipid levels were measured by an Olympus Demand system (Olympus, Lake Success, New York). In the RS, serum total cholesterol level was determined by an automated enzymatic procedure by using the CHOD-PAP reagent agent (Roche Diagnostics, Basel, Switzerland) and serum HDL cholesterol level was measured with the HDL cholesterol assay (Roche Diagnostics) by using polyethylene glycol–modified enzymes and dextran sulfate. Diabetes mellitus was defined as current use of antidiabetic medication or a random or postload serum glucose level greater than 11.0 mmol/L (>198.2 mg/dL). A 12-lead resting electrocardiogram was obtained and stored electronically in both cohorts. In the CHS, the ECG reading center used the Novacode ECG measurement and classification system (23, 44) to analyze ECG data. In the RS, ECG data were computer-analyzed by the MEANS program (27, 45). The presence of ECG-LVH was defined according to the Sokolow–Lyon voltage criteria (46). In the CHS, fasting serum chemistry analyses were done with the Kodak Ektachem 700 Analyzer (Eastman Kodak, Rochester, New York). In the RS, CRP was measured by Rate Near Infrared Particle Immunoassay (Immage Immunochemistry System; Beckman Coulter, Brea, California) (47). The ABI was calculated as the ratio of the systolic BP of the posterior tibial artery, as assessed by an 8-MHz continuous wave Doppler probe and a random-zero sphygmomanometer, to the systolic BP at the arm. The lowest ABI of the left- and right-side readings was used for analysis (48). Duplex ultrasonography of both carotid arteries was performed with a 5.0 MHz transducer (SSA-270A; Toshiba America Medical Systems, Tustin, California) in the CHS and a 7.5-MHz transducer (ATL UltraMark IV; Advanced Technology Laboratories, Bothell, Washington) in the RS. Common cIMT was determined as described elsewhere (49). The maximum common cIMT was determined as the average of the maximum cIMT of near- and far-wall measurements over a length of 1 cm, and the average of left and right maximum common cIMT was computed and used for analysis.

End Point Ascertainment

Appendix Table 3 compares the underlying definitions and Appendix Table 2 provides the incidence of CHD end points (nonfatal MI and coronary death) for the CHS and the RS.

In the CHS, events were ascertained through regular surveys (surveillance calls, annual visits, newspaper obituaries, and reviews of medical records and Medicare data) from the field centers or by participants contacting the sites. Potential events were classified by a study-wide review committee on the basis of death certificates; autopsy and coroner forms; hospital records; and interviews with attending physicians, next of kin, and witnesses (23, 28).

In the RS, information on end points was obtained from general practitioners and discharge reports from medical specialists. All events were classified independently by 2 research physicians. If the physicians disagreed, a consensus was reached in a special session. Finally, all events were verified by a medical specialist affiliated with the study (27).

For the CHS, the first examination cycle began in 1989, with annual examinations through 1999. A second cohort was enrolled in 1992 with the additional recruitment of African Americans. In the RS, baseline examinations were conducted from 1990 to 1993. The censoring date was 30 June 2006 in the CHS and 1 January 2007 in the RS.

For our analysis, only 1 CHS participant and 6 RS participants were lost to follow-up after 10 years.

Additional Details About the Statistical Analysis

Competing-risk methods were used throughout, and model development was based on the Fine and Gray model (29) as implemented in the kmi package for R (30, 31). The cumulative incidence function, which describes the absolute risk for failing from CHD as time progresses (18), is of primary prognostic significance in competing-risk analyses. In line with this function, we used the Fine and Gray model, a multivariate regression model that directly associates covariate effects with the cumulative incidence function via the subdistribution hazard. Hence, the regression coefficients of the model have a direct prognostic interpretation for CHD events (21, 50, 51).

Possible cohort–risk factor interactions were assessed by introducing interaction terms. Total cholesterol level showed a stronger association with CHD among women in the RS than those in the CHS (P = 0.004); we therefore introduced an interaction term in the final competing-risks model in women (Table 3 and Appendix Tables 4 and 15). Nonlinearity of predictors was assessed by comparing natural cubic splines (with 4 and 5 degrees of freedom) with the linear fit by using a likelihood ratio test (17). Because strong evidence indicated a nonlinear effect of age on CHD incidence in women, we included a linear and a quadratic age term in the model (6), which fitted the model equally well in terms of the Akaike Information Criterion as a more complex model using a natural cubic spline function. We tested the proportional subdistribution hazards assumption on the basis of scaled Schoenfeld residuals.

Fewer than 1% of the values for traditional risk factors were missing in CHS participants and up to 4.4% were missing in RS participants who visited the research center at baseline (7153 participants). For variables used as additional risk markers in our extended model, values were missing for up to 11.6% of the variables in the RS, with the exception of cIMT (23.0%). We imputed missing covariates separately for men and women and for the CHS and RS cohorts, defining imputation models that included the outcomes of CHD and competing noncoronary death (52). Multiple imputation of missing data was performed with the contributed mice package in R, and analyses were based on 5 imputed data sets (53). All analyses were additionally done on complete cases to check for potential differences between results based on imputed data and those based on complete cases. With the exception of the baseline characteristics (Table 1), results are reported for imputed data. When calculating the c-statistic, we used multiple imputation of potential censoring times for competing events (30, 31) instead of treating competing events as “censored at infinity” (21).

For graphical display, the actual 10-year risk for the mutually exclusive CHD and competing noncoronary death events was estimated on the basis of the Fine and Gray models with age (included as a natural cubic spline) as the only covariate. The Figure displays the actual predicted risks.

Statistical assessments of the model showed that in men and women, the proportional hazards assumption was violated for age but for none of the remaining covariates. The reported age effects should therefore be interpreted as the weighted average effect over the entire follow-up (50).

We used the cumulative incidence function to estimate the number of events and nonevents displayed in Table 4 and Appendix Tables 6 to 18.

The FPS is designed for persons aged 79 years or younger. Participants aged 80 years or older were assigned an age of 79 years for the purpose of calculating their predicted risks using the FPS. Because the highest possible risk category of the FPS is denoted as 30% or greater but not in terms of a single value, we used a value of 35% for participants with a predicted risk of 30% or greater.

Example of CHD Risk Computation

We show the computation of the predicted risks for CHD at 3, 5, 7, and 10 years for a U.S. or European man or woman aged 65 years or older. Given the advanced age and limited life expectancy of this population, the risk prediction horizon can be adapted flexibly (Appendix Table 5). The baseline cumulative subdistribution hazard refers to a U.S. or European man or woman aged 75 years whose systolic BP is 130 mm Hg, total cholesterol level is 5 mmol/L (193.05 mg/dL), and HDL cholesterol level is 1 mmol/L (38.61 mg/dL) and who does not smoke, is not receiving BP-lowering medication, and is not diabetic.

The actual probability or predicted risk for an individual to have a CHD event at time t depends on the covariate vector (x_il, …, x_ip) and on the baseline cumulative subdistribution hazard (21):

$$I_i(t\mid x_i)=1-exp\left(-exp\left(\sum _{k=1}^p{\beta }_k{x}_{ik}\right)·{\int }_0^t{\overline{\lambda }}_{1,0}(s)ds\right)$$

where

$${\int }_0^t{\overline{\lambda }}_{1,0}(s)ds$$

refers to the cumulative baseline subdistribution hazard (given for different time points in Appendix Table 5), β refers to the vector of coefficients from the Fine and Gray model (29) provided in Appendix Table 4, and exp(β) refers to the hazard ratios given in Table 3.

The following example illustrates the computation of the 10-year CHD risk for a U.S. man aged 72 years who is receiving BP-lowering medication; whose systolic BP is 136 mm Hg, total cholesterol level is 4.4 mmol/L (169.88 mg/dL), and HDL cholesterol level is 1.56 mmol/L (60.23 mg/dL); who is not diabetic; and who is a current smoker.

$$\begin{array}{c}\text{Linear}\text{Predictor}=\text{LP}=\sum _{k=1}^p{\beta }_k{x}_{ik}\\ =[0.205\times (72-75)/10]+[0.410\\ \times 1(\text{use}\text{of}\text{BP}-\text{lowering}\text{medication})]+[-0.0005\times (136\\ -130)/10]+[0.112\times (4.4-5)]\\ +[-0.372\times (1.56-1)]+[0.306\times 0(\text{absence}\text{of}\text{diabetes})]\\ +[0.134\times 1(\text{current}\text{smoking})]=0.2067\end{array}$$

Predicted 10-year risk for CHD = 1 − exp[−0.174 × exp(0.2067)] = 0.192 = 19.2%.

An online calculator, developed from our model, for predicting risk at 3, 5, 7, and 10 years is available online at www.ceb-institute.org/evibox/chd/.

Appendix Figure.

Calibration of predictions computed from the CORE model for men and women in the CHS and the RS.

CORE = coronary risk in the elderly; CHD = coronary heart disease; CHS = Cardiovascular Health Study; RS = Rotterdam Study.

Appendix Table 1.

Steps in Development, Assessment, and Extension of the CORE Model^*

Assumption/Modeling Steps	Test/Comparison/Predictor	Result		Decision
		Men	Women
Linearity assumption of continuous predictors	Restricted cubic spline transformation, 4 to 5 knots; likelihood ratio test against nontransformed (linear) term
	Age	P = 0.60	P < 0.001	Men: Accept linearity Women: Reject linearity assumption; age + age2 appropriate
	Total cholesterol level	P = 0.98	P = 0.21	Accept linearity
	HDL cholesterol level	P = 0.60	P = 0.72	Accept linearity
	Systolic blood pressure	P = 0.22	P = 0.25	Accept linearity
Risk factor–cohort interaction	All predictors	P > 0.05
	Total cholesterol level		P = 0.004	Add total cholesterol–cohort interaction term
	Remaining predictors		P > 0.05
Proportional subdistribution hazard assumption	Age	P < 0.001	NA	Interpret as weighted average effect over the follow-up (50)
	Age + age2	NA	P < 0.001
	Remaining predictors	P > 0.05	P > 0.05	Interpret as weighted average effect over the follow-up (50)
Model extension	ABI	P = 0.001; best fit as linear term	P < 0.001; best fit as linear term	Retain term
	ECG-LVH	P < 0.001	P < 0.001	Retain term
	cIMT	P > 0.05	P < 0.001; best fit as log(cIMT)	Men: Reject term Women: Retain term as log(cIMT)
	CRP	P > 0.05	P < 0.001; best fit as log(CRP)	Men: Reject term Women: Retain term as log(CRP)
	BMI	P > 0.05	P > 0.05	Reject term

ABI = ankle–brachial index; BMI = body mass index; cIMT = carotid intima–media thickness; CORE = coronary risk in the elderly; CRP = C-reactive protein; ECG-LVH = electrocardiographic left ventricular hypertrophy; HDL = high-density lipoprotein; NA = not applicable.

^*Tables 3 and 5 and Appendix Tables 4 and 5 provide more details.

Appendix Table 2.

Summary Event Table With 10-Year Cumulative Incidence of CHD or Competing Death in the CHS and the RS

Variable	CHS		RS
	Men (n = 1917)	Women (n = 3029)	Men (n = 1454)	Women (n= 2849)
Overall CHD events (10-y cumulative incidence), n (%)	563 (19.9)	603 (11.5)	283 (15.8)	415 (10.4)
Nonfatal MI (10-y cumulative incidence), n (%)	343 (12.6)	338 (7.0)	128 (7.3)	121 (3.0)
Fatal MI (10-y cumulative incidence), n (%)	47 (1.5)	51 (1.0)	24 (1.4)	49 (1.3)
Atherosclerotic CHD death (10-y cumulative incidence), n (%)	173 (5.8)	214 (3.5)	131 (7.1)	245 (6.1)
Competing non-CHD death (10-y cumulative incidence), n (%)	839 (26.6)	1161 (19.1)	777 (38.5)	1467 (35.5)
Median follow-up, y	16.6	16.5	14.8	14.9
Person-years of follow-up	19 664	36 845	12 965	27 876

CHD = coronary heart disease; CHS = Cardiovascular Health Study; MI = myocardial infarction; RS = Rotterdam Study.

Appendix Table 3.

Definitions of CHD End Points Used in the CHS and the RS^*

End Point	CHS	RS
Nonfatal MI	ECG and/or cardiac enzyme changes	ECG and cardiac enzyme changes
Fatal MI	MI ≤28 d before death and no known nonatherosclerotic cause of death	MI ≤28 d before death and no known nonatherosclerotic cause of death
Atherosclerotic CHD death	Chest pain ≤72 h before death and no known nonatherosclerotic cause of death	Cardiac pain ≤72 h before death and no known nonatherosclerotic cause of death
	History of chronic ischemic heart disease in the absence of valvular heart disease or nonischemic cardiomy- opathy and no known nonatherosclerotic cause of death	History of ischemic heart disease in the absence of valvular heart disease or nonischemic cardiomyopathy and no known nonatherosclerotic cause of death
	Death certificate consistent with atherosclerotic CHD death and no known nonatherosclerotic cause of death	Mode of death consistent with CHD in the absence of significant valvular heart disease or nonischemic cardiomy- opathy and no known nonatherosclerotic cause of death
	Coronary death related to CHD procedures, such as PCI or CABG	Coronary death related to CHD procedures, such as PCI or CABG

CABG = coronary artery bypass grafting; CHD = coronary heart disease; CHS = Cardiovascular Health Study; ECG = electrocardiography; MI = myocardial infarction; PCI = percutaneous coronary intervention; RS = Rotterdam Study.

^*From references 27 (RS) and 28 (CHS).

Appendix Table 4.

Coefficients for Men and Women in the CORE Model

Predictor	Scaling	Men	Women
Age, y
Linear	(Age – 75)/10	0.205	0.463
Quadratic	[(Age – 75)/10]2	NA	−0.262
Treatment vs. no treatment of hypertension	None	0.410	0.288
SBP
Treated (per 10–mm Hg increase)	(SBP – 130)/10	−0.0005	0.080
Untreated (per 10–mm Hg increase)	(SBP – 130)/10	0.107	0.127
Cholesterol level (per 1-mmol/L [38.6-mg/dL] increase)
TC	TC – 5	0.112	U.S.: 0.041 European: 0.198
HDL-C	HDL-C – 1	−0.372	−0.432
Current or former vs. never smoking	None	0.134	0.125
Presence vs. absence of diabetes	None	0.306	0.330

CORE = coronary risk in the elderly; HDL-C = high-density lipoprotein cholesterol; NA = not applicable; SBP = systolic blood pressure; TC = total cholesterol.

Appendix Table 5.

Cumulative Baseline Subdistribution Hazard for Different Time Horizons for Elderly U.S. and European Persons^*

Population	3 Years	5 Years	7 Years	10 Years
U.S. elderly persons
Men	0.049	0.086	0.120	0.174
Women	0.025	0.046	0.074	0.125
European elderly persons
Men	0.040	0.065	0.074	0.125
Women	0.019	0.032	0.047	0.064

^*Data are based on the Cardiovascular Health Study for U.S. persons and the Rotterdam Study for European persons.

Appendix Table 6.

Risk Classification According to the CORE Model and FPS in Nondiabetic Participants

Variable	Men				Women
	<10% Risk	10%–20% Risk	>20% Risk	Total	<10% Risk	10%–20% Risk	>20% Risk	Total
CHS
CORE model
Total, n (%)	42 (2.6)	945 (59.2)	609 (38.2)	1596 (100)	1516 (57.6)	1018 (38.7)	98 (3.7)	2632 (100)
Events, n (%)	2 (0.7)	140 (47.9)	150 (51.4)	292 (100)	85 (32.2)	159 (60.2)	20 (7.6)	264 (100)
Nonevents, n (%)	40 (3.1)	805 (61.7)	459 (35.2)	1304 (100)	1431 (60.4)	859 (36.3)	78 (3.3)	2368 (100)
Observed risk, %	4.8	14.8	24.6	–	5.6	15.6	20.4	–
FPS
Total, n (%)	214 (13.4)	1060 (66.4)	322 (20.2)	1596 (100)	1784 (67.8)	691 (26.3)	157 (6.0)	2632 (100)
Events, n (%)	23 (7.9)	189 (64.7)	80 (27.4)	292 (100)	123 (46.6)	112 (42.4)	29 (11.0)	264 (100)
Nonevents, n (%)	191 (14.6)	871 (66.8)	242 (18.6)	1304 (100)	1661 (70.1)	579 (24.5)	128 (5.4)	2368 (100)
Observed risk, %	10.7	17.8	24.8	–	6.9	16.2	18.5	–
	Difference (CORE Model – FPS), percentage points
Change in true-positive rate*		24.0 (51.4 to 27.4)				−3.4 (7.6 to 11.0)
Change in false-positive rate†		16.6 (35.2 to 18.6)				−2.1 (3.3 to 5.4)
Change in false-negative rate‡		−7.2 (0.7 to 7.9)				−14.4 (32.2 to 46.6)
Change in true-negative rate§		−11.6 (3.1 to 14.6)				−9.7 (60.4 to 70.1)
	<10% Risk	10%–20% Risk	>20% Risk	Total	<10% Risk	10%–20% Risk	>20% Risk	Total
RS
CORE model
Total, n (%)	183 (14.0)	957 (73.2)	168 (12.8)	1308 (100)	1516 (60.7)	890 (35.6)	92 (3.7)	2498 (100)
Events, n (%)	13 (6.7)	134 (68.7)	48 (24.6)	195 (100)	93 (37.6)	130 (52.6)	24 (9.8)	247 (100)
Nonevents, n (%)	170 (15.3)	823 (73.9)	120 (10.8)	1113 (100)	1423 (63.2)	760 (33.8)	68 (3.0)	2251 (100)
Observed risk, %	7.1	14.0	28.6	–	6.1	14.6	26.3	–
FPS
Total, n (%)	70 (5.4)	735 (56.2)	503 (38.5)	1308 (100)	1066 (42.7)	999 (40.0)	433 (17.3)	2498 (100)
Events, n (%)	4 (2.1)	92 (47.2)	99 (50.8)	195 (100)	55 (22.2)	107 (43.3)	85 (34.5)	247 (100)
Nonevents, n (%)	66 (5.9)	643 (57.8)	404 (36.3)	1113 (100)	1011 (44.9)	892 (39.6)	348 (15.5)	2251 (100)
Observed risk, %	5.7	12.5	19.7	–	5.2	10.7	19.7	–
	Difference (CORE Model – FPS), percentage points
Change in true-positive rate*		−26.1 (24.6 to 50.8)				−24.7 (9.8 to 34.5)
Change in false-positive rate†		−25.5 (10.8 to 36.3)				−12.4 (3.0 to 15.5)
Change in false-negative rate‡		4.6 (6.7 to 2.1)				15.4 (37.6 to 22.2)
Change in true-negative rate§		9.3 (15.3 to 5.9)				18.3 (63.2 to 44.9)

CHS = Cardiovascular Health Study; CORE = coronary risk in the elderly; FPS = Framingham point scores; RS = Rotterdam Study.

^*Difference in proportion of events categorized as >20% risk.

^†Difference in proportion of nonevents categorized as >20% risk.

^‡Difference in proportion of events categorized as <10% risk.

^§Difference in proportion of nonevents categorized as <10% risk.

Appendix Table 7.

Risk Reclassification for Men in the CHS and RS After Extending the CORE Model With ABI^*

Risk Classification in the CORE Model	Risk Classification in the CORE Model + ABI
	CHS				RS
	<10%	10%–20%	>20%	Total	<10%	10%–20%	>20%	Total
<10%
Total, n	20	15	0	35	81	19	2	102
Events, n	1	1	0	2	7	0	1	8
Nonevents, n	19	14	0	33	74	19	1	94
10%–20%
Total, n	10	903	55	968	32	862	81	975
Events, n	0	127	15	142	1	109	17	127
Nonevents, n	10	776	40	826	31	753	64	848
>20%
Total, n	0	145	769	914	0	101	276	377
Events, n	0	21	217	238	0	17	77	94
Nonevents, n	0	124	552	676	0	84	199	283
Total
Total, n	30	1063	824	1917	113	982	359	1454
Events, n	1	149	232	382	8	126	95	229
Nonevents, n	29	914	592	1535	105	856	264	1225

ABI = ankle– brachial index; CHS = Cardiovascular Health Study; CORE = coronary risk in the elderly; RS = Rotterdam Study.

^*Numbers in interior cells were rounded because of participants with incomplete follow-up and may not add up to the margin total (36).

Appendix Table 8.

Risk Reclassification for Men in the CHS and RS After Extending the CORE Model With ECG-LVH^*

Risk Classification in the CORE Model	Risk Classification in the CORE Model + ECG-LVH
	CHS				RS
	<10%	10%–20%	>20%	Total	<10%	10%–20%	>20%	Total
<10%
Total, n	30	5	0	35	96	5	1	102
Events, n	2	0	0	2	7	1	0	8
Nonevents, n	28	5	0	33	89	4	1	94
10%–20%
Total, n	4	930	34	968	9	915	51	975
Events, n	0	133	9	142	0	115	12	127
Nonevents, n	4	797	25	826	9	800	39	848
>20%
Total, n	0	106	808	914	0	87	290	377
Events, n	0	18	220	238	0	15	79	94
Nonevents, n	0	88	588	676	0	72	211	283
Total
Total, n	34	1041	842	1917	105	1007	342	1454
Events, n	2	151	229	382	7	131	91	229
Nonevents, n	32	890	613	1535	98	876	251	1225

CHS = Cardiovascular Health Study; CORE = coronary risk in the elderly; ECG-LVH = electrocardiographic left ventricular hypertrophy; RS = Rotterdam Study.

^*Numbers in interior cells were rounded because of participants with incomplete follow-up and may not add up to the margin total (36).

Appendix Table 9.

Risk Reclassification for Men in the CHS and RS After Extending the CORE Model With Both ABI and ECG-LVH^*

Risk Classification in the CORE Model	Risk Classification in the CORE Model + ABI + ECG-LVH
	CHS				RS
	<10%	10%–20%	>20%	Total	<10%	10%–20%	>20%	Total
<10%
Total, n	20	15	0	35	81	19	2	102
Events, n	1	1	0	2	7	0	1	8
Nonevents, n	19	14	0	33	74	19	1	94
10%–20%
Total, n	10	903	55	968	32	862	81	975
Events, n	0	127	15	142	1	109	17	127
Nonevents, n	10	776	40	826	31	753	64	848
>20%
Total, n	0	145	769	914	0	101	276	377
Events, n	0	21	217	238	0	17	77	94
Nonevents, n	0	124	552	676	0	84	199	283
Total
Total, n	30	1063	824	1917	113	982	359	1454
Events, n	1	149	232	382	8	126	95	229
Nonevents, n	29	914	592	1535	105	856	264	1225

ABI = ankle– brachial index; CHS = Cardiovascular Health Study; CORE = coronary risk in the elderly; ECG-LVH = electrocardiographic left ventricular hypertrophy; RS = Rotterdam Study.

^*Numbers in interior cells were rounded because of participants with incomplete follow-up and may not add up to the margin total (36).

Appendix Table 10.

Risk Reclassification for Women in the CHS and RS After Extending the CORE Model With ABI^*

Risk Classification in the CORE Model	Risk Classification in the CORE Model + ABI
	CHS				RS
	<10%	10%–20%	>20%	Total	<10%	10%–20%	>20%	Total
<10%
Total, n	1324	71	0	1395	1429	111	0	1540
Events, n	76	2	0	78	83	11	0	94
Nonevents, n	1248	69	0	1317	1346	100	0	1446
10%–20%
Total, n	81	1250	61	1392	123	947	79	1149
Events, n	14	173	19	206	11	134	19	164
Nonevents, n	67	1077	42	1186	112	813	60	985
>20%
Total, n	0	54	188	242	0	23	137	160
Events, n	0	7	57	64	0	5	34	39
Nonevents, n	0	47	131	178	0	18	103	121
Total
Total, n	1405	1375	249	3029	1552	1081	216	2849
Events, n	90	182	76	348	94	150	53	297
Nonevents, n	1315	1193	173	2681	1458	931	163	2552

ABI = ankle– brachial index; CHS = Cardiovascular Health Study; CORE = coronary risk in the elderly; RS = Rotterdam Study.

^*Numbers in interior cells were rounded because of participants with incomplete follow-up and may not add up to the margin total (36).

Appendix Table 11.

Risk Reclassification for Women in the CHS and RS After Extending the CORE Model With cIMT^*

Risk Classification in the CORE Model	Risk Classification in the CORE Model + cIMT
	CHS				RS
	<10%	10%–20%	>20%	Total	<10%	10%–20%	>20%	Total
<10%
Total, n	1290	104	1	1395	1445	95	0	1540
Events, n	68	10	0	78	82	12	0	94
Nonevents, n	1222	94	1	1317	1363	83	0	1446
10%–20%
Total, n	157	1145	90	1392	186	906	57	1149
Events, n	17	171	18	206	15	132	17	164
Nonevents, n	140	974	72	1186	171	774	40	985
>20%
Total, n	0	54	188	242	0	33	127	160
Events, n	0	6	58	64	0	5	34.3	39.3
Nonevents, n	0	48	130	178	0	28	92.7	120.7
Total
Total, n	1447	1303	279	3029	1631	1034	184	2849
Events, n	85	187	76	348	97	149	51	297
Nonevents, n	1362	1116	203	2681	1534	885	133	2552

CHS = Cardiovascular Health Study; cIMT = carotid intima–media thickness; CORE = coronary risk in the elderly; RS = Rotterdam Study.

^*Numbers in interior cells were rounded because of participants with incomplete follow-up and may not add up to the margin total (36).

Appendix Table 12.

Risk Reclassification for Women in the CHS and RS After Extending the CORE Model With CRP^*

Risk Classification in the CORE Model	Risk Classification in the CORE Model + CRP
	CHS				RS
	<10%	10%–20%	>20%	Total	<10%	10%–20%	>20%	Total
<10%
Total, n	1318	77	0	1395	1491	49	0	1540
Events, n	69	9	0	78	87	7	0	94
Nonevents, n	1249	68	0	1317	1404	42	0	1446
10%–20%
Total, n	93	1254	45	1392	141	989	19	1149
Events, n	10	183	13	206	12	146	6	164
Nonevents, n	83	1071	32	1186	129	843	13	985
>20%
Total, n	0	32	210	242	0	23	137	160
Events, n	0	5	59	64	0	9	30	39
Nonevents, n	0	27	151	178	0	14	107	121
Total
Total, n	1411	1363	255	3029	1632	1061	156	2849
Events, n	79	197	72	348	99	162	36	297
Nonevents, n	1332	1172	177	2681	1533	899	120	2552

CHS = Cardiovascular Health Study; CORE = coronary risk in the elderly; CRP = C-reactive protein; RS = Rotterdam Study.

^*Numbers in interior cells were rounded because of participants with incomplete follow-up and may not add up to the margin total (36).

Appendix Table 13.

Risk Reclassification for Women in the CHS and RS After Extending the CORE Model With ECG-LVH^*

Risk Classification in the CORE Model	Risk Classification in the CORE Model + ECG-LVH
	CHS				RS
	<10%	10%–20%	>20%	Total	<10%	10%–20%	>20%	Total
<10%
Total, n	1373	22	0	1395	1475	65	0	1540
Events, n	76	2	0	78	86	8	0	94
Nonevents, n	1297	20	0	1317	1389	57	0	1446
10%–20%
Total, n	48	1303	41	1392	94	999	56	1149
Events, n	2	189	15	206	9	141	14	164
Nonevents, n	46	1114	26	1186	85	858	42	985
>20%
Total, n	0	47	195	242	0	28	132	160
Events, n	0	6	58	64	0	8	31	39
Nonevents, n	0	41	137	178	0	20	101	121
Total
Total, n	1421	1372	236	3029	1569	1092	188	2849
Events, n	78	197	73	348	95	157	45	297
Nonevents, n	1343	1175	163	2681	1474	935	143	2552

CHS = Cardiovascular Health Study; CORE = coronary risk in the elderly; ECG-LVH = electrocardiographic left ventricular hypertrophy; RS = Rotterdam Study.

^*Numbers in interior cells were rounded because of participants with incomplete follow-up and may not add up to the margin total (36).

Appendix Table 14.

Risk Reclassification for Women in the CHS and RS After Extending the CORE Model With ABI, cIMT, CRP, and ECG-LVH^*

Risk Classification in the CORE Model	Risk Classification in the CORE Model + ABI + cIMT + CRP + ECG-LVH
	CHS				RS
	<10%	10%–20%	>20%	Total	<10%	10%–20%	>20%	Total
<10%
Total, n	1260	134	1	1395	1384	156	0	1540
Events, n	62	16	0	78	76	18	0	94
Nonevents, n	1198	118	1	1317	1308	138	0	1446
10%–20%
Total, n	218	1064	110	1392	238	800	111	1149
Events, n	26	145	35	206	22	105	37	164
Nonevents, n	192	919	75	1186	216	695	74	985
>20%
Total, n	0	79	163	242	0	36	124	160
Events, n	0	10	54	64	0	7	32	39
Nonevents, n	0	69	109	178	0	29	92	121
Total
Total, n	1478	1277	274	3029	1622	992	235	2849
Events, n	88	171	89	248	98	130	69	297
Nonevents, n	1390	1106	185	2781	1524	862	166	2552

ABI = ankle– brachial index; CHS = Cardiovascular Health Study; cIMT = carotid intima–media thickness; CORE = coronary risk in the elderly; CRP = C-reactive protein; ECG-LVH = electrocardiographic left ventricular hypertrophy; RS = Rotterdam Study.

^*Numbers in interior cells were rounded because of participants with incomplete follow-up and may not add up to the margin total (36).

Appendix Table 15.

Risk Reclassification According to the CORE Model and FPS in CHS Participants^*

Risk Classification in the FPS	Risk Classification in the CORE Model
	Men				Women
	<10%	10%–20%	>20%	Total	<10%	10%–20%	>20%	Total
<10%
Total, n	26	186	18	230	1350	618	8	1976
Events, n	1	21	2	24	74	74	2	150
Nonevents, n	25	165	16	206	1276	544	6	1826
10%–20%
Total, n	9	749	497	1255	45	695	93	833
Events, n	1	113	123	237	4	117	23	144
Nonevents, n	8	636	374	1018	41	578	70	689
>20%
Total, n	0	33	399	432	0	79	141	220
Events, n	0	8	112	121	0	15	39	54
Nonevents, n	0	25	287	311	0	64	102	166
Total
Total, n	35	968	914	1917	1395	1392	242	3029
Events, n	2	142	238	382	78	206	64	348
Nonevents, n	33	826	676	1535	1317	1186	178	2681

CHS = Cardiovascular Health Study; CORE = coronary risk in the elderly; FPS = Framingham point scores.

^*Numbers in interior cells were rounded because of participants with incomplete follow-up and may not add up to the margin total (36).

Appendix Table 16.

Risk Reclassification According to the CORE Model and FPS in RS Participants^*

Risk Classification in the FPS	Risk Classification in the CORE Model
	Men				Women
	<10%	10%–20%	>20%	Total	<10%	10%–20%	>20%	Total
<10%
Total, n	48	28	0	76	1023	129	3	1155
Events, n	3	1	0	4	52	12	0	64
Nonevents, n	45	27	0	72	971	117	3	1091
10%–20%
Total, n	54	649	100	803	491	636	32	1159
Events, n	5	80	22	107	40	83	5	128
Nonevents, n	49	569	78	696	451	553	27	1031
>20%
Total, n	0	298	277	575	26	384	125	535
Events, n	0	46	72	118	2	69	34	105
Nonevents, n	0	252	205	457	24	315	91	430
Total
Total, n	102	975	377	1454	1540	1149	160	2849
Events, n	8	127	94	229	94	164	39	297
Nonevents, n	94	848	283	1225	1446	985	121	2552

CORE = coronary risk in the elderly; FPS = Framingham point scores; RS = Rotterdam Study.

^*Numbers in interior cells were rounded because of participants with incomplete follow-up and may not add up to the margin total (36).

Appendix Table 17.

Risk Reclassification According to the CORE Model and FPS in Nondiabetic CHS Participants^*

Risk Classification in the FPS	Risk Classification in the CORE Model
	Men				Women
	<10%	10%–20%	>20%	Total	<10%	10%–20%	>20%	Total
<10%
Total, n	32	175	7	214	1439	342	3	1784
Events, n	0	22	1	23	81	42	0	123
Nonevents, n	32	153	6	191	1358	300	3	1661
10%–20%
Total, n	10	732	318	1060	77	584	30	691
Events, n	2	108	79	189	4	99	9	112
Nonevents, n	8	624	239	871	73	485	21	579
>20%
Total, n	0	38	284	322	0	92	65	157
Events, n	0	10	70	80	0	18	11	29
Nonevents, n	0	28	214	242	0	74	54	128
Total
Total, n	42	945	609	1596	1516	1018	98	2632
Events, n	2	140	150	292	85	159	20	264
Nonevents, n	40	805	459	1304	1431	859	78	2368

CHS = Cardiovascular Health Study; CORE = coronary risk in the elderly; FPS = Framingham point scores.

^*Numbers in interior cells were rounded because of participants with incomplete follow-up and may not add up to the margin total (36).

Appendix Table 18.

Risk Reclassification According to the CORE Model and FPS in Nondiabetic RS Participants^*

Risk Classification in the FPS	Risk Classification in the CORE Model
	Men				Women
	<10%	10%–20%	>20%	Total	<10%	10%–20%	>20%	Total
<10%
Total, n	59	11	0	70	985	80	1	1066
Events, n	3	1	0	4	48	7	0	55
Nonevents, n	56	10	0	66	937	73	1	1011
10%–20%
Total, n	118	589	28	735	506	474	19	999
Events, n	10	75	7	92	42	63	2	107
Nonevents, n	108	514	21	643	464	411	17	892
>20%
Total, n	6	357	140	503	25	336	72	433
Events, n	0	58	41	99	3	60	22	85
Nonevents, n	6	299	99	404	22	276	50	348
Total
Total, n	183	957	168	1308	1516	890	92	2498
Events, n	13	134	48	195	93	130	24	247
Nonevents, n	170	823	120	1113	1423	760	68	2251

CORE = coronary risk in the elderly; FPS = Framingham point scores; RS = Rotterdam Study.

^*Numbers in interior cells were rounded because of participants with incomplete follow-up and may not add up to the margin total (36).