Failure of Traditional Risk Factors to Adequately Predict Cardiovascular Events in Older Populations

Abstract

Background:

Accurate assessment of atherosclerotic cardiovascular disease (ASCVD) risk across heterogeneous populations is needed for effective primary prevention. Little is known about the performance of standard cardiovascular risk factors in older adults.

Objective:

To evaluate the performance of the American College of Cardiology/American Heart Association Pooled Cohort Equations (PCE) risk model, as well as the underlying cardiovascular risk factors, among adults over age 65.

Design and Setting:

Retrospective cohort derived from a regional referral system’s electronic medical records.

Participants:

25,349 patients who were aged ≥65 years at study baseline (date of the first outpatient lipid panel taken between 2007 and 2010).

Measurements:

Exposures of interest were traditional cardiovascular risk factors as defined by inclusion in the PCE model. The primary outcome was major ASCVD events, defined as a composite of myocardial infarctions, stroke and cardiovascular death.

Results:

The PCE and internally-estimated models produced similar risk distributions for Caucasian men aged 65–74. For all other groups, PCE predictions were generally lower than those of the internal models, particularly for African Americans. Discrimination of the PCE was poor for all age groups, with concordance index [95% confidence interval] estimates of 0.62 [0.60—0.64], 0.56 [0.54—0.57], and 0.52 [0.49—0.54] among patients aged 65–74, 75–84, and ≥85 years, respectively. Re-estimating relationships within these age groups resulted in better calibration but negligible improvements in discrimination. Blood pressure, total cholesterol and diabetes were either not associated at all or had inverse associations in the older age groups.

Conclusion:

Traditional clinical risk factors for cardiovascular disease failed to accurately characterize risk in a contemporary population of Medicare-aged patients. Among those aged ≥85 years, some traditional risk factors were not associated with ASCVD events. Better risk models are needed in order to appropriately inform treatment decision making for the growing population of older adults.

Introduction

Effective primary prevention of atherosclerotic cardiovascular disease (ASCVD) requires the accurate assessment of risk across a broad, heterogeneous population to identify those most likely to benefit from medical therapy. The American College of Cardiology and the American Heart Association (ACC/AHA) have established the Pooled Cohort Equations (PCE) risk model to inform initial ASCVD risk assessment and advocated for its use in decisions to prescribe cholesterol-lowering and antihypertensive therapies.^1,2

The PCE includes standard cardiovascular risk factors (age; sex; race; blood pressure; total and high density lipoprotein [HDL] cholesterol; smoking; diabetes; and antihypertensive medication use) and was derived from cohort data on 24,626 individuals aged 40–79 years. Several studies have investigated the discrimination (ability to differentiate between individuals who will and who will not experience ASCVD events, defined throughout this paper as nonfatal myocardial infarction, nonfatal stroke or cardiovascular death, unless otherwise noted) and calibration (agreement between estimated ASCVD event-free survival probabilities and observed frequencies) of the PCE. Validation studies have suggested both over- and under-estimation of risk, depending on the contexts.^3–6 A recent study found that incrementally improved overall calibration is achievable with the use of more contemporary data.⁷

However, these models assume a fixed set of relationships within four distinct populations defined according to combinations of race and sex. While age is included as a risk factor, the models do not allow for changes in risk relationships as patients age. While the models are not intended for individuals over age 79, some online risk calculators nonetheless provide estimates for adults over age 79 by truncating age to 79 years^*. Clinicians who use ASCVD risk estimation tools that disallow age values of 80 or higher, such as the official ACC tool^†, may nonetheless truncate age to 79 years. Validation studies of these models to date have included relatively small numbers of the oldest old.⁶

Increases in the size and diversity of the older population have intensified the need for research examining the determinants of cardiovascular risk within that population. In this study, we (1) evaluated the performance of the PCE among older populations, and (2) evaluated the extent to which accuracy of cardiovascular risk models that are based on established clinical risk factors might be improved by estimating model coefficients separately for older patients.

Methods

Under approval by the Cleveland Clinic Institutional Review Board, we obtained retrospective clinical data on a cohort of 26,772 Cleveland Clinic Health System patients above age 65 who had no ASCVD diagnoses and received an outpatient lipid panel between 2007 and 2010 and who lived in Cuyahoga County or its neighboring counties at the time the lipid panel was performed. These data were extracted from the electronic health record and supplemented with vital records data from the Ohio Department of Health (via a Data Use Agreement). The date of the first qualifying outpatient lipid panel served as study baseline. We excluded individuals who were not of Caucasian or African American race due to the fact that the PCE is defined only for these races. Also excluded were a small number of patients with incomplete data on the cardiovascular risk factors involved in the PCE risk equations.

Patients who were prescribed statins within the preceding year were included in the main analysis after adjustments to total cholesterol and HDL cholesterol to account for the effects of statin therapy. We assumed statins would reduce total cholesterol by 21% and increase HDL cholesterol by 3.5%.⁸ We also adjusted 5-year ASCVD risk estimates downward by 21% to account for the average relative risk reduction for major ASCVD events estimated from pooled analyses of randomized clinical trial data^‡.⁸ We conducted a sensitivity analysis in which patients who were prescribed statins prior to baseline were excluded.

Study Variables

The outcome variable was time to major ASCVD event, defined as stroke, myocardial infarction or cardiovascular death. This outcome was considered to be censored at the earlier of a) the start date of any contiguous 2-year period with no documented clinical encounters or b) the date of noncardiac death or death due to unspecified cause. Myocardial infarction and stroke were defined by using International Classification of Diseases, Ninth Revision, Clinical Modification, or International Classification of Diseases, Tenth Revision (ICD-10), diagnosis codes from all CCHS encounters during follow-up. Cardiovascular death was defined on the basis of ICD-10 cause-of-death codes I00 to I79.

PCE-predicted 5-year ASCVD event risk was calculated using the stratified Cox proportional hazards regression model coefficients reported in Goff et al. (2014)¹ and 5-year baseline hazard estimates reported in the appendix of Muntner et al. (2014).⁶

Age at baseline was used as a continuous variable in calculating the PCE risk equations. In re-estimating the risk model within age strata, we used the following categorization: 65–74 years, 75–84 years, and ≥85 years.

Analytic Approach

Demographic, socioeconomic and clinical characteristics were summarized separately for the baseline age categories given above, using standard univariable summary statistics (mean and standard deviation for symmetrically-distributed continuous variables; median and quartiles for asymmetrically-distributed continuous variables; or counts and percentages for categorical variables). PCE-estimated 5-year ASCVD event risk was also summarized for these age categories, both in aggregate and within subgroups defined according to sex and race.

Performance of the PCE was evaluated within each of the three chronological age groups using concordance indices and calibration plots. The concordance index measures the ability of a given risk model to differentiate among events and non-events; it ranges in value from 0.5 (representing no discriminative ability beyond random guessing) to 1.0 (perfect discrimination of events from non-events). We estimated 95% confidence intervals for the concordance indices using normal approximation theory for proportions. For the calibration plots, we used quadratic functions to model potential nonlinearities.

To assess the extent to which re-estimation of the association between traditional cardiovascular risk factors and time to first ASCVD event improved risk model performance within our data, we derived internal 5-year ASCVD event risk estimates based on a series of stratified Cox models. Separate models were estimated for the three age groups, each of which allowing for distinct baseline hazard functions for males and females. Included as predictors in these models were the same cardiovascular risk factors as that of the PCE. An interaction term involving systolic blood pressure and antihypertensive use was considered for the models but removed due to lack of statistical significance. Discrimination and calibration of these models were assessed as described above. Proportional hazards assumptions were evaluated by estimation correlations between model predictors and scaled Schoenfeld residuals. We evaluated potential heterogeneity of age group-specific relationships relating to sex by estimating different (age group-specific) models that each incorporated distinct hazard ratio estimates for males and females. (These models did not include separate baseline hazard functions.)

All analyses were conducted using R version 3.5.0 (R Foundation for Statistical Computing, Vienna, Austria, 2018). A reproducible analytic workflow, including an image of the analytic dataset used for study results and all statistical code, was produced via the R package ‘projects’⁹ and is archived on secure research servers at Cleveland Clinic.

Results

Of the 26,772 patients meeting initial inclusion criteria, 1,150 (4.3%) were of neither African American nor Caucasian race. Of the remaining 25,622 patients, an additional 273 (1.1%) were removed due to missing risk factor data (229 missing cholesterol and 46 missing blood pressure). The main analysis included 25,349 patients.

Distributions of ASCVD risk factors in the sample reflected decreased proportions of male sex and African American race in older age groups relative to younger age groups (Table 1). Advancing age was also associated with increased prevalence of antihypertensive use, decreases in prevalence of smoking and Type II diabetes, increased pulse pressure, and decreased total cholesterol.

Table 1.

Summary of cardiovascular risk factors (presented as either prevalence; mean ± standard deviation; or median [first quartile, third quartile]) in relation to categories defined according to chronological age at study baseline. HDL = high-density lipoprotein; PCE = Pooled Cohort Equations Risk Model; ASCVD = atherosclerotic cardiovascular disease.

	Age at Study Baseline
Variable	65–74 Years	75–84 Years	85+ Years
	(N = 15,290)	(N = 8,082)	(N = 1,977)
Sex (% male)	45	40	34
Age (years)	69 [66, 71]	79 [76, 81]	87 [86, 89]
African American Race (%)	14	14	12
Total Cholesterol (mg/dL)	175 ± 46	168 ± 46	164 ± 45
HDL Cholesterol (mg/dL)	57 ± 17	58 ± 17	58 ± 17
Antihypertensive Use (%)	69	80	86
Statin Use (%)	53	55	52
Type II Diabetes Mellitus (%)	22	23	18
Smoking (%)	11	6	3
Systolic Blood Pressure (mmHg)	130 ± 14	132 ± 15	133 ± 16
Diastolic Blood Pressure (mmHg)	76 ± 9	73 ± 9	70 ± 9
PCE-Estimated 5-Year ASCVD Event Risk (%)	5.3 [3.2, 8.3]	12.6 [8.9, 17.7]	26.4 [19.8, 35.2]
- Among African American Males	8.9 [6.2, 12.3]	10.7 [7.8, 15.2]	13.7 [10.4, 17.1]
- Among Caucasian Males	7.8 [5.6, 10.7]	15.7 [11.9, 20.5]	25.8 [21.3, 32.9]
- Among African American Females	5.6 [3.7, 8.2]	8.8 [6.0, 13.3]	13.3 [9.3, 18.7]
- Among Caucasian Females	3.3 [2.2, 4.9]	11.2 [8.0, 15.9]	29.4 [22.3, 39.1]

There were 5,608 patients (22.1%) whose outcome data were censored prior to 5 years post study baseline. Hazard ratios from the internally-estimated models are provided in Table 2; whereas some traditional risk factors, such as age, low HDL and smoking, were associated with outcomes at all ages, others, such as blood pressure, total cholesterol and diabetes, were either not associated at all or had inverse associations. Risk equations for calculating 5-year ASCVD event probability estimates based on these models are provided in eTable 1 in the Supplement.

Table 2.

Hazard ratios and corresponding 95% confidence intervals from internally estimated stratified Cox regression models for time to first ASCVD event relative to study baseline. Separate models were estimated for each age category. Hazard ratios associated with patient sex are not included due to the fact that the models incorporated separate baseline hazard functions for males and females (baseline 5-year cumulative hazard estimates are provided in the Appendix)

		Age at Study Baseline
Parameter	Level	65–74 Years	75–84 Years	≥85 Years
Age (per 5 years)		1.36 [1.23, 1.50]	1.38 [1.24, 1.52]	1.47 [1.26, 1.71]
African American race		1.51 [1.31, 1.74]	1.11 [0.94, 1.31]	1.13 [0.87, 1.46]
Total Cholesterol	≤150 mg/dL	Reference	Reference	Reference
	151–200 mg/dL	0.75 [0.66, 0.86]	0.93 [0.82, 1.06]	0.71 [0.57, 0.87]
	201–250 mg/dL	0.70 [0.59, 0.83]	0.80 [0.67, 0.95]	0.65 [0.50, 0.86]
	>250 mg/dL	0.78 [0.57, 1.07]	1.04 [0.76, 1.41]	0.74 [0.44, 1.24]
HDL Cholesterol	≤40 mg/dL	1.35 [1.13, 1.60]	1.39 [1.15, 1.67]	1.44 [1.08, 1.93]
	41–50 mg/dL	1.05 [0.90, 1.24]	1.18 [1.01, 1.40]	1.08 [0.83, 1.41]
	51–60 mg/dL	Reference	Reference	Reference
	>60 mg/dL	0.83 [0.70, 0.97]	0.98 [0.84, 1.15]	0.96 [0.76, 1.23]
Systolic Blood Pressure	≤120 mmHg	1.16 [1.00, 1.36]	1.04 [0.89, 1.22]	1.13 [0.88, 1.46]
	121–130 mmHg	Reference	Reference	Reference
	131–140 mmHg	1.06 [0.91, 1.25]	0.92 [0.78, 1.07]	1.09 [0.85, 1.39]
	>140 mmHg	1.32 [1.13, 1.55]	1.06 [0.91, 1.24]	1.07 [0.83, 1.37]
Antihypertensive use		1.62 [1.38, 1.90]	1.52 [1.28, 1.80]	1.42 [1.05, 1.93]
Type II diabetes mellitus		1.50 [1.33, 1.70]	1.20 [1.06, 1.37]	1.03 [0.83, 1.30]
Current smoking		1.70 [1.45, 1.98]	1.40 [1.13, 1.73]	2.14 [1.43, 3.19]

Distributions of 5-year ASCVD event risk estimates are displayed for both the PCE and the internally-derived stratified Cox models in Figure 1. The PCE and the internal risk models produced similar distributions of risk for Caucasian men aged 65–74. For all other groups, the PCE tended to produce risk predictions that were lower than those of the internal models (see eTable 2 in the Supplement). This was particularly true for African Americans.

Figure 1:

Density plots of predicted 5-year ASCVD event risk from the ACC/AHA Pooled Cohort Equations Risk Model and from an internally derived stratified Cox proportional hazards regression model, by age category, sex and race.

Discrimination and calibration results for the PCE and internal risk models are reported for each baseline age category in Figure 2. The discrimination of the PCE was poor for all age groups, with concordance index [95% confidence interval] estimates of 0.62 [0.60, 0.64] among patients aged 65–74 years; 0.56 [0.54, 0.57] among patients aged 75–84 years; and 0.52 [0.49, 0.54] among patients aged ≥85 years. Performance in this last group was not significantly different from chance. Calibration of the PCE was reasonable among patients aged 65–74 years, although PCE estimates were consistently lower than observed 5-year event rates by a small margin. Among the other two age groups, calibration was poorer, with almost no relationship between observed and predicted event rates among patients aged ≥85 years. In contrast, the internally-estimated stratified Cox model performed better than the PCE with respect to both discrimination and calibration, although concordance indices were only in the moderate range. C-statistics ranged from 0.67 among patients aged 65–74 years to 0.61 for patients over 75 years. There was little evidence of violation of the proportional hazards assumption for our internal risk models, based on the scaled Schoenfeld residuals. Re-estimated internal risk models that allowed for sex-related heterogeneity in relationships indicated that African American race and current smoking were more strongly associated with worse outcomes among women age 65–84 than among men (see eTable 3 in the Supplement). There were no other notable differences in relationships between men and women. Prediction performance of these models was comparable to our primary internal risk models.

Figure 2:

Calibration plots of predicted 5-year ASCVD event risk from the ACC/AHA Pooled Cohort Equations Risk Model and from an internally derived stratified Cox proportional hazards regression model, by age category. Curves are visualized for the middle 95% of the distribution of predicted risk within each age category. Given in the lower right corner of the panels are concordance index (C) estimates, with corresponding 95% confidence intervals.

Results of our sensitivity analysis, which included the subset of 11,806 patients who were not prescribed statins in the year prior to study baseline, are given in the Supplement (see eMethods, eTables 4 and 5, and eFigures 1 and 2). Among this subgroup of patients, risk densities from the PCE better aligned with internally-derived risk densities for patients aged 65–74 years, while the overall underestimation of risk persisted among African Americans in the 75–84 year and ≥85 year age groups. Calibration relationships and concordance index estimates were largely similar to those obtained from the primary analysis.

Discussion

In this large sample of patients representing the typical elderly population of a large regional health system, we found substantial changes in the relationship between traditional cardiovascular risk factors and ASCVD events among patients over age 65. In particular, relationships involving systolic blood pressure, total cholesterol and diabetes weakened as a function of age. We also found that the PCE, which assigns fixed weights to these risk factors regardless of age, performed reasonably well for Caucasian males aged 65–74, but underestimated risk for everyone else. This was particularly true for African Americans and females aged 75–84 years. Most surprising was the total lack of relationship between risk predicted by the PCE and observed outcomes among those aged ≥85 years.

Our findings are important for two reasons. First, recent guidelines for the treatment of cardiovascular disease, including the prescribing of statins, antihypertensives and aspirin, recommend a risk-based approach.¹⁰ In order to appropriately manage risk, we must be able to accurately measure it. Second, our understanding of cardiovascular disease is predicated on the association of risk factors with outcomes. If particular risk factors are not associated with outcomes after a certain age, it indicates an incomplete understanding of the causal pathway. In particular, the lack of relationship between total cholesterol and cardiovascular events among older patients casts doubt on the efficacy of cholesterol lowering therapies in this high risk group.

Beginning in 2013, the ACC/AHA guidelines recommend statin therapy be based on cardiovascular risk as determined by the PCE. The guidelines recommend statins for adults aged 40–75 of any race if they have a 10-year risk of >7.5%. Treatment for patients over 75 years of age is controversial. The Choosing Wisely Campaign recommends against it, whereas the British National Institute for Health and Care Excellence recommends statins for primary prevention up to age 84.¹¹ The ACC/AHA recommends shared decision making for primary prevention after age 75, based on the idea that all such patients are high risk, but that older patients may also suffer an increased rate of side effects, and if they have comorbid illness may not live long enough to benefit from statin therapy.¹⁰ One exception is for older patients with diabetes, for whom they recommend statins. They also recommend continuing statins for those who began them before age 75. The guideline document notes that evidence for these recommendations is weak. In all of these cases, age-based risk is the foundation of the recommendation, but our findings highlight the dangers of simply extrapolating the PCE (or applying estimates for individuals over age 80 using the maximum allowed age for the model of 79 years) to assess risk in this population. Even for patients aged 65–74, the predictions were inaccurate for non-white patients and women. Moreover, we found that after age 75, diabetes was not as strong a risk factor for CHD and after 85, was not a risk factor at all, raising questions about the ACC/AHA recommendation in this age group.

Higher rates of frailty and functional limitations are not captured by common clinical assessments of cardiovascular risk.^12–15 Factors such as reduced mobility, loss of muscle mass, and loss of bone density are likely to contribute to biological processes that promote ASCVD and/or ASCVD-related events.^16–19 Moreover, the prevalence, chronicity and severity of conditions are more pronounced among older adults.¹⁸ Future efforts to improve upon prediction of ASCVD events in aged populations will need to consider incorporating general and specific measures of frailty as well as other descriptors of risk that are concentrated in this population.²⁰

Other studies have considered the accuracy of the PCE in older populations. The first, by Muntner et al., did not have sufficient size to assess its performance in these subgroups: They attempted to validate the PCE using data from the Reasons for Geographic and Racial Differences in Stroke study.⁶ Among 3,333 Medicare participants who were aged ≥65 years and not taking statins at study baseline, they found generally good model calibration (concordance index 0.67), but most patients were white and the mean age was 70.7 years¹², a population in whom we also found reasonably good performance. Similarly, Nanna et al. applied the PCE to 2663 participants of four prospective cardiovascular cohorts who were aged ≥75 years and also found poor discriminative performance (concordance index 0.62) and miscalibration.²¹

In acknowledging the limitations of the PCE as a tool for individualized treatment decisions, the new ACC/AHA guidelines recommend considering “risk enhancers” (such as chronic kidney disease, metabolic syndrome, certain chronic inflammatory diseases, family history of cardiovascular disease, and certain biomarkers) in tailoring risk-based prevention decisions.¹⁰ However, the recommendation was made without formal incorporation of such factors into quantitative risk algorithms. The evidence does not support unaided physician judgments to consistently and accurately provide effect sizes for these factors²² and thus additional study is warranted to determine the extent to which these and other risk enhancers are important for predicting risk among older patients.

The changing relationship between traditional risk factors and outcomes as patients age has not been thoroughly investigated, despite literature indicating as such dating back to at least 1994, in which Krumholz et al. found that elevated total cholesterol, low HDL cholesterol and high total cholesterol to HDL cholesterol ratio were unassociated with all-cause mortality, mortality from coronary heart disease and hospitalization for myocardial infarction or unstable angina.²³ For men, mortality pressures associated with ASCVD begin to increase between ages 40 and 65, and for women, these pressures begin to increase between ages 50 and 65.²⁴ The groups of patients in our study therefore represent a series of residual cohorts.²⁵ It may be that individuals vary in their susceptibility to the risks of high cholesterol, and that individuals with higher susceptibility may not survive past these ages, whereas those who do survive are less likely to develop coronary disease or have other natural protections against the effects of cholesterol. Paradoxically, we found that reduced levels of total cholesterol were generally associated with increased risk of ASCVD events. These results persisted in our sensitivity analysis of patients who were not being prescribed statins, and suggest a need to better understand the mechanisms that lead to cholesterol decline in the elderly. Smoking and male sex remained risk factors at all ages, implying a similar mechanism of action, and highlighting the potential benefits of smoking cessation, even among adults over the age of 85.

Our study is limited by evaluating patients from a single regional health system. Cleveland Clinic patients generally are reflective of a metropolitan, middle-to-upper class patient population. While we studied a large cohort of 25,349 patients, this cohort accounted for approximately 13% of the population of adults over the age of 65 who lived in Cuyahoga County in 2010.

We assumed homogeneous effects of statin therapy on total and HDL cholesterol and ensuing risk of ASCVD events. Previous research has found that the relative risk reduction for ASCVD events attributed to statins is greater among patients at lower risk levels than among patients at higher risk levels.²⁶ More broadly, treatment effectiveness can be influenced by a variety of factors, including socioeconomic characteristics, heterogeneity in statin effects across the lifespan, patient preferences, and behaviors; this remains an underexplored area of research. Nonetheless, our sensitivity analysis which excluded those who were previously prescribed statins yielded comparable results to our primary analysis which included these patients.

Ascertainment of ASCVD events might be reduced due to our use of routinely collected clinical data. However, our cohort of patients consisted of patients at the extremes of age that were many times larger than typically available in prospective cohort studies. Patients in the oldest age category were more likely to have had censored ASCVD outcome data than patients in the youngest age category (38% vs. 19% censoring rate). Lastly, deaths that are documented as occurring as a result of cardiovascular disease within state vital records may be nonspecific to whether or not ASCVD events actually occurred prior to death.

In conclusion, we found that traditional clinical risk factors for cardiovascular disease failed to accurately characterize risk in a contemporary population of Medicare-aged patients. Among those aged ≥85 years, traditional risk factors were not associated with cardiovascular events or mortality. The PCE exhibited clinically significant underestimates of risk in African American and to a lesser extent Caucasian female populations. Re-estimating equations within age categories based on the same set of traditional risk factors failed to improve model performance to a degree that would be supportive of risk-based prevention decisions. Investigations of other risk factors for ASCVD outcomes are needed in order to appropriately inform treatment decision making for the growing population of older adults.