Abstract
Background
Assessment of atherosclerotic cardiovascular disease (ASCVD) risk is crucial for prevention and management, but the performance of the pooled cohort equations in older adults with frailty and multimorbidity is unknown. We evaluated the pooled cohort equations in these subgroups and the impact of competing risks.
Methods and Results
In 4249 community‐dwelling adults, aged ≥65 years, from the CHS (Cardiovascular Health Study), we calculated 10‐year risk of hard ASCVD. Frailty was determined using the Fried phenotype. Latent class analysis was used to identify individuals with multimorbidity patterns using chronic conditions. We assessed discrimination using the C‐statistic and calibration by comparing predicted ASCVD risks with estimated risk using cause‐specific and cumulative incidence models, by multimorbidity patterns and frailty status. A total of 917 (21.6%) participants had an ASCVD event, and 706 (16.6%) had a competing event of death. C‐statistic was 0.68 in men and 0.69 in women; calibration was good when compared with cause‐specific and cumulative incidence estimated risks (males, −0.1% and 3.3%; females, 0.6% and 1.4%). Latent class analysis identified 4 patterns: minimal disease, cardiometabolic, low cognition, musculoskeletal‐lung depression. In the cardiometabolic pattern, ASCVD risk was overpredicted compared with cumulative incidence risk in men (7.4%) and women (6.8%). Risk was underpredicted in men (−10.7%) and women (−8.2%) with frailty compared with cause‐specific risk. Miscalibration occurred mostly at high predicted risk ranges.
Conclusions
ASCVD prediction was good in this cohort of adults aged ≥65 years. Although calibration varied by multimorbidity patterns, frailty, and competing risks, miscalibration was mostly present at high predicted risk ranges and thus less likely to alter decision making for primary prevention therapy.
Keywords: atherosclerotic cardiovascular disease, frailty, multimorbidity, older adults, risk prediction
Subject Categories: Aging, Cardiovascular Disease, Primary Prevention, Risk Factors
Nonstandard Abbreviations and Acronyms
- ASCVD
atherosclerotic cardiovascular disease
- CHS
Cardiovascular Health Study
- IR
incidence rate
- PCE
pooled cohort equation
- PY
person‐years
Clinical Perspective
What Is New?
Overall atherosclerotic cardiovascular disease prediction by the pooled cohort equations was good in adults aged ≥65 years from the CHS (Cardiovascular Health Study), but calibration varied by multimorbidity patterns, frailty, and competing risks.
What Are the Clinical Implications?
Miscalibration occurred mostly at high predicted risk ranges and is thus less likely to alter decision making for primary prevention therapy for atherosclerotic cardiovascular disease.
Older adults are disproportionately affected by atherosclerotic cardiovascular diseases (ASCVDs). 1 Assessing the risk for ASCVD is important for prevention, management, and risk communication for shared decision making. The 2013 and 2019 updated American College of Cardiology/American Heart Association practice guideline recommends using the pooled cohort equations (PCEs) to estimate 10‐year risk of events to initiate primary prevention. 2 , 3 The accuracy of the PCE and guideline recommendations has been questioned in the general population and in specific subpopulations, 4 particularly those aged >75 years. 5 , 6 Previous studies have reported overestimation of ASCVD risks by the PCE and have suggested updating or recalibrating the prediction models. 7 , 8 , 9 , 10 The performance of the PCE remains underexplored in older adults, in whom risk prediction is complicated by heterogeneity in health status, such as frailty and multimorbidity, 11 , 12 and competing health problems. 13
Frailty 14 is associated with adverse cardiovascular outcomes and may add prognostic information beyond traditional risk factors. 15 Frailty status at baseline has been used to stratify older adults before coronary artery bypass graft and transcatheter aortic valve replacement. 16 The relation between cardiovascular risk factors and ASCVD may differ across frailty levels, 17 , 18 and frailty status might modify the relation between PCE risk estimation and observed outcomes. In addition to frailty, ASCVD risk in older adults may be modified by other baseline comorbidities besides traditional risk factors. Multimorbidity, defined as the co‐occurrence of ≥2 chronic conditions, 19 has been associated with functioning, hospitalizations, and emergency department visits in older adults. 20 , 21 Identifying common multimorbidity patterns and assessing their implications on ASCVD risk prediction may further help characterize the PCE performance in this population.
Although chronological age is a major risk factor for ASCVD, 1 it is also associated with increasing rates of competing risk of death by non‐ASCVD causes (eg, cancer, lung diseases, and dementia). Although including older participants from the CHS (Cardiovascular Health Study) as one of the cohorts, the PCEs were derived in relatively younger aggregate population and using cause‐specific modeling, which assumes that individuals who are censored after experiencing a competing event would remain at risk for ASCVD. In populations with lower competing risk, the choice of competing risk modeling strategy may not significantly influence calibration of ASCVD risk estimates. However, as the risk for competing events increases in older adults, 13 the development of PCEs for cause‐specific estimates may result in overestimation of true risk and overtreatment. 22 The difference between using cause‐specific versus cumulative incidence estimates of events has not been described for PCE predictions of ASCVD in older adults.
In this study, we analyzed data from the CHS, one of the cohorts used in deriving the PCE, to assess the performance (discrimination and calibration) of PCE for 10‐year risk of ASCVD in older adults, specifically investigating subgroups of frailty and multimorbidity classes. We also aimed to quantify the impact of competing risk on ASCVD risk estimation by comparing PCE‐predicted risk with cause‐specific and cumulative incidence estimates.
Methods
Cohort
Data are from the CHS, which enrolled participants in 1988 to 1989 and a supplementary cohort of Black participants in 1992 to 1993. 23 The CHS enrolled 5888 adults aged ≥65 years from 4 US communities, excluding people who were institutionalized, were in a hospice program, were under active treatment for cancer, were cognitively unable to sign an informed consent, did not expect to remain in the community for 3 years, or required a proxy respondent. 24 In concordance with the PCE derivation population, we further excluded 1639 participants with ASCVD (coronary heart disease, stroke, transient ischemic attack, congestive heart failure, percutaneous coronary intervention, and coronary artery bypass grafting) and atrial fibrillation at baseline. The final cohort comprised 4249 participants. This secondary analysis study was approved by the institutional review board at the Centre Hospitalier de l'Université de Montréal. The data are not directly available on request; request for access to the original data may be directed to the CHS Coordinating Center.
Predictors and Estimation of ASCVD Risk
We computed 10‐year ASCVD risk according to the equations provided by the Pooled Cohort Equations Work Group with the following predictors assessed at baseline: age, sex, race (White, other [which included Native Americans/Alaskan natives, Asian, Pacific Islander, and those who reported other race/ethnicity], or Black), total cholesterol, high‐density lipoprotein cholesterol, systolic blood pressure, treatment for high blood pressure, diabetes mellitus, and smoking status. 2 Total cholesterol, high‐density lipoprotein cholesterol, and systolic blood pressure were measured on the first visit. Medication use was directly collected from prescription bottles. 23
Measurement of Frailty
We operationalized frailty using the frailty phenotype based on 5 criteria measured at baseline: exhaustion, low physical activity, slowness, weakness, and shrinking. 14 Exhaustion was present when participants reported that “everything I did was an effort” or “I could not get going” at least 3 to 4 days per week. Low physical activity was defined as <383 kcal of physical activity per week for men and <270 kcal of physical activity per week for women on the Minnesota Leisure Time Activity questionnaire. Weakness and slowness were defined using the absolute cutoffs from the original definition of the physical frailty phenotype (handgrip strength in the lowest quintile of 8 sex–body mass index categories [Table S1] and gait speed in the lowest quintile of 4 sex‐height categories [Table S2] in the full CHS cohort). 14 Shrinking was present when participants reported losing more than 10 pounds unintentionally. Participants meeting ≥3 criteria were considered “frail”; those with 1 or 2 criteria, “prefrail”; and those without any criterion, “robust.”
Chronic Conditions and Multimorbidity
Of the 9 chronic conditions, 8 were assessed by asking participants whether a physician had told them that they had: hypertension, diabetes mellitus, kidney disease, arthritis, osteoporosis, lung disease, depression, and cancer. Low cognition was defined as a Modified Mini‐Mental State Examination score <80. Chronic conditions assessed at baseline or the second year were combined. We performed latent class analysis using these 9 variables (see Analysis below) to identify multimorbidity patterns.
Outcome and Competing Risks
The primary composite end point over 10 years of follow‐up was hard ASCVD comprising nonfatal or fatal myocardial infarction or nonfatal or fatal stroke, consistent with the outcome definition used by the PCE. The ASCVD end points were adjudicated by the CHS Events Committee. 23 , 25 Coronary heart disease deaths were ascertained by a study‐wide Mortality Review Committee using information from death certificates, autopsy and coroner's form, hospital records, and interviews. We categorized causes of competing risks of mortality as “noncardiovascular death” and “other ASCVD or cardiovascular death.” Other ASCVD deaths were considered as competing risks to mirror the original PCE outcome definition.
Descriptive Statistics
Descriptive statistics are presented using mean and median for continuous variables and counts and percentages for categorical variables, stratified by frailty levels and by multimorbidity patterns.
Multiple Imputation and Latent Class Analysis
Analyses were performed using R 3.5.1 (R Foundation for Statistical Computing, Vienna, Austria). Single multivariate imputation using chained equations (micepackage) was used to impute baseline missing data for PCE variables (<1%), component of frailty, and chronic conditions (9.5% for osteoporosis, 5.6% for grip strength, 5.0% for weight loss, 4.7% for the Modified Mini‐Mental State Examination, and <2% for the remaining variables) using available information on demographic, frailty, chronic conditions, self‐reported health, and disability. To identify discrete patterns of multimorbidity, we used latent class analysis (poLCA package) including information on 9 chronic conditions in the entire cohort. We examined models with 2 to 7 multimorbidity classes, and we selected the final model based on a combination of Akaike information criterion, Bayes information criterion, and clinical interpretability of resulting classes. We assigned participants to a class based on the highest probability of class membership.
Model Performance: Discrimination and Calibration
To evaluate the discrimination of PCE predictions, we estimated the C‐statistic, 26 , 27 using the ASCVD predicted risk as predictor. The CIs were adjusted for false discovery rate (q=0.05). 28 For calibration, we computed the mean predicted probability of event and compared with the mean observed probability of event across the whole cohort. 29 Discrimination and calibration were evaluated for the whole cohort and by sex, multimorbidity pattern grouping, and frailty phenotype (robust, prefrail, and frail) subgroups. We examined calibration in the large and calibration plots (cutoffs: 0%, 7.5%, 20%, 30%, 50%, and 100%) in all subgroups comparing 2 different competing risk modeling strategies for estimating the observed risks: (1) cause‐specific model, which assumes that a participant with a competing risk event is censored uninformatively (Kaplan‐Meier estimate) at the time of the competing event; and (2) cumulative incidence model, which assumes that a participant having a competing risk event remains in the risk pool but is “immune” to ASCVD for the remaining follow‐up (because of death from another cause). 22 , 30 Because CHS was one of the original derivation cohorts for the PCE, analyses of discrimination and calibration in the large were computed using bootstrap resampling (n=1000). Finally, we estimated the incidence rate (IR) of competing events for each multimorbidity pattern and frailty status.
Results
Among the 4249 participants, the mean age was 72.4 (SD, 5.4) years and 643 (38.7%) were men. At baseline, 1654 (38.9%) were robust, 2188 (51.5%) were prefrail, and 407 (9.6%) were frail. Age and the prevalence of chronic conditions increased with frailty level. The full demographic characteristics, prevalence of self‐reported chronic conditions, frailty components, PCE variables, and IRs for ASCVD and competing risk events are presented in Table 1 for the overall cohort and by subgroups of frailty status and multimorbidity classes (see below). Arthritis (50.7%) and hypertension (40.4%; 38.3% treated) were the most common self‐reported chronic conditions. The prevalence of all 9 chronic conditions progressively increased with greater levels of frailty; chronic conditions were better separated by multimorbidity classes as identified by latent class analysis.
Table 1.
Variable | Overall | Frailty Phenotype Status | Multimorbidity Class | |||||
---|---|---|---|---|---|---|---|---|
Robust | Prefrail | Frail | Minimal Disease | Cardiometabolic | Low Cognition | Musculoskeletal‐Lung Depression | ||
Sample size, n (%) | 4249 | 1654 (38.9) | 2188 (51.5) | 407 (9.6) | 2617 (61.6) | 307 (7.2) | 351 (8.3) | 974 (22.9) |
Age, y (%) | 72.4 (5.4) | 71.1 (4.5) | 72.7 (5.5) | 75.5 (6.7) | 71.8 (5.1) | 72.2 (5.1) | 76.2 (6.8) | 72.6 (5.4) |
Men, n (%) | 1643 (38.7) | 708 (42.8) | 826 (37.8) | 109 (26.8) | 1109 (42.4) | 127 (41.4) | 180 (51.3) | 227 (23.3) |
Self‐reported comorbidities, n (%) | ||||||||
Hypertension | 1718 (40.4) | 598 (36.2) | 921 (42.1) | 199 (48.9) | 867 (33.1) | 305 (99.3) | 156 (44.4) | 390 (40.0) |
Diabetes mellitus | 585 (13.8) | 166 (10.0) | 317 (14.5) | 102 (25.1) | 161 (6.2) | 298 (97.1) | 69 (19.7) | 57 (5.9) |
Kidney disease | 88 (2.1) | 21 (1.3) | 49 (2.2) | 18 (4.4) | 2 (0.1) | 26 (8.5) | 7 (2.0) | 53 (5.4) |
Arthritis | 2155 (50.7) | 703 (42.5) | 1178 (53.8) | 274 (67.3) | 894 (34.2) | 177 (57.7) | 149 (42.5) | 935 (96.0) |
Osteoporosis | 337 (7.9) | 93 (5.6) | 185 (8.5) | 59 (14.5) | 33 (1.3) | 0 (0.0) | 6 (1.7) | 298 (30.6) |
Lung disease | 959 (22.6) | 336 (20.3) | 504 (23.0) | 119 (29.2) | 291 (11.1) | 81 (26.4) | 17 (4.8) | 570 (58.5) |
Depression | 821 (19.3) | 136 (8.2) | 498 (22.8) | 187 (45.9) | 219 (8.4) | 65 (21.2) | 99 (28.2) | 438 (45.0) |
Low cognition* | 445 (10.5) | 78 (4.7) | 266 (12.2) | 101 (24.8) | 0 (0.0) | 35 (11.4) | 351 (100.0) | 59 (6.1) |
Cancer | 586 (13.8) | 230 (13.9) | 296 (13.5) | 60 (14.7) | 348 (13.3) | 48 (15.6) | 30 (8.5) | 160 (16.4) |
Frailty components, n (%) | ||||||||
Low grip strength | 987 (23.2) | 0 (0.0) | 693 (31.7) | 294 (72.2) | 496 (19.0) | 78 (25.4) | 128 (36.5) | 285 (29.3) |
Low gait speed | 1225 (28.8) | 0 (0.0) | 883 (40.4) | 342 (84.0) | 571 (21.8) | 126 (41.0) | 178 (50.7) | 350 (35.9) |
Low activity | 888 (20.9) | 0 (0.0) | 586 (26.8) | 302 (74.2) | 448 (17.1) | 89 (29.0) | 127 (36.2) | 224 (23.0) |
Exhaustion | 684 (16.1) | 0 (0.0) | 452 (20.7) | 232 (57.0) | 236 (9.0) | 71 (23.1) | 86 (24.5) | 291 (29.9) |
Weight loss | 479 (11.3) | 0 (0.0) | 325 (14.9) | 154 (37.8) | 263 (10.0) | 42 (13.7) | 49 (14.0) | 125 (12.8) |
Other PCE variables | ||||||||
Cholesterol, mean (SD), mg/dL | 213 (39) | 212 (37) | 214 (39) | 211 (42) | 213 (38) | 207 (41) | 210 (42) | 215 (38) |
HDL, mean (SD), mg/dL | 56 (16) | 56 (16) | 56 (16) | 55 (17) | 56 (16) | 49 (13) | 55 (16) | 58 (16) |
SBP, mean (SD), mm Hg | 136 (21) | 135 (21) | 137 (21) | 139 (23) | 135 (21) | 149 (23) | 140 (22) | 136 (21) |
Hypertension treated, n (%) | 1626 (38.3) | 544 (32.9) | 880 (40.2) | 202 (49.6) | 824 (31.5) | 269 (87.6) | 150 (42.7) | 383 (39.3) |
Active smoker, n (%) | 535 (12.6) | 189 (11.4) | 295 (13.5) | 51 (12.5) | 308 (11.8) | 31 (10.1) | 53 (15.1) | 143 (14.7) |
High‐risk (≥20%) PCE prediction, n (%) | 2113 (49.7) | 714 (43.2) | 1131 (51.7) | 268 (65.8) | 1127 (43.1) | 284 (92.5) | 254 (72.4) | 448 (46.0) |
IR of ASCVD events, per 1000 PY (95% CI) | 25.9 (24.3–27.7) | 20.1 (17.9–22.6) | 27.0 24.7–29.5) | 49.5 (41.6–58.5) | 21.6 (19.7–23.6) | 50.4 (41.5–60.6) | 41.9 (34.3–50.8) | 26.4 (22.9–30.2) |
IR of competing events, per 1000 PY (95% CI) | 20.0 (18.5–21.5) | 13.8 (12.0–15.8) | 21.7 (19.6–23.9) | 41.2 (34.0–49.5) | 16.0 (14.4–17.8) | 25.4 (19.2–32.9) | 43.5 (35.7–52.5) | 22.1 (19.0–25.6) |
ASCVD indicates atherosclerotic cardiovascular disease; HDL, high‐density lipoprotein; IR, incidence rate; PCE, pooled cohort equation; PY, person‐years; and SBP, systolic blood pressure.
Modified Mini‐Mental State Examination score <80.
Multimorbidity Classes
Of models that allowed 2 to 7 classes, the 3‐class model had the optimal Akaike information criterion and the 5‐class model had the optimal Bayes information criterion (Table S3). The 4‐class model had the second‐lowest statistics when considering both Akaike information criterion and Bayes information criterion and was selected because of best clinical interpretability. The class selection process, model classification results, and distribution of chronic conditions are detailed in Data S1, Table S4, and Figures S1, S2. Classes were named according to chronic conditions having excess prevalence compared with population prevalence. Participants were classified into the following classes: minimal disease (n=2617, 61.6%), cardiometabolic (n=307, 7.2%), low cognition (n=351, 8.3%), and musculoskeletal‐lung depression (n=974, 22.9%). For example, in the cardiometabolic class, 99.3% of participants had hypertension, 97.1% had diabetes mellitus, and 8.5% had kidney disease.
Outcomes and Competing Events
Over the 10‐year follow‐up, 917 (21.6%) participants had a hard ASCVD event: 414 (9.7%) had nonfatal myocardial infarction, 271 (6.4%) had nonfatal stroke, 144 (3.4%) died because of coronary heart disease, and 88 (2.1%) died because of stroke. No participant was lost to follow‐up. The IR of ASCVD event overall was 25.9 per 1000 person‐years (PY; 95% CI, 24.3–27.7 PY) and increased with greater frailty level from 20.1 to 49.5 per 1000 PY. Among multimorbidity classes, IR of ASCVD event was lowest in the minimal disease class (21.6 per 1000 PY; 95% CI, 19.7–23.6 PY) and highest in the cardiometabolic class (50.4 per 1000 PY; 95% CI, 41.5–60.6 PY). Of note, 706 (16.6%) participants had a competing event: 664 (15.6%) died because of noncardiovascular causes and 42 (1.0%) died because of other ASCVD or other cardiovascular causes. Similarly to the IR of ASCVD event, the IR of competing risk event increased with greater frailty level; however, among multimorbidity classes, competing risk events were highest in the low cognition class (43.5 per 1000 PY; 95% CI, 35.7–52.5 PY). Figure 1 shows ASCVD events and the increasing proportion of competing events with follow‐up years, even exceeding ASCVD events during years 8 to 10.
PCE Discrimination and Calibration, and Comparison With Events
Table 2 presents the discrimination and calibration of PCE predictions with competing events, by sex, multimorbidity classes, and frailty levels. In the overall cohort, the C‐statistic for discrimination was 0.68 (95% CI, 0.65–0.71) in men and 0.69 (95% CI, 0.67–0.72) in women. The PCE was well calibrated in the large with minimal differences between predicted ASCVD risks and the observed risk estimates from cause‐specific models in both men (−0.1%) and women (0.6%); when comparing with the observed risks from cumulative incidence models, predicted risks were only slightly overpredicted in men (3.2%) and in women (1.4%).
Table 2.
Population | Sample Size, n | PCE Predicted Risks | Cause‐Specific Estimates | Cumulative Incidence Estimates |
C‐Statistic (95% CI; Adjusted for False‐Coverage Rate) |
Incidence Rate of Competing Events, per 1000 PY | ||
---|---|---|---|---|---|---|---|---|
Observed Risks, % | Difference Between Predicted and Observed Risks, % | Observed Risks, % | Difference Between Predicted and Observed Risks, % | |||||
Men | ||||||||
Overall | 1643 | 29.2 | 29.3 | −0.1 | 26.1 | 3.1 | 0.68 (0.65–0.71) | 26.8 |
Multimorbidity patterns | ||||||||
Minimal disease | 1109 | 26.1 | 24.8 | 1.3 | 22.6 | 3.5 | 0.66 (0.62–0.70) | 21.8 |
Cardiometabolic | 127 | 51.8 | 50.2 | 1.6 | 44.4 | 7.4 | 0.56 (0.45–0.65) | 34.7 |
Low cognition | 180 | 33.1 | 39.0 | −5.9 | 31.1 | 2.0 | 0.64 (0.56–0.73) | 48.2 |
Musculoskeletal‐lung depression | 227 | 28.7 | 34.1 | −5.4 | 29.2 | −0.5 | 0.70 (0.63–0.77) | 34.7 |
Frailty phenotype | ||||||||
Robust | 708 | 26.8 | 25.9 | 0.9 | 24.0 | 2.8 | 0.67 (0.63–0.72) | 17.6 |
Prefrail | 826 | 30.2 | 30.3 | −0.1 | 26.3 | 3.9 | 0.67 (0.63–0.71) | 32.6 |
Frail | 109 | 36.6 | 47.3 | −10.7 | 38.5 | −1.9 | 0.65 (0.55–0.76) | 54.2 |
Women | ||||||||
Overall | 2606 | 20.7 | 20.1 | 0.6 | 18.7 | 2.0 | 0.69 (0.67–0.72) | 16.1 |
Multimorbidity class | ||||||||
Minimal disease | 1508 | 17.7 | 16.5 | 1.2 | 15.7 | 2.0 | 0.69 (0.65–0.73) | 12.2 |
Cardiometabolic | 180 | 38.5 | 34.6 | 3.9 | 31.7 | 6.8 | 0.64 (0.55–0.72) | 19.9 |
Low cognition | 171 | 31.3 | 33.7 | −2.4 | 28.5 | 2.8 | 0.63 (0.54–0.72) | 39.2 |
Musculoskeletal‐lung depression | 747 | 19.9 | 21.4 | −1.5 | 19.4 | 0.5 | 0.67 (0.62–0.72) | 18.8 |
Frailty phenotype | ||||||||
Robust | 946 | 16.4 | 13.7 | 2.7 | 13.0 | 3.4 | 0.63 (0.58–0.69) | 11.2 |
Prefrail | 1362 | 21.7 | 21.2 | 0.5 | 19.7 | 2.0 | 0.70 (0.67–0.73) | 15.8 |
Frail | 298 | 29.7 | 37.9 | −8.2 | 32.3 | −2.6 | 0.64 (0.57–0.71) | 37.3 |
ASCVD indicates atherosclerotic cardiovascular disease; PCE, pooled cohort equation; and PY, person‐years.
In subgroups by multimorbidity patterns, the C‐statistic for discrimination in men ranged from 0.56 (cardiometabolic class; 95% CI, 0.45–0.65) to 0.70 (musculoskeletal‐lung depression class; 95% CI, 0.63–0.77); in women, the C‐statistic ranged from 0.63 (low cognition class; 95% CI, 0.54–0.72) to 0.69 (minimal disease class; 95% CI, 0.65–0.73). Using cumulative incidence observed risk estimates, the PCE overpredicted the risk of events in the cardiometabolic class for both men (7.4%) and women (6.8%). For calibration using cause‐specific observed risk estimates, the PCE underestimated risk in men with the low cognition (−5.9%) and musculoskeletal‐lung depression class (−5.4%).
In subgroups by frailty status, the C‐statistics in both sexes ranged from 0.63 to 0.70 without noticeable patterns by frailty level. Calibration by frailty status using cumulative incidence observed risk estimates did not show important variations by frailty status. However, when using cause‐specific observed risk estimates, risk was underpredicted in those with frailty in both sexes (−10.7% in men and −8.2% in women). Differences between the observed risk estimated by cause‐specific modeling versus cumulative incidence ranged from 0.7% to 8.8%, with the greatest difference in men with low cognition pattern (8.8%) and frailty (7.9%). Differences between cause‐specific versus cumulative incidence modeling were commensurate with the IR of competing risk event, which was highest in low cognition and frailty subgroups in men (48.2 and 54.2 per 1000 PY) and women (39.2 and 37.3 per 1000 PY) and lowest in the minimal disease and robust subgroups (21.8 and 17.6 events per 1000 PY) and women (17.6 and 11.2 events per 1000 PY). Figure 2 summarizes the comparison between PCE predicted ASCVD risk, cause‐specific and cumulative incidence observed risks, and the rates of competing risks.
Figures 3 and 4 present calibration plots comparing PCE predictions with cause‐specific and cumulative incidence observed risks by sex, multimorbidity patterns, and frailty subgroups. Miscalibration occurred mostly at the highest predicted risk categories and was less important when using cumulative incidence compared with cause‐specific observed risks. Similar to calibration‐in‐the‐large results, PCE underpredicted risk in men with the low cognition and musculoskeletal‐lung depression class at moderate risk of ASCVD when compared with cause‐specific observed risk estimates. In contrast, the overprediction using cumulative incidence in the cardiometabolic men and women was mostly present in the highest predicted risk categories. Using both cause‐specific and cumulative incidence observed risks, the PCE underpredicted risk in those with frailty in both sexes and overpredicted risk in women who were robust, for those at moderate risk of ASCVD.
Discussion
Although the PCEs were not designed to be used beyond in individuals aged >75 years, 31 there is still a need for accurate ASCVD risk prediction to inform the initiation of preventive therapy in older adults. 32 Our findings from a population aged ≥65 years show that calibration and discrimination were good in the overall cohort. However, when assessed by multimorbidity class and frailty status, and compared with different competing risk models (cumulative or cause‐specific incidence), calibration varied. In particular, compared with the estimates using cumulative incidence, which are more appropriate for clinical decision making and prognosis, 22 , 33 the PCE overpredicted ASCVD risk in men and women with the cardiometabolic multimorbidity pattern (7.8% and 6.8%). Compared with estimates using cause‐specific estimated risk, the PCE underpredicted risk in men and women with frailty (−10.7% and −8.2%). As expected, cause‐specific estimates of observed risk were higher compared with cumulative incidence estimates; this overestimation was greatest for the low cognition pattern and frailty subgroups.
Despite finding variations in PCE calibration in older adults by multimorbidity patterns and frailty subgroups, their direct clinical implications warrant discussion. First, the PCE performed well, and did not vary significantly by choice of competing risk estimation, in older adults likely to undergo ASCVD risk assessment: those belonging to the minimal disease class and those considered prefrail. Second, although the PCE overpredicted risk in those with the cardiometabolic class using cumulative incidence observed risks, this overprediction was mostly in individuals with high predicted risk of ASCVD, which would already have an indication for treatment. Moreover, because the cardiometabolic class included individuals with a high prevalence of diabetes mellitus (97%) and thus with an indication for ASCVD preventive therapy, 3 the low discrimination in this class does not strongly alter management. Third, in individuals with frailty, PCE underestimation was driven by men at high predicted risk; in women, there was PCE underestimation in those with frailty but overestimation of those classified as robust at lower and clinically relevant risk thresholds. This may suggest the need to look beyond basic PCE prediction and age to incorporate frailty to refine ASCVD risk estimation in women aged ≥65 years. Fourth, the PCEs were derived using cause‐specific events, 2 which considers that those who experienced a competing event remain at future risk for ASCVD and can thus inflate risk prediction in the context of high competing risks (16.6% of participants in our cohort). 22 However, comparing calibration using cause‐specific and cumulative incidence estimates indicates that inflated risk prediction predominantly arises at the high predicted ASCVD risk range, rather than at lower thresholds, where clinical decisions are made. Although competing risks do alter risk prediction in older adults, they would have to be extremely strong or prevalent to significantly alter risk prediction for those in the 20% and lower range of ASCVD predicted risk; or, in those with higher PCE predicted risk, to reduce the true predicted risk <20%. Ignoring competing risk will systematically overestimate benefit, 34 but our results suggest that in most cases, net benefit will remain positive for older adults, 35 even with multimorbidity or frailty.
Our findings may also have methodological implications for the development of ASCVD risk prediction instruments. Previous studies have suggested that the low calibration of the PCE may be attributable to using a noncontemporary and nonethnically diverse population, ascertainment bias with underreporting of ASCVD events, concurrent preventive drug therapy, and choice of statistical methods. 7 , 8 , 9 , 10 , 36 , 37 , 38 As such, proposals to ameliorate ASCVD risk prediction have included using novel markers, a more diverse population, and improved statistical methods. Our results indicate that inadequate modeling of competing risk may be a further reason for miscalibration in the large, with miscalibration at higher predicted ASCVD risk. 33 Previous work has included competing risk modeling in ASCVD prediction without finding substantial improvement in performance by using ASCVD‐related predictors to model both ASVCD risk and competing risks. 39 , 40 Exploring additional age‐related variables, such as specific comorbidities or multimorbidity patterns, frailty, or disability might improve ASCVD risk prediction by better prediction of competing risks. 22 An alternative may be to recalibrate models using different baseline hazards by age, multimorbidity, frailty, or other subgroups. 10
Strengths and Limitations
Main strengths of our study included the availability of high‐quality follow‐up data over 10 years for the ascertainment of ASCVD and competing events. In addition, because we used data from the CHS, we were able to derive the original description of the frailty phenotype 14 and use individual‐level measurement of chronic conditions to determine multimorbidity classes. Our study has a few important limitations that deserve mention. First, the original CHS data predated contemporary management of ASCVD risk factors, and our analyses may thus underestimate the extent of overestimation of PCE risk prediction. 8 , 36 Second, because the CHS was one of the derivation cohorts for the PCE, and even if we used bootstrap resampling for evaluation, our results may overestimate PCE performance (optimism). 41 However, using one of the original derivation cohorts strengthens the identification of competing risk as a core source of miscalibration, because it cannot be attributed to ascertainment bias or to preventive therapies. Third, we used 9 (8 self‐reported) chronic conditions commonly available in epidemiologic studies to identify multimorbidity classes. Although the classes identified might have differed if other chronic conditions had been included, the patterns we identified were clinically interpretable and showed differential calibration and competing risks. Fourth, self‐reported physician‐diagnosed chronic conditions are subject to informational bias. 42 As a sensitivity analysis, we conducted latent class analysis using estimated glomerular filtration rate for kidney disease (≤45 mL/min); class membership agreement with self‐report was almost perfect (unweighted κ=0.90; Table S5). Fifth, we used single multivariate imputation: although missing data for PCE variables were low, reported CIs may not fully reflect the uncertainty related to class membership.
Conclusions
Overall ASCVD risk prediction was good in a cohort of adults aged ≥65 years. Although calibration varied according to multimorbidity patterns, frailty status, and competing risks, miscalibration was mostly present at high predicted ASCVD risks ranges and thus less likely to alter clinical decision making for primary prevention therapy.
Sources of Funding
This research was supported by contracts HHSN268201200036C, HHSN268200800007C, HHSN268201800001C, N01HC55222, N01HC85079, N01HC85080, N01HC85081, N01HC85082, N01HC85083, and N01HC85086 and grants U01HL080295 and U01HL130114 from the National Heart, Lung, and Blood Institute, with additional contribution from the National Institute of Neurological Disorders and Stroke. Additional support was provided by R01AG023629 from the National Institute on Aging. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. This work was supported by the Fondation du Centre Hospitalier de lʹUniversité de Montréal to Dr Nguyen; the Fonds de Recherche Québec Santé to Dr Nguyen; and the Canadian Institutes of Health Research to Dr Nguyen.
Disclosures
Dr Odden has served as a consultant for Cricket Health, Inc. The remaining authors have no disclosures to report.
Supporting information
Acknowledgments
A full list of principal CHS (Cardiovascular Health Study) investigators and institutions can be found at CHS‐NHLBI.org.
(J Am Heart Assoc. 2020;9:e016003 DOI: 10.1161/JAHA.119.016003.)
For Sources of Funding and Disclosures, see page 11.
References
- 1. Benjamin EJ, Muntner P, Alonso A, Bittencourt MS, Callaway CW, Carson AP, Chamberlain AM, Chang AR, Cheng S, Das SR, et al. Heart disease and stroke statistics‐2019 update: a report from the American Heart Association. Circulation. 2019;139:e56–e528. [DOI] [PubMed] [Google Scholar]
- 2. Goff DC Jr, Lloyd‐Jones DM, Bennett G, Coady S, D’Agostino RB, Gibbons R, Greenland P, Lackland DT, Levy D, O’Donnell CJ, et al. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014;129:S49–S73. [DOI] [PubMed] [Google Scholar]
- 3. Arnett DK, Blumenthal RS, Albert MA, Buroker AB, Goldberger ZD, Hahn EJ, Himmelfarb CD, Khera A, Lloyd‐Jones D, McEvoy JW, et al. 2019 ACC/AHA guideline on the primary prevention of cardiovascular disease: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Circulation. 2019;140:e596–e646. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Ridker PM, Cook NR. Statins: new American guidelines for prevention of cardiovascular disease. Lancet. 2013;382:1762–1765. [DOI] [PubMed] [Google Scholar]
- 5. Weinberger Y, Han BH. Statin treatment for older adults: the impact of the 2013 ACC/AHA cholesterol guidelines. Drugs Aging. 2015;32:87–93. [DOI] [PubMed] [Google Scholar]
- 6. Gurwitz JH, Go AS, Fortmann SP. Statins for primary prevention in older adults. JAMA. 2016;316:1971. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Pylypchuk R, Wells S, Kerr A, Poppe K, Riddell T, Harwood M, Exeter D, Mehta S, Grey C, Wu BP, et al. Articles cardiovascular disease risk prediction equations in 400 000 primary care patients in New Zealand: a derivation and validation study. Lancet. 2018;391:1897–1907. [DOI] [PubMed] [Google Scholar]
- 8. Yadlowsky S, Hayward RA, Sussman JB, McClelland RL, Min Y‐I, Basu S. Clinical implications of revised pooled cohort equations for estimating atherosclerotic cardiovascular disease risk. Ann Intern Med. 2018;169:20–29. [DOI] [PubMed] [Google Scholar]
- 9. Rana JS, Tabada GH, Solomon MD, Lo JC, Jaffe MG, Sung SH, Ballantyne CM, Go AS. Accuracy of the atherosclerotic cardiovascular risk equation in a large contemporary, multiethnic population. J Am Coll Cardiol. 2016;67:2118–2130. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Pennells L, Kaptoge S, Wood A, Sweeting M, Zhao X, White I, Burgess S, Willeit P, Bolton T, Moons KGM, et al. Equalization of four cardiovascular risk algorithms after systematic recalibration: individual‐participant meta‐analysis of 86 prospective studies. Eur Heart J. 2019;40:621–631. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Nguyen QD, Wu C, Odden MC, Kim DH. Multimorbidity patterns, frailty, and survival in community‐dwelling older adults. J Gerontol A Biol Sci Med Sci. 2019;74:1265–1270. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Vetrano DL, Palmer K, Marengoni A, Marzetti E, Lattanzio F, Roller‐Wirnsberger R, Lopez Samaniego L, Rodríguez‐Mañas L, Bernabei R, Onder G. Frailty and multimorbidity: a systematic review and meta‐analysis. J Gerontol A Biol Sci Med Sci. 2019;74:659–666. [DOI] [PubMed] [Google Scholar]
- 13. Berry SD, Ngo L, Samelson EJ, Kiel DP. Competing risk of death: an important consideration in studies of older adults. J Am Geriatr Soc. 2010;58:783–787. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Fried LP, Tangen CM, Walston J, Newman AB, Hirsch C, Gottdiener J, Seeman T, Tracy R, Kop WJ, Burke G, et al. Frailty in older adults: evidence for a phenotype. J Gerontol A Biol Sci Med Sci. 2001;56:808–813. [DOI] [PubMed] [Google Scholar]
- 15. Afilalo J, Alexander KP, Mack MJ, Maurer MS, Green P, Allen LA, Popma JJ, Ferrucci L, Forman DE. Frailty assessment in the cardiovascular care of older adults. J Am Coll Cardiol. 2014;63:747–762. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Kim DH, Kim CA, Placide S, Lipsitz LA, Marcantonio ER. Preoperative frailty assessment and outcomes at 6 months or later in older adults undergoing cardiac surgical procedures. Ann Intern Med. 2016;165:650. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Peralta CA, Katz R, Newman AB, Psaty BM, Odden MC. Systolic and diastolic blood pressure, incident cardiovascular events, and death in elderly persons. Hypertension. 2014;64:472–480. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Odden MC, Wu C, Shlipak MG, Psaty BM, Katz R, Applegate WB, Harris T, Newman AB, Peralta CA. Blood pressure trajectory, gait speed, and outcomes: the Health, Aging, and Body Composition Study. J Gerontol A Biol Sci Med Sci. 2016;71:1688–1694. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Van Den Akker M, Buntinx F, Knottnerus JA. Comorbidity or multimorbidity: what’s in a name? A review of literature. Eur J Gen Pract. 1996;2:65–70. [Google Scholar]
- 20. Whitson HE, Johnson KS, Sloane R, Cigolle CT, Pieper CF, Landerman L, Hastings SN. Identifying patterns of multimorbidity in older Americans: application of latent class analysis. J Am Geriatr Soc. 2016;64:1668–1673. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Olaya B, Moneta MV, Caballero FF, Tyrovolas S, Bayes I, Ayuso‐Mateos JL, Haro JM. Latent class analysis of multimorbidity patterns and associated outcomes in Spanish older adults: a prospective cohort study. BMC Geriatr. 2017;17:186. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Austin PC, Lee DS, Fine JP. Introduction to the analysis of survival data in the presence of competing risks. Circulation. 2016;133:601–609. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Fried LP, Borhani NO, Enright P, Furberg CD, Gardin JM, Kronmal RA, Kuller LH, Manolio TA, Mittelmark MB, Newman A, et al. The Cardiovascular Health Study: design and rationale. Ann Epidemiol. 1991;1:263–276. [DOI] [PubMed] [Google Scholar]
- 24. Tell GS, Fried LP, Hermanson B, Manolio TA, Newman AB, Borhani NO. Recruitment of adults 65 years and older as participants in the Cardiovascular Health Study. Ann Epidemiol. 1993;3:358–366. [DOI] [PubMed] [Google Scholar]
- 25. Ives DG, Fitzpatrick AL, Bild DE, Psaty BM, Kuller LH, Crowley PM, Cruise RG, Theroux S. Surveillance and ascertainment of cardiovascular events: the Cardiovascular Health Study. Ann Epidemiol. 1995;5:278–285. [DOI] [PubMed] [Google Scholar]
- 26. Harrell F, Califf R, Pryor D, Lee K, Rosati R. Evaluating the yield of medical tests. J Am Med Assoc. 1982;247:2543–2546. [PubMed] [Google Scholar]
- 27. Pencina MJ, D’Agostino RB. Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Stat Med. 2004;23:2109–2123. [DOI] [PubMed] [Google Scholar]
- 28. Benjamini Y, Yekutieli D. False discovery rate‐adjusted multiple confidence intervals for selected parameters. J Am Stat Assoc. 2005;100:71–81. [Google Scholar]
- 29. Royston P, Altman DG. External validation of a Cox prognostic model: principles and methods. BMC Med Res Methodol. 2013;13:33. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Lau B, Cole SR, Gange SJ. Competing risk regression models for epidemiologic data. Am J Epidemiol. 2009;170:244–256. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Stone NJ, Robinson JG, Lichtenstein AH, Bairey Merz CN, Blum CB, Eckel RH, Goldberg AC, Gordon D, Levy D, Lloyd‐Jones DM, et al. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014;129:S1–S45. [DOI] [PubMed] [Google Scholar]
- 32. Mason NR, Sox HC, Whitlock EP. A patient‐centered approach to comparative effectiveness research focused on older adults: lessons from the Patient‐Centered Outcomes Research Institute. J Am Geriatr Soc. 2019;67:21–28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Koller MT, Raatz H, Steyerberg EW, Wolbers M. Competing risks and the clinical community: irrelevance or ignorance? Stat Med. 2012;31:1089–1097. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Wolbers M, Koller MT, Witteman JCM, Steyerberg EW. Prognostic models with competing risks: methods and application to coronary risk prediction. Epidemiology. 2009;20:555–561. [DOI] [PubMed] [Google Scholar]
- 35. Leening MJG, Cook NR, Ridker PM. Should we reconsider the role of age in treatment allocation for primary prevention of cardiovascular disease? Eur Heart J. 2017;38:1542–1547. [DOI] [PubMed] [Google Scholar]
- 36. DeFilippis AP, Young R, Carrubba CJ, McEvoy JW, Budoff MJ, Blumenthal RS, Kronmal RA, McClelland RL, Nasir K, Blaha MJ. An analysis of calibration and discrimination among multiple cardiovascular risk scores in a modern multiethnic cohort. Ann Intern Med. 2015;162:266–275. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Cook NR, Ridker PM. Calibration of the pooled cohort equations for atherosclerotic cardiovascular disease: an update. Ann Intern Med. 2016;165:786–794. [DOI] [PubMed] [Google Scholar]
- 38. Muntner P, Colantonio LD, Cushman M, Goff DC, Howard G, Howard VJ, Kissela B, Levitan EB, Lloyd‐Jones DM, Safford MM. Validation of the atherosclerotic cardiovascular disease pooled cohort risk equations. J Am Med Assoc. 2014;311:1406–1415. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Koller MT, Leening MJG, Wolbers M, Steyerberg EW, Hunink MGM, Schoop R, Hofman A, Bucher HC, Psaty BM, Lloyd‐Jones DM, et al. Development and validation of a coronary risk prediction model for older U.S. and European persons in the Cardiovascular Health Study and the Rotterdam Study. Ann Intern Med. 2012;157:389–397. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Demissei BG, Postmus D, Valente MA, van der Harst P, van Gilst WH, Van den Heuvel ER, Hillege HL. Should non‐cardiovascular mortality be considered in the SCORE model? Findings from the Prevention of Renal and Vascular End‐stage Disease (PREVEND) cohort. Eur J Epidemiol. 2015;30:47–56. [DOI] [PubMed] [Google Scholar]
- 41. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21:128–138. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42. Okura Y, Urban LH, Mahoney DW, Jacobsen SJ, Rodeheffer RJ. Agreement between self‐report questionnaires and medical record data was substantial for diabetes, hypertension, myocardial infarction and stroke but not for heart failure. J Clin Epidemiol. 2004;57:1096–1103. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.