Risk Prediction Models for Atherosclerotic Cardiovascular Disease in Patients with Chronic Kidney Disease: The CRIC Study

Significance Statement

Patients with CKD are typically considered to be at high risk for atherosclerotic cardiovascular disease, but CKD is a heterogeneous condition and there are no validated atherosclerotic cardiovascular disease risk stratification tools for this population. Our analysis of 2604 participants in the Chronic Renal Insufficiency Cohort study found that newly developed risk prediction models, using clinically available variables and novel biomarkers, improved discrimination, calibration, and reclassification of nonevents compared with the traditional American College of Cardiology/American Heart Association pooled cohort equations developed for the general population. The new equations may improve risk stratification in patients with CKD and improve shared decision making for preventive therapy to reduce atherosclerotic cardiovascular disease incidence and mortality.

Visual Abstract

Abstract

Background

Individuals with CKD may be at high risk for atherosclerotic cardiovascular disease (ASCVD). However, there are no ASCVD risk prediction models developed in CKD populations to inform clinical care and prevention.

Methods

We developed and validated 10-year ASCVD risk prediction models in patients with CKD that included participants without self-reported cardiovascular disease from the Chronic Renal Insufficiency Cohort (CRIC) study. ASCVD was defined as the first occurrence of adjudicated fatal and nonfatal stroke or myocardial infarction. Our models used clinically available variables and novel biomarkers. Model performance was evaluated based on discrimination, calibration, and net reclassification improvement.

Results

Of 2604 participants (mean age 55.8 years; 52.0% male) included in the analyses, 252 had incident ASCVD within 10 years of baseline. Compared with the American College of Cardiology/American Heart Association pooled cohort equations (area under the receiver operating characteristic curve [AUC]=0.730), a model with coefficients estimated within the CRIC sample had higher discrimination (P=0.03), achieving an AUC of 0.736 (95% confidence interval [CI], 0.649 to 0.826). The CRIC model developed using clinically available variables had an AUC of 0.760 (95% CI, 0.678 to 0.851). The CRIC biomarker-enriched model had an AUC of 0.771 (95% CI, 0.674 to 0.853), which was significantly higher than the clinical model (P=0.001). Both the clinical and biomarker-enriched models were well-calibrated and improved reclassification of nonevents compared with the pooled cohort equations (6.6%; 95% CI, 3.7% to 9.6% and 10.0%; 95% CI, 6.8% to 13.3%, respectively).

Conclusions

The 10-year ASCVD risk prediction models developed in patients with CKD, including novel kidney and cardiac biomarkers, performed better than equations developed for the general population using only traditional risk factors.

CKD affects more than 10% of the population globally and is strongly associated with cardiovascular disease (CVD) morbidity and mortality.^1–3 Therefore, individuals with CKD are typically assumed to be at high risk for atherosclerotic CVD (ASCVD).² However, the accurate classification of ASCVD risk among individuals with CKD is still controversial. The Kidney Disease: Improving Global Outcomes (KDIGO) 2013 lipid guideline considers CKD patients aged 50 years or older as high risk regardless of CKD severity or the status of traditional CVD risk factors.⁴ The European Society of Cardiology guideline categorizes eGFR 30–59 ml/min per 1.73 m² as high risk, and eGFR <30 ml/min per 1.73 m² or patients with diabetes and albuminuria as very high risk.⁵ The 2013 American Heart Association (AHA) and American College of Cardiology (ACC) guideline did not include CKD measures in ASCVD prediction using the pooled cohort equations (PCEs).⁶

Recent primary prevention guidelines classify CKD as a risk-enhancing factor.⁷ Although the KDIGO guideline recommends predicting ASCVD risk in CKD patients younger than 50 years old, there are few established prediction tools developed for this clinical population.⁸ Overall, inconsistencies in major clinical guidelines are mainly due to inconsistent previous findings.^9–12 The Framingham equations and the ACC/AHA PCEs may inadequately predict ASCVD risk in individuals with CKD.^10,13 Furthermore, it is unclear whether incorporating additional novel risk factors will improve risk prediction for ASCVD.^14,15

The availability of accurate ASCVD risk prediction equations for individuals with CKD could be informative for patients and clinicians and improve shared decision making for preventive therapy to reduce ASCVD incidence and mortality.¹⁶ Therefore, we developed novel ASCVD risk prediction models for use among patients with CKD using data from the Chronic Renal Insufficiency Cohort (CRIC) study.

Methods

Study Design and Participants

The CRIC study is a prospective, longitudinal cohort study of participants with mild-to-moderate CKD based on an eGFR entry criterion of 20–70 ml/min per 1.73 m². A racially and ethnically diverse group of 3939 men and women aged 21–74 years was enrolled from seven clinical centers in the United States (Ann Arbor, MI; Baltimore, MD; Chicago, IL; Cleveland, OH; New Orleans, LA; Philadelphia, PA; and San Francisco, CA) in phase I, between May 2003 and August 2008.¹⁷ Patients with New York Heart Association class 3 and 4 heart failure, cirrhosis, HIV infection, polycystic kidney disease, renal cell carcinoma, those receiving chronic dialysis or an organ transplant, and those taking immunosuppressive medications were excluded from the CRIC study. For the current analysis, we excluded those with a self-reported history of CVD at baseline (n=1335), yielding 2604 participants. The study was approved by the institutional review board at each institution and all participants provided written informed consent.

Candidate Predictor Variables

A more detailed description of the methods is available in the Supplemental Methods. Candidate predictor variables considered for inclusion in the prediction models are displayed in Supplemental Figure 1. Candidate variables included those in the ACC/AHA PCEs⁶: age, sex, race and ethnicity, total cholesterol, HDL cholesterol, systolic BP, use of BP-lowering medications, history of diabetes, and current smoking. Additional candidate variables were chosen based on associations with ASCVD among patients with CKD identified in prior reports, and included metabolic factors, kidney disease, lipid metabolism, mineral metabolism, and inflammation factors, and cardiac biomarkers. Because self-reported race is a controversial and ill-defined social construct,¹⁸ we used the CKD Epidemiology Collaboration cystatin C equation as a measure of eGFR, which does not include a race adjustment factor.¹⁹ We considered two sets of variables for development of the prediction models: one set including only readily clinically available variables (Supplemental Figure 1; black text) and another set containing all candidate predictor variables, including biomarkers that are less likely to be measured in routine clinical practice (Supplemental Figure 1; blue text). In a secondary analysis, we evaluated the performance of the “CKD patch,” which incorporates eGFR and urinary albumin-creatinine ratio (ACR) measurements into the ACC/AHA PCEs. In brief, the CKD patch equations were applied to the PCEs recalibrated to the CRIC sample, with expected eGFR and urinary ACR centered at CRIC study-specific averages, as recommended.⁸

Natural logarithm and nonlinear (e.g., polynomial, square root) transformations of continuous variables were evaluated and retained if they improved model fit. Potential interactions between all candidate predictors were assessed. However, the inclusion of interaction terms was consistently associated with poorer cross-validated model performance and they were not retained in the final models. We could not evaluate some potentially important variables (e.g., coronary artery calcium, pulse-wave velocity, and electrocardiogram variables) because these variables were not available among the majority of CRIC participants. Missing data for other candidate predictors was <5% for all variables except homocysteine (11.3%) and neutrophil gelatinase-associated lipocalin ( 5.7%). Missing values were imputed using a nearest neighbors model.²⁰

Outcome Assessment

The primary end point for the prediction models was incident ASCVD: a composite of incident fatal or nonfatal stroke or myocardial infarction.⁶ Events are ascertained every 6 months from medical records, and diagnosis was confirmed by two blinded reviewers as possible, probable, or definite events.²¹ Follow-up time was censored at the earliest occurrence of death, loss to follow-up (n=203; 7.8%), or 10 years of complete follow-up, consistent with prior models developed for ASCVD.⁶ All deaths were confirmed by death certificate.

Statistical Analyses

Model Development

A priori, we evaluated the ACC/AHA PCEs in the CRIC sample using two approaches: (1) predicted risk from the published model coefficients⁶; and (2) variables included in the ACC/AHA PCEs with coefficients recalculated using the CRIC sample. Furthermore, we evaluated several modeling algorithms and selection procedures to develop novel models. Our primary approach used Cox proportional hazards regression models to predict the probability of having an incident ASCVD event within 10 years of baseline.⁶ Additionally, we used the Fine and Gray subdistribution hazards model to account for the competing risk of death.²² Within both modeling algorithms, we evaluated three predictor selection procedures: (1) backward elimination with a P<0.05 criterion; (2) backward elimination with an Akaike information criterion; and (3) LASSO (“least absolute shrinkage and selection operator”) method with a regularization penalty (λ) one SEM higher than that which minimized cross-validated error.²³

Model Validation

All modeling strategies were internally validated using 10 × 10-fold cross-validation, similar to the derivation procedure used to develop the ACC/AHA PCEs.⁶ The best-performing (see “Performance Metrics” section) modeling procedure was used on the full CRIC sample to develop final clinical and biomarker-enriched models. In a sensitivity analysis, we evaluated the best-performing modeling procedure from the 10 × 10-fold cross-validation using an internal-external validation approach. In this approach, one clinical center is left out at a time to cross-validate models developed in the other centers, which provides additional information on the potential performance of the models in external datasets similar in composition to the CRIC study sample.²⁴

We validated our models externally using data from two independent, community-based prospective cohort studies: the Atherosclerosis Risk in Communities (ARIC) study²⁵ and the Multi-Ethnic Study of Atherosclerosis (MESA).²⁶ We pooled individual-level data among 2222 participants with CKD, based on an eGFR <60 ml/min per 1.73 m² or urinary ACR ≥30 mg/g. The equations developed in the CRIC study were applied directly to the external validation data, and model performance was assessed.

Performance Metrics

We evaluated the performance of developed models using measures of discrimination, calibration, and overall goodness of fit.²⁷ Discrimination refers to the ability of a model to correctly identify those who will or will not experience the outcome and was assessed using the time-dependent (i.e., 10-year) area under the receiver operating characteristic curve (AUC).²⁸ Calibration refers to the agreement between observed outcomes and model predictions, and was assessed by plotting the observed versus predicted risk across deciles of predicted risk. Observed risk was estimated using the cumulative incidence function accounting for the competing risk of death. The scaled Brier score, or index of prediction accuracy (IPA), incorporates information on both discrimination and calibration, and was used to evaluate overall goodness of fit.²⁹ A higher IPA indicates a better performing model, an IPA of 1.0 indicates a perfect model, and an IPA ≤0 indicates a useless model. Performance results from the 10 × 10-fold cross-validation were summarized as the mean for point estimates and the 2.5th and 97.5th percentiles for 95% confidence intervals (CIs). We used the categorical net reclassification improvement (NRI) statistic to evaluate the clinical utility of the models,^30,31 using predicted risk thresholds used in current clinical practice guidelines (i.e., 7.5% and 20% per the ACC/AHA primary prevention guideline).⁷ Bootstrapping with 1000 replicates was used to generate 95% CIs for the NRI.

Analyses were conducted using SAS version 9.4 (SAS Institute, Cary, NC) and R version 4.1.1 (R Project for Statistical Computing). Two-sided P values <0.05 were considered statistically significant for all analyses.

Results

Baseline Characteristics

Among 2604 participants included in the analyses, the mean (SD) age was 55.8 (11.7) years, 52.0% were male, 41.6% had diabetes, and the mean (SD) eGFR was 56.0 (24.7) ml/min per 1.73 m². During the first 10 years of follow-up from baseline, 252 incident ASCVD events occurred (11.9 events per 1000 person-years). Participants who had an ASCVD event within 10 years were more likely to be older, Black race, currently smoke, have a history of diabetes, and use BP-lowering medications compared with those who did not have an ASCVD event (Table 1). On average, those who had an ASCVD event had higher levels of glycated hemoglobin (HbA1c), systolic and diastolic BP, urinary ACR, troponin-T, and N-terminal prohormone of brain natriuretic peptide (NT-proBNP); and lower levels of HDL cholesterol, eGFR, and hemoglobin compared with those who did not have an event. Within 10 years, 411 participants died from noncardiovascular causes.

Table 1.

Baseline characteristics of CRIC study participants

Variables	Overall (n=2604)	ASCVD Event within 10 Years
		No (n=2352)	Yes (n=252)
Age, mean (SD), yr	55.8 (11.7)	55.5 (11.8)	59.0 (9.8)
Male, No. (%)	1354 (52.0)	1212 (51.5)	142 (56.3)
Black race, No. (%)	1013 (38.9)	898 (38.2)	115 (45.6)
Body mass index, mean (SD), kg/m2	31.7 (7.8)	31.7 (7.9)	32.0 (7.1)
Current smoking, No. (%)	314 (12.1)	262 (11.1)	52 (20.6)
HbA1c, mean (SD), %	6.4 (1.5)	6.4 (1.5)	7.0 (1.6)
Diabetes mellitus, No. (%)	1082 (41.6)	931 (39.6)	151 (59.9)
Use of glucose-lowering medications, No. (%)	620 (24.0)	534 (22.9)	86 (34.3)
Systolic BP, mean (SD), mm Hg	127.0 (21.0)	126.0 (20.7)	136.1 (21.6)
Diastolic BP, mean (SD), mm Hg	72.7 (12.4)	72.5 (12.2)	74.3 (13.7)
Use of BP-lowering medications, No. (%)	2282 (87.6)	2041 (86.8)	241 (95.6)
Total cholesterol, mean (SD), mg/dl	188.4 (44.5)	188.2 (43.7)	190.2 (51.3)
LDL cholesterol, mean (SD), mg/dl	106.6 (35.3)	106.5 (34.9)	107.6 (38.8)
HDL cholesterol, mean (SD), mg/dl	48.8 (16.1)	49.1 (16.2)	45.8 (15.0)
Apolipoprotein B, mean (SD), mg/dl	85.6 (24.4)	85.4 (24.1)	87.7 (27.1)
Use of statin medications, No. (%)	1184 (45.8)	1045 (44.8)	139 (55.4)
eGFR, mean (SD), ml/min per 1.73 m2	56.0 (24.7)	56.9 (25.0)	47.2 (20.4)
Urine ACR, median [IQR], mg/g	39.1 [7.3, 391.2]	34.7 [6.7, 344.7]	224.0 [18.3, 1062.3]
Hemoglobin, mean (SD), g/dl	12.7 (1.8)	12.8 (1.8)	12.3 (1.9)
hsCRP, median [IQR], mg/L	2.4 [1.0, 6.1]	2.3 [1.0, 6.0]	3.0 [1.2, 7.9]
Troponin-T, median [IQR], pg/ml	9.7 [4.5, 18.9]	9.3 [4.1, 17.4]	16.0 [8.9, 31.6]
NT-proBNP, median [IQR], pg/ml	106.5 [49.3, 249.7]	99.3 [46.5, 229.5]	207.9 [95.7, 457.5]

IQR, interquartile range.

Model Development and Internal Validation

The results from the 10 × 10-fold internal cross-validation are presented in Supplemental Table 1. Across all candidate variables and selection procedures, the Cox proportional hazards regression models obtained higher estimates of AUC and IPA than the Fine and Gray subdistribution hazards models, whereas the Fine and Gray model improved calibration in the highest predicted probability decile (Supplemental Figure 2). According to the 10 × 10-fold cross-validation, Cox proportional hazards regression with backward elimination using a criterion of P<0.05 had the highest AUC and IPA. In a sensitivity analysis using internal-external validation leaving one CRIC clinical center out at a time, prediction performance was similar to 10 × 10-fold cross-validation, although AUCs were slightly higher and IPAs were slightly lower (Supplemental Table 2).

Model Characteristics and Risk Calculations

Table 2 provides the hazard ratios (HRs) with 95% CIs, β coefficients, and information for calculating predicted 10-year risk of ASCVD for the final models. Model 1 included the ACC/AHA PCEs variables with coefficients recalculated in the CRIC study sample. Model 2 (CRIC clinical model) included age, HDL cholesterol, systolic BP, current smoking, urinary ACR, HbA1c, and hemoglobin. Model 3 (CRIC enriched model) included age, total cholesterol, HDL cholesterol, current smoking, urinary ACR, HbA1c, apolipoprotein B, high-sensitivity C-reactive protein (hsCRP), troponin-T, and NT-proBNP. Example predicted risk calculations using each of the three models are available in Supplemental Tables 3–5.

Table 2.

Novel CKD ASCVD risk prediction models developed in the CRIC study

Model Parameters	Model 1: ACC/AHA PCEs Variables		Model 2: CRIC Clinical Model		Model 3: CRIC Enriched Model
Variables	HR (95% CI)	β, per 1-unit	HR (95% CI)	β, per 1-unit	HR (95% CI)	β, per 1-unit
Age, per 5 yr	1.15 (1.09 to 1.21)	0.0275	1.23 (1.16 to 1.31)	0.0420	1.19 (1.12 to 1.26)	0.0347
Male sex	1.09 (0.87 to 1.38)	0.0900
Black race	1.21 (0.96 to 1.51)	0.1885
Total cholesterol, per 10 mg/dl	1.02 (1.00 to 1.04)	0.0020			1.08 (1.01 to 1.15)	0.0073
HDL cholesterol, per 10 mg/dl	0.87 (0.80 to 0.95)	−0.0134	0.88 (0.81 to 0.95)	−0.0127	0.85 (0.77 to 0.94)	−0.0164
Systolic BP, per 10 mm Hg	1.13 (1.08 to 1.19)	0.0126	1.08 (1.02 to 1.14)	0.0077
Use of BP-lowering medications	1.54 (0.97 to 2.46)	0.4342
History of diabetes	1.88 (1.49 to 2.37)	0.6300
Current smoking	2.09 (1.57 to 2.78)	0.7382	2.03 (1.53 to 2.69)	0.7086	1.94 (1.46 to 2.57)	0.6615
Log (urine ACR), mg/g			1.17 (1.11 to 1.23)	0.1553	1.12 (1.05 to 1.19)	0.1127
HbA1c, per 0.5%			1.09 (1.05 to 1.12)	0.1634	1.08 (1.05 to 1.12)	0.1616
Hemoglobin, per 5 g/dl			0.64 (0.46 to 0.88)	−0.0903
Apolipoprotein B, per 10 mg/dl					0.89 (0.79 to 0.99)	−0.0121
Log (hsCRP), mg/L					1.11 (1.02 to 1.21)	0.1045
Log (troponin-T), pg/ml					1.23 (1.09 to 1.39)	0.2090
Log (NT-proBNP), pg/ml					1.22 (1.11 to 1.35)	0.2011
Score calculations
Baseline survival at 10 years, S(10)	0.9045		0.9075		0.9100
Predicted probability of ASCVD at 10 yearsa	1– S(10)exp(ΣbX–3.6963)		1– S(10)exp(ΣbX–3.3283)		1– S(10)exp(ΣbX–4.5580)

^aThe predicted 10-year probability for each model can be calculated as 1– S(10)^{exp(ΣbX–betaavg)} where b is the regression coefficient (β), X is the individual patient’s level for each risk factor, and betaavg is the sample mean of the β×value sum.

Relative Variable Importance

Figure 1 shows the relative importance of variables included in models 2 and 3. In both models, age accounted for the greatest proportion of the overall model chi-squared. The top five predictors in model 2 were age, urinary ACR, current smoking, HbA1c, and HDL cholesterol. The top five predictors in model 3 were age, HbA1c, current smoking, NT-proBNP, and urinary ACR.

Figure 1.

Relative importance of variables included in CKD ASCVD risk prediction models. Proportion of overall chi-squared for (A) CRIC clinical model and (B) CRIC enriched model. Higher values for the proportion of overall chi-squared indicate more important variables relative to other variables in the model.

Discrimination and Calibration

All models were well-calibrated, but each overestimated risk within the highest decile of predicted risk (Figure 2 and Supplemental Figure 3). The models developed in the CRIC data were better calibrated across the range of predicted risks compared with the published ACC/AHA PCEs. Applying the published ACC/AHA PCEs coefficients to the CRIC sample yielded an AUC of 0.730 and IPA of 0.058. Model 1 including the ACC/AHA PCEs variables recalculated in the CRIC sample achieved an AUC of 0.736 (95% CI, 0.649 to 0.826) and IPA of 0.072 (95% CI, −0.016 to 0.137), which reflect improved performance compared with the ACC/AHA PCEs using published coefficients (Table 3). Model 2 (AUC, 0.760; 95% CI, 0.678 to 0.851; and IPA, 0.095; 95% CI, 0.013 to 0.173) and model 3 (AUC, 0.771; 95% CI, 0.674 to 0.853; and IPA, 0.105; 95% CI, 0.033 to 0.168) both further improved prediction performance. Model 3 performed best, and was significantly better than model 2 in terms of AUC (P=0.001) and IPA (P=0.007). Applying the CKD patch to the CRIC study sample in a secondary analysis yielded an AUC of 0.752 and IPA of 0.096 (Supplemental Figure 4). Prediction performance was similar in participants with CKD stage 2–3 versus stage 4 (Supplemental Figure 5).

Figure 2.

Discrimination and calibration of risk prediction models for ASCVD in CKD. Results were obtained by aggregating predicted probabilities from 10 × 10-fold cross-validation and then assessing their discrimination (AUC), calibration (predicted probability versus observed events, with each circle representing a decile of predicted probability), and overall goodness of fit (IPA). Observed probability was estimated using the cumulative incidence function accounting for the competing risk of death. Higher values for AUC (95% CI) and IPA (95% CI) indicate better performing models.

Table 3.

Performance characteristics of novel CKD ASCVD risk prediction models developed in the CRIC study

Performance Characteristics	Model 1: ACC/AHA PCEs Variables	Model 2: CRIC Clinical Model	Model 3: CRIC Enriched Model
Discrimination
AUC (95% CI)a	0.736 (0.649 to 0.826)	0.760 (0.678 to 0.851)	0.771 (0.674 to 0.853)
P value comparing with  ACC/AHA PCEsb	0.03	<0.001	<0.001
P value comparing  successive modelsc	–	0.01	0.001
Overall goodness of fit
IPA (95% CI)a	0.072 (-0.016 to 0.137)	0.095 (0.013 to 0.173)	0.105 (0.033 to 0.168)
P value comparing with  ACC/AHA PCEsb	0.01	<0.001	<0.001
P value comparing  successive modelsc	–	0.004	0.007

^aHigher values for AUC and IPA indicate better performing models.

^bThe AUC and IPA for the published ACC/AHA PCEs in the CRIC study are 0.730 and 0.058, respectively.

^cComparison of model 2 with model 1, and model 3 with model 2.

Among the 2222 participants from the external validation cohorts, 337 ASCVD events occurred during the first 10 years of follow-up, and the observed 10-year risk of ASCVD was higher compared with the CRIC study (15.5% versus 10.5%) (Supplemental Table 6). In the external validation data, the ACC/AHA PCEs had an AUC of 0.702 and IPA of 0.053 (Supplemental Figure 6). When the CRIC equations were applied to the external validation data, discrimination and calibration were adequate and improved upon the ACC/AHA PCEs, achieving AUCs of 0.709 and 0.747, and IPAs of 0.087 and 0.117, for the clinical and enriched models, respectively (Supplemental Figure 6).

NRI

Model 1 did not improve reclassification of events nor nonevents compared with the published ACC/AHA PCEs (Table 4). Model 2 significantly improved reclassification of nonevents compared with the published ACC/AHA PCEs (6.6%; 95% CI, 3.7% to 9.6%). Model 3 significantly improved reclassification of nonevents compared with both the published ACC/AHA PCEs (10.0%; 95% CI, 6.8% to 13.3%) and model 2 (3.7%; 95% CI, 1.4% to 6.0%), and was the best-performing model in terms of NRI. The numbers of participants reclassified according to each model are presented in Supplemental Tables 7 and 8.

Table 4.

Reclassification of events and nonevents in the CRIC study

Model Comparisonsa	Proportion of Participants Correctly Reclassifiedb		Net Reclassification Index (95% CI)
	Events Reclassified, % (95% CI)	Nonevents Reclassified, % (95% CI)
Comparing with ACC/AHA PCEs
Model 1 versus ACC/AHA PCEs	−10.0 (−2.5 to −16.5)	1.5 (−0.9 to 3.9)	−8.5 (−1.0 to −15.3)
Model 2 versus ACC/AHA PCEs	−4.8 (−12.2 to 2.8)	6.6 (3.7 to 9.6)	1.8 (−6.2 to 10.0)
Model 3 versus ACC/AHA PCEs	−1.4 (−8.9 to 6.4)	10.0 (6.8 to 13.3)	8.6 (0.2 to 16.6)
Comparing with model 1
Model 2 versus 1	6.7 (−0.3 to 13.5)	5.2 (2.8 to 8.0)	11.9 (4.6 to 18.9)
Model 3 versus 1	11.7 (3.6 to 19.4)	8.7 (5.9 to 11.7)	20.4 (12.3 to 28.6)
Comparing with model 2
Model 3 versus 2	5.8 (−0.1 to 11.2)	3.7 (1.4 to 6.0)	9.5 (3.3 to 15.3)

^aModel 1 indicates the ACC/AHA PCEs variables recalculated in the CRIC study; model 2 indicates the CRIC clinical model; and model 3 indicates the CRIC enriched model.

^bModels are compared for reclassification of events to a higher predicted risk category and nonevents to a lower predicted risk category, using cut points of 7.5% and 20%. More detailed information can be found in Supplemental Tables 7 and 8.

Discussion

Using a sample of 2604 participants with CKD stages 2–4 and free of clinical CVD, we developed and validated a set of predictive models for 10-year risk of ASCVD. We found that the published ACC/AHA PCEs had moderate predictive ability for 10-year risk of ASCVD among participants of the CRIC study. However, novel models developed from a pool of >50 candidate predictors afforded significant prediction improvements. Both a model developed using clinically available variables and a biomarker-enriched model significantly improved discrimination and calibration beyond the ACC/AHA PCEs, and significantly improved reclassification of nonevents. These findings support the use of CKD-specific tools to predict the long-term risk of ASCVD among patients with CKD, who may have different risk profiles compared with the general population, for whom prior models were developed.

Patients with CKD are typically considered to be at high risk for ASCVD, irrespective of other individual patient characteristics or laboratory values.^4,5 As such, current clinical practice guidelines do not have specific recommendations among patients with CKD other than to consider kidney function markers, including eGFR and albuminuria, as “risk-enhancing factors.”⁷ Our findings confirm the value of CKD markers for prediction, with urinary ACR in particular being a strong predictor in both our clinical and enriched models. Notably, eGFR was not included in our final models, suggesting that urinary ACR has a relatively stronger prognostic value for development of ASCVD among patients with CKD compared with eGFR.

We found that several other variables were particularly important for prediction of ASCVD among patients with CKD. In both our clinical and enriched models, age was the most important predictor of ASCVD at 10 years, which is an important consideration in an aging US population.³² Beyond age, we note the inclusion in our models of several important predictors including current smoking, HbA1c, hsCRP, and cardiac biomarkers. Diabetes is highly associated with both CKD and CVD.³³ Although diabetes status was included in the ACC/AHA PCEs,⁶ absolute values of long-term glycemia were not incorporated. Our models suggest the information afforded by the values of a glycemic measure, specifically HbA1c, may be superior for ASCVD risk prediction in CKD populations.

The cardiac biomarkers NT-proBNP and troponin-T showed particularly high relative variable importance in our biomarker-enriched model. These biomarkers are increasingly recognized as being strongly associated with risk for ASCVD among patients with CKD, and were more predictive of CVD than other nontraditional kidney biomarkers among 8622 participants from the ARIC study.¹⁴ Both NT-proBNP and troponin-T are circulating proteins that become elevated in response to cardiac stretch and injury.^34,35 Although typically associated with heart failure among patients with CKD,³⁶ these biomarkers may also provide information on a greater burden of subclinical cardiac damage that contributes directly to the risk of ASCVD. It should be acknowledged that elevated levels of NT-proBNP or troponin-T may reflect, at least in part, impaired renal clearance among CKD patients, but our findings and the findings of others suggest these biomarkers are valuable for ASCVD risk prediction in this population.^14,37 Collection of nontraditional, although widely available, laboratory tests could improve clinical evaluation and management of ASCVD among patients with CKD.

There are several clinical and research implications of our findings. CKD is a heterogeneous condition³⁸ and the ischemic heart disease encountered in CKD often involves distal, small arteries that are not amenable to coronary revascularization.³⁹ Frequently CKD patients undergo invasive diagnostic procedures because they are considered to have high CVD risk. Using a prediction tool better equipped to discern ASCVD events could help reduce the number of unnecessary cardiac angiography procedures and associated morbidity. Other recent advances in this area include the development of a CKD patch to incorporate eGFR and ACR measurements into established risk prediction equations.⁴⁰ An analysis conducted among approximately 9 million participants included in the CKD Prognosis Consortium identified the utility of the CKD patch in improving performance beyond the ACC/AHA PCEs and the Systematic COronary Risk Evaluation (SCORE) model.⁸ In the current analysis, the CKD patch improved discrimination of ASCVD events compared with the ACC/AHA PCEs. However, the CKD patch did not consider potential improvements afforded by additional biomarkers, nor models specifically developed among patients with CKD. In the current CRIC study sample, our new models provided more accurate predictions compared with previous risk prediction models, particularly in those with 10-year ASCVD risk <20%. Therefore, our models significantly improved reclassification of nonevents compared with the standard ACC/AHA PCEs. Given the heterogeneity of risk among patients with CKD, it may be warranted to employ prediction models specifically developed for this population. For those with predicted risk of 20% or more, it may be prudent to conduct further evaluation, such as coronary artery calcium scoring.⁴¹ In addition, our models could be used in future clinical trials to more accurately recruit high risk patients, allowing for smaller sample sizes and more efficient conduct of trials.

There are several strengths in the current analysis. First, we considered >50 candidate predictor variables, measured using standardized methodology in the CRIC study, which improves accuracy and minimizes bias. Second, ASCVD events are rigorously adjudicated. Third, we have a relatively large population of patients with CKD stages 2–4 which was enrolled based on CKD status, which is an improvement over prior analyses that selected CKD patients from general population cohorts based on one assessment of kidney function. However, there are several limitations. First, although we externally validated our models in two population-based cohort studies, the individuals in these datasets were selected based on only one assessment of kidney function and had a higher baseline risk of ASCVD, on average. Our results suggest adequate discrimination and calibration of our models, but future analyses should externally validate and refine our models among large and diverse populations with CKD and varying baseline risks for ASCVD. Second, these models could not incorporate potentially important imaging variables including coronary artery calcium scores and electrocardiogram data. Finally, our models overestimated risk in the highest predicted probability groups in the CRIC study. However, this finding is common to most CVD prediction models, and the overpredictions occur at probabilities well above clinical decision-making cutoff points.⁷

In conclusion, we have developed new models for 10-year risk prediction of ASCVD among patients with CKD, including a model composed of readily clinically available variables and a biomarker-enriched model. These models significantly improve prediction performance beyond the ACC/AHA PCEs in this population of patients with CKD stages 2–4, particularly for those with 10-year ASCVD risk <20%. Future analyses should externally validate and refine these models in multiple diverse CKD cohorts.

Disclosures

N.B. Allen reports research funding with AHA and National Institutes of Health (NIH)/National Heart, Lung, and Blood Institute (NHLBI). N. Bansal reports associate editor for Kidney360. J.B. Cohen reports honoraria from UpToDate; and research funding with AHA and NIH/NHLBI and NIH/National Center for Advancing Translational Sciences. R. Deo reports consultancy agreements with Biotronik, Boehringer Ingelheim, Janssen Pharmaceuticals, and Pfizer; research funding with iRhythm Technologies; editorial boards for Circulation and Heart Rhythm O2; and steering committee for Kidney Disease: Improving Global Outcomes. M.A. Dobre reports honoraria from Relypsa Inc. and Tricida; and scientific advisor or membership with Tricida (Metabolic Acidosis Working Group) and Relypsa (Resistant Hypertension Working Group). H.I. Feldman reports consultancy with InMed, Inc., Kyowa Hakko Kirin, National Kidney Foundation, and University of California, Los Angeles; steering committee for the CRIC study; member of advisory board of the National Kidney Foundation; editor-in-chief of American Journal of Kidney Disease. J.P. Lash reports scientific advisor or membership with Kidney360. K. Matsushita reports consultancy agreements with Akebia, Kyowa Hakko Kirin, and Mitsubishi Tanabe; research funding from Kyowa Hakko Kirin; honoraria from Fukuda Denshi; and scientific advisor or membership with American Journal of Kidney Disease, Circulation Reports, and Kidney International. M. Rahman reports consultancy with Barologics; research funding from Bayer Pharmaceuticals and Duke Clinical Research Institute; honoraria from Reata, Relypsa, and Bayer; associate editor for CJASN; and editorial board member for American Journal of Nephrology. S. Seliger reports consultancy agreements with Tricida, Inc (Endpoint Adjudication Committee); research funding from Kadmon Pharmaceuticals, Palladio Biosciences, Reata Pharmaceuticals, Roche Diagnostics, and Sanofi US; patents and inventions with University of Maryland, Baltimore and University of Texas, Southwestern: Methods for assessing differential risk for developing heart failure; associate editor for CJASN; member of medical review board for ESRD Network 5; chair of Board of Directors for ESRD Network 5; member of Endpoint Adjudication Committee and of VALOR CKD trial (Tricida, Inc.); and member of the editorial board for Circulation. T. Shafi reports consultancy agreements with Siemens; research funding as clinical trial site investigator with Baxter, CVS, and Natera; honoraria from Cara Therapeutics, NIH, Siemens, State University of New York Downstate, and University of Virginia; and scientific advisor or membership with American Journal of Medicine, American Journal of Kidney Diseases, CJASN, and Kidney360. M. Wolf reports consultancy agreements with Akebia, Amgen, Ardelyx, AstraZeneca, Bayer, Pharmacosmos, Unicycive, and Walden; ownership interest with Akebia, Unicycive, and Walden; honoraria from Akebia, Amgen, Ardelyx, AstraZeneca, Bayer, Pharmacosmos, Unicycive, and Walden; and scientific advisor or membership with Akebia, Unicycive, and Walden. W. Yang reports statistical editor for American Journal of Kidney Disease. J. Flack reports consultancy agreements with Teva Pharmaceuticals, Amgen, Sanofi, Fibrogen, and Janssen; ownership interest with Novocure, Google, Netflix, Albermarle, Alexandria REIT, Amazon, Ambarella, Arch Cap Group, Chargepoint, Enphase, Intercontinental Exchange, Johnson Controls, Estee Lauder, Lucid, Marvell Technology, Mastercard, Microsoft, Nova, Nvidia, Palantir, Paypal, Plains All American Pipeline, Quantamscape, STMicroelectronics, Reconnaissance Energy, Lightbridge, Advanced MicroDevices, Apollo Commercial Real Estate, Ballard Power, Emerson Electric, Essential Utils, IDT Corp, Meta Platforms, and Neogenomics; research funding from Glaxo Smith Kline, Idorsia, Quantam Genomics, ReCor Medical, Vascular Dynamics, and Rox Medical; honoraria from Amgen, Sanofi, and Teva; and advisory or leadership roles with American College of Physicians Board of Regents, American Journal of Hypertension Associate Editor, and Sanofi. All remaining authors have nothing to disclose.

Funding

Funding for the CRIC study was obtained under a cooperative agreement from the National Institute of Diabetes and Digestive and Kidney Diseases (U01DK060990, U01DK060984, U01DK061022, U01DK061021, U01DK061028, U01DK060980, U01DK060963, U01DK060902, and U24DK060990). In addition, this work was supported in part by the Perelman School of Medicine at the University of Pennsylvania Clinical and Translational Science Award, National Center for Advancing Translational Sciences (UL1TR000003), Johns Hopkins University (UL1TR000424), University of Maryland General Clinical Research Center (M01 RR-16500), Clinical and Translational Science Collaborative of Cleveland, the National Center for Advancing Translational Sciences component of the NIH roadmap for Medical Research (UL1TR000439), Michigan Institute for Clinical and Health Research (UL1TR000433), University of Illinois at Chicago Clinical and Translational Science Award (UL1RR029879), Tulane COBRE for Clinical and Translational Research in Cardiometabolic Diseases (P20 GM109036), Kaiser Permanente, National Center for Research Resources (UCSF-CTSI UL1 RR-024131), and Department of Internal Medicine, University of New Mexico School of Medicine Albuquerque, New Mexico (R01DK119199). J.D. Bundy was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development career development grant (K12HD043451). The ARIC study is supported by the National Heart, Lung, and Blood Institute (HHSN268201700001I, HHSN268201700002I, HHSN268201700003I, HHSN268201700004I, and HHSN268201700005I). MESA is supported by the National Heart, Lung, and Blood Institute (75N92020D00001, HHSN268201500003I, N01-HC-95159, 75N92020D00005, N01-HC-95160, 75N92020D00002, N01-HC-95161, 75N92020D00003, N01-HC-95162, 75N92020D00006, N01-HC-95163, 75N92020D00004, N01-HC-95164, 75N92020D00007, N01-HC-95165, N01-HC-95166, N01-HC-95167, N01-HC-95168, and N01-HC-95169), and by the National Center for Advancing Translational Sciences (UL1TR000040, UL1TR001079, and UL1TR001420).

Author Contributions

J.D. Bundy, J. He, K. Matsushita, and M. Rahman conceptualized the study; J. Chen, H.I. Feldman, J. He, J.P. Lash, M. Rahman were responsible for funding acquisition; H.I. Feldman, J. He, J.P. Lash, and M. Rahman were responsible for data curation; J.D. Bundy and B.C. Jaeger were responsible for formal analysis and software; J.D. Bundy, J. He, and B.C. Jaeger were responsible for methodology; J.D. Bundy was responsible for visualization; N. Allen, N. Bansal, and K. Matsushita were responsible for validation; J. He provided supervision; J.D. Bundy wrote the original draft; and all authors reviewed and edited the manuscript.

Supplemental Material

This article contains the following supplemental material online at http://jasn.asnjournals.org/lookup/suppl/doi:10.1681/ASN.2021060747/-/DCSupplemental.

Supplemental Methods.

Supplemental Figure 1. Candidate predictor variables considered for inclusion in prediction models.

Supplemental Figure 2. Calibration slope plots of Cox proportional hazards and Fine and Gray subdistribution hazards modeling algorithms.

Supplemental Figure 3. Calibration of the ACC/AHA PCEs in the CRIC study.

Supplemental Figure 4. Prediction performance of the CKD predictor patch in the CRIC study.

Supplemental Figure 5. Prediction performance of novel CRIC study models by CKD stage.

Supplemental Figure 6. Prediction performance of novel CRIC study models in external validation data.

Supplemental Table 1. Discrimination and IPA for 10-year risk prediction of ASCVD.

Supplemental Table 2. Internal-external validation of 10-year risk prediction of ASCVD in the CRIC study.

Supplemental Table 3. Example calculations of predicted 10-year risk of ASCVD (ACC/AHA PCEs variables).

Supplemental Table 4. Example calculations of predicted 10-year risk of ASCVD (CRIC clinical model).

Supplemental Table 5. Example calculations of predicted 10-year risk of ASCVD (CRIC enriched model).

Supplemental Table 6. Baseline characteristics of development (CRIC study) and external validation (ARIC and MESA) cohort participants.

Supplemental Table 7. Reclassification of events for risk prediction models for ASCVD in CKD.

Supplemental Table 8. Reclassification of nonevents for risk prediction models for ASCVD in CKD.