Cardiovascular Risk Prediction Scores in CKD: What Are We Missing?

The burden of atherosclerotic cardiovascular disease (ASCVD) is high in CKD, with varied but linked manifestations, including coronary heart disease, cerebrovascular disease, and peripheral arterial disease. ASCVD in CKD is a complex interplay of inflammation, oxidative stress, and endothelial dysfunction. More traditional risk factors play a role, but ASCVD in the setting of CKD appears to differ by factors such as qualitative modifications in lipid profiles, location and composition of vascular calcifications, and the very high prevalence and severity of coronary disease.^1,2

This unique pathophysiology and epidemiology likely underlie the reduced accuracy of traditional tools used to predict cardiovascular risk in the general population and has sparked interest in risk prediction scores and therapeutic trials specific to the CKD population.³ A CKD-specific quantitative risk score is fundamentally needed to direct appropriate therapies in individuals with CKD. Furthermore, dynamic reassessment every few years, to reflect the duration and longitudinal change in associated risk factors (particularly CKD progression), is likely to be important in contrast to recommendations for reassessment every 4–6 years in the general population. In addition, there is a growing focus on the utility of risk stratification for secondary prevention with the availability of different therapeutic options.

The Pooled Cohort Equations (PCEs) are recommended by the American College of Cardiology/American Heart Association for primary risk prediction in the United States.⁴ Despite recognizing CKD as a risk-enhancing factor, neither CKD nor its markers are included in this equation. Adding moderate CKD to the base equations on the basis of eGFR <60 ml/min per 1.73m² did not significantly improve overall model discrimination, but there were very few participants with CKD stage 4 in this analysis. Furthermore, whether addition of albuminuria to the model improves performance has not been evaluated.⁴ Therefore, guidance on how to best classify cardiovascular risk in individuals with advanced CKD remains ambiguous.

In this issue of JASN, Bundy et al.⁵ used the Chronic Renal Insufficiency Cohort (CRIC) to evaluate the performance of the PCEs in the CKD population and construct new models using CKD-specific risk factors, including routinely available laboratory values and novel biomarkers for the prediction of incident ASCVD events. Their work is commendable. The authors describe cohort selection, characteristics, variable measurements and selection, model development well. Risk factors such as troponin-T and urinary albumin-creatinine ratio (UACR) were selected from a pool of 50 candidate predictor variables. The new models (one using routine clinical variables and a biomarker-enriched model) improved discrimination, calibration, and reclassification of nonevents to the published PCE equation and one recalibrated for the CRIC cohort. To their credit, the authors performed internal validation using 10 × 10-fold crossvalidation and external validation in two independent prospective cohort studies (Atherosclerosis Risk in Communities study and Multi-ethnic Study of Atherosclerosis), where the performance metrics remained improved compared with PCEs, although the improvement was expectedly smaller than in the CRIC cohort. Of further interest, the final models included UACR but not eGFR, whereas age and glycemic control as assessed by hemoglobin A1C were potent risk factors, whose inclusion led to an improvement in model performance.

Although these results are promising, and suggest further validation of this tool is warranted, a few caveats apply. Although the biomarker-enhanced CRIC equation in particular had markedly better performance than the PCE equation, improvement compared with the CKD-patched equation (which adds UACR and eGFR) was marginal particularly for the CRIC equation using only standard clinical labs (area under the curve 0.76 versus 0.75). Furthermore, the CRIC model using standard clinical labs was only marginally better than the standard PCE equation (area under the curve 0.71 versus 0.70) in the external validation cohort. Lastly, labs such as UACR can have significant intraindividual variability, particularly if random samples are utilized. These issues raise concerns regarding whether the improvement in performance with the CRIC models is sufficient to merit the effort needed to broadly implement use of a novel equation, in place of the more widely recognized and available PCE equations.

More broadly, several cardiovascular risk-prediction calculators are available, but all tend to under- or overestimate risk in specific populations.⁶ Performance is limited by the lack of diversity in study cohorts or lower event rates in underlying clinical trials population, and inherent limitations of using population-level estimates to predict individual-level risk. This often results in modest performance of the risk scores on external validation, and may explain the failure to widely incorporate risk scores into clinical practice. A risk score will also be less compelling to clinicians if they are unsure of how it was created, and if the underlying factors are chosen on the basis of statistical models rather than on the basis of causal or physiologic underpinnings. The ingredients that made the recipe are critical and should be completely transparent to judge its clinical significance rather than just the statistical significance and model performance in a curated study cohort (Figure 1).

Figure 1.

Schema for successful implementation of a quantitative risk tool. Created with BioRender.com.

An even broader issue is that prediction scores are typically validated for a single baseline calculation, but atherosclerosis is a dynamic process. Given changes in aging, lifestyle, and environment, medications, and progression of medical comorbidities, risk assessment should not be static, but should ideally update with new information to prevent model decay. Finally, risk calculators are intended to reduce use of low-yield therapy in those at low risk, increase use of therapies in those at high risk, and facilitate treatment or further testing discussions between physicians and patients in those at intermediate risk. Thus, it is of paramount importance that the scores provide readily understandable and communicable information to patients. Unfortunately, in the absence of these implementation steps, manuscripts reporting on even highly accurate static risk scores may be much ado about nothing with regards to improving clinical care.

With the widespread success of predictive tools in shaping choices and behavior in online commerce, the failure of predictive analytics to match this success in health care is notable. As the paper by Bundy et al. shows, CKD-specific scores have the potential to improve prediction in the context of CKD, particularly when all available data such as novel biomarkers are utilized.

We “predict” a wave of additional risk assessment research that will likely include -omics, behavioral, and lifestyle data, and not just routine markers of existing subclinical disease. However, we would suggest future research needs to go beyond static risk prediction and do the necessary work to validate longitudinally updated models and their accompanying implementation. In conclusion, the aphorism by famous British statistician George Box still holds true that “all models are wrong, but some are useful.” To the scientists interested in data analytics and predictive modeling, we would urge they make them simple, easy to understand, and, above all, useful.

Disclosures

D.M. Charytan reports receiving personal fees or fees paid by Janssen Pharmaceuticals to the Baim Institute; reports receiving consulting fees from Allena Pharmaceuticals, Amgen, AstraZeneca, CSL Behring, Eli Lilly/Boehringer Ingelheim, Fresenius, Gilead, GlaxoSmithKline, Janssen (steering committee), Medtronic/Covidien, Merck, Novo Nordisk, PLC Medical, Renalytix, Zogenix, and Zoll; reports reciving research support from Amgen, Bioporto clinical trial support, Gilead, Medtronic, and NovoNordisk; reports having an advisory or leadership role with CJASN; and reports other interests or relationships through receiving expert witness fees related to proton pump inhibitors. The remaining author has nothing to disclose.

Funding

None.