Predicting Outcomes in Nephrology: Lots of Tools, Limited Uptake: How Do We Move Forward?

As doctors and health care providers, we focus on both individuals and populations and gain insights from clinical experience and published literature and use that combination of experience and knowledge to predict events in the future to improve the outcomes of our patients.

Ideally, our thoughtful application of all information available to us will help to optimize the initiation of medications, enable informed discussion around decision making, and risk-stratify individuals or groups so that resources used to care for them are appropriately allocated.

Predicting outcomes in CKD is important from several perspectives: in the clinical realm for individuals, in health policy to inform strategies and resource allocation, and in research to improve enrollment and inform sample size.

The outcomes we choose to predict include short term within 2–5 years, medium and longer term >5–10 years, and are usually those outcomes for which data to predict are easily measured or ascertained from medical records. In the nephrology world, we have been preoccupied with predicting progression to kidney failure or death and to some extent cardiovascular events because these are the outcomes most relevant to our patients. We recognize the very long horizon that this chronic condition has and that events occur throughout the life cycle and may have different implications at different points in time for individuals and health care systems.

In medicine, the most used prediction equations include Framingham Risk Score (10-year CV risk) and the Congestive Heart Failure, Hypertension, Age and Diabetes score (risk for stroke in nonrheumatic A fib), and others include Respiratory Failure Risk Index (postoperative respiratory failure risk) and Fracture Risk Assessment Score (10-year osteoporotic fracture risk).^1–4 The addition of kidney variables to Systematic Coronary Risk Evaluation and Probability of Cardiovascular Event scores, which predict CV events in the general population, has been used to further optimize CV risk assessment in those with CKD.⁵ Note that some CV prediction models were developed in very remote cohorts and were not always internationally representative, but nonetheless are used in clinical practice ubiquitously. The 2023 Scientific Statement from the American Heart Association has suggested new sex-specific race-free risk equation for 10-year and 30-year estimates for total cardiovascular disease including eGFR, with other models that add in other factors such as urine albumin creatinine ratio hemoglobin A1c, and social determinants of health.⁶

In nephrology, we have the KFRE (kidney failure risk equation with 2-year and 5-year risk for kidney failure), Hemodialysis Mortality Risk (6-month mortality risk in those on maintenance dialysis), and the Kidney Donor Risk Index (risk of kidney failure in donor after donation).^7–9 All of these tools have been developed using the best methodology at the time and externally validated.

However, despite the development of several tools to predict events that are important to us and to our patients, the uptake of them into clinical practice is quite poor. Note that although the Framingham risk equation predicts risk of only >10% of a CV event in 10 years, it has been used to initiate statin use for decades. Contrast that with KFRE which predicts kidney failure requiring dialysis or transplantation within 2 or 5 years: A very dire event within a short time frame and yet still not heavily used in clinical practice even a decade after extensive validation in numerous populations around the world.

Multiple performance metrics for prediction equations exist to help us evaluate them: discrimination, calibration, Net Reclassification Index (NRI), Integrated Discrimination Index, and net benefit. Discrimination refers to the predicative ability of the model for an event of interest, calibration is a measure which determines agreement between observed and predicted outcomes, NRI is a sum of the differences between appropriate and inappropriate reclassification, Integrated Discrimination Index is a sum of NRI over all the possible cutoffs for the possible outcomes, and net benefit assesses the usefulness of a prediction model in clinical decision making. Each of these performance metrics has benefits and shortcomings, and perhaps given the mathematical nature of them, they are not well understood in the context of clinical practice. As pointed out in the article by Milders et al., in this edition of the journal, there is large variation in the quality of reporting and model development and not all these metrics are reported.¹⁰ The scoping review describes the publication of novel models, external validation of existing models, and updating of models used in CKD, dialysis, and transplant populations. The authors note an underrepresentation of patients from Africa, South America, and Australia, which may limit their applicability in those regions. They note that models for predicting patient-reported outcomes (like quality of life or life participation) are scarce or nonexistent and that often sample sizes are small, reporting guidelines not adhered to, and is some, no, or inappropriate performance metrics reported. Perhaps one of the most important findings was that very few of the models published were presented in a useable format (regression formula or risk score) which hampers both validation and subsequent implementation.

In the past 7 years, in the kidney space, a variety of authors have described the potential value of using prediction models in clinical practice. Potok et al. described the improved accuracy of KFRE in predicting dialysis needs in a cohort of patients which exceeded the estimates of the physicians¹¹; others have examined the utility of KFRE as adjuncts to optimizing the timing of vascular access.¹² The use of components of kidney function (eGFR and albuminuria) in different risk equations to facilitate decision making for different outcomes (kidney failure, CV events, AKD, and death) has been described.⁵

A key question is why there is a poor uptake of prediction tools in nephrology: Do we not trust them, have we not spent time understanding when and how to use them, or do we doubt their utility in individual circumstances? Do we need to develop implementation programs to demonstrate their utility in clinical practice and that their use leads to better decision making or outcomes for patients or health care systems? Until recently there were few interventions to delay progression of CKD or effectively prevent CV events: So have we been reluctant to use prediction tools because of our inherent nihilistic attitude that there is little we can do, so why bother to predict?

How do we bridge the gap between promising prognostication models that may help one to identify people at risk for specific events, clarify time points in disease trajectory for decision making or intensified therapy, and optimize outcomes for individuals and optimize use of limited resources within health care systems?

With the advent and increasing availability and sophistication of electronic medical records (EMRs) worldwide, embedding the risk equations into EMRs with decision thresholds and proposed action plans seems an obvious and reasonable approach. Where EMRs do not exist, or cannot embed the tools, easy-to-access downloadable applications onto smart phones should be accessible to all. However, perhaps the step before that is to convince nephrologists and non-nephrologists that using these prediction tools actually does facilitate care and can improve efficiency and appropriateness of interventions. In clinical practice, we need to demonstrate that these tools improve clarity of conversations with patients and colleagues, lead to improved timing of access referral, or referral for dialysis education or transplantation. We would need to examine the best way to use the tools, and how best to communicate numeric risks to patients and colleagues. To date this aspect of implementation has been understudied. In research, perhaps including risk prediction equations as part of eligibility criteria for specific studies would help with recruitment and enrichment of study cohorts with those with a higher likelihood of the outcomes of interest. The socialization of the potential utility of prediction models remains a challenge, although their use is encouraged in the upcoming guidelines for the management and evaluation of CKD.

We should invest in research to test the utility and performance of existing clinical prediction models in diverse patient populations and in different health care systems to inform care pathways and decisions. How and if patients and clinicians accept the use of these tools and if they would accept their actions being guided by them remains unknown and requires study. Without understanding both components (true impact on decision making or outcomes and acceptance by patients and clinicians), the call for widespread implementation of validated prediction models will not be heard. Refocusing research efforts on evaluating the effect of using both existing and patient-centered prediction tools may be the value proposition required to truly improve clinical outcomes. In parallel, evaluating the impact of using them in clinical trials, to streamline study enrollment and execution, may also encourage their use in clinical practice.

Disclosures

A. Levin reports Consultancy: Astra Zeneca, Bayer, Boeingher Ingleheim; Chinook Therapeutics; GSK; Janssen, Kidney Foundation of Canada; NIH, Novo Nordisk; Occuryx/Certa, Otsuka, REATA; Research Funding: Astra Zeneca; Boehringer Ingelheim; Canadian Institute of Health Research (CIHR); CITF (Canadian Immunology Task Force), GSK, Health Research BC, Kidney Foundation of Canada; MOH BC, Shared Care BC; Honoraria: AstraZeneca; Bayer; GSK; Janssen; NIH; Advisory or Leadership Role: AstraZeneca, Boehringer Ingelheim, and NIDDK, DSMB for NIDDK, Kidney Precision Medicine, U Washington Kidney Research Institute Scientific Advisory Committee; International Society of Nephrology Research Committee; KRESCENT (Kidney Scientist Education Research National Training Program); GSK, Chinook Therapeutics, BC Renal (Exec Director), Steering Committee Chair CURE Consortium, CADTH, CITF, Co Lead Long Covid Web Grant; PI CanSOLVE CKD; and Other Interests or Relationships: CREDENCE National Coordinator from Janssen, directed to her academic team; NIDDK CURE Chair Steering Committee; International Society of Nephrology; Canadian Society of Nephrology; Kidney Foundation of Canada, Steering Committee ALIGN trial, DSMB Chair RESOLVE Trial (Australian Clinical Trial Network)l ASPIRE Trial (ACTN), Executive Steering Committee AstraZeneca Clinical Trial.

Funding

None.

Author Contributions

Conceptualization: Adeera Levin.

Writing – original draft: Adeera Levin.

Writing – review & editing: Adeera Levin.