Artificial Intelligence in Acute Kidney Injury: From Static to Dynamic Models

Abstract

Artificial intelligence (AI) is the development of computer systems that normally require human intelligence. In the field of acute kidney injury (AKI) AI has led to an evolution of risk prediction models. In the past, static prediction models were developed using baseline (eg, preoperative) data to evaluate AKI risk. Newer models which incorporated baseline as well as evolving data collected during a hospital admission have shown improved predicative abilities. In this review, we will summarize the advances made in AKI risk prediction over the last several years, including a shift toward more dynamic, real-time, electronic medical record-based models. In addition, we will be discussing the role of electronic AKI alerts and decision support tools. Recent studies have demonstrated improved patient outcomes through the use of these tools which monitor for nephrotoxin medication exposures as well as provide kidney focused care bundles for patients at high risk for severe AKI. Finally, we will briefly discuss the pitfalls and implications of implementing these scores, alerts, and support tools.

Artificial intelligence (AI), a term first coined by John McCarthy in 1956, was initially defined as “the science and engineering of making intelligent machines”.¹ The field of medicine, which has been dependent on advancing technologies, has unsurprisingly adapted the use of AI in various clinical aspects including but not limited to drug development, health monitoring, medical data management, and disease diagnosis.² AI in medicine is often split into 2 categories: physical and virtual.¹ Physical AI refers to instruments that directly assist in patient care with examples including robot-assisted surgeries and advanced neural prostheses to aid physically disabled patients. Virtual AI, or machine learning, uses computer algorithms to analyze large quantities of data, or “big data”, to discover patterns that aid in medical decision-making.^3,4 Across the medical spectrum, the processing of big data contained within the electronic health records has led to the creation of risk prediction models that have anticipated outcomes ranging from postoperative inpatient mortality to the development of myocardial infarctions to the need for ICU transfer.^5,6

At the Acute Disease Quality Initiative consensus conference in 2015, acute kidney injury (AKI) was recognized as an ideal disease state to apply machine learning and big data.⁷ This is because AKI not only affects a large portion of hospitalized patients but is associated with high morbidity and mortality making disease prediction extremely valuable to patient outcomes.^5,8 In addition, with serial monitoring of renal function, there exists predisease data sets to allow for prediction and temporal detection of AKI.⁹ Finally, given that standard definitions to diagnose AKI have been widely recognized through various criteria including RIFLE, AKIN, and KDIGO, AKI prediction models can be universally applied.^10,11 Our current markers for AKI including serum creatinine (SCr) and urine output are indicators of kidney function but not of kidney damage causing potential delays in detection. In theory, if AI can predict AKI early enough, preventative interventions can be applied in a timely fashion to avoid severe disease progression.

Numerous traditional risk assessment models have been created to predict AKI in various patient populations including those undergoing general surgery, those admitted to intensive care units, and those receiving iodinated contrast.^12–14 This is not meant to diminish the importance of traditional risk scores as when well-constructed and externally validated they may be of use. Recently Bell and colleagues have published a much simpler 4-variable model that was developed from population level data (including outpatient data) in 273,450 patients from the Tayside region of Scotland and then externally validated in data from Kent, UK, and Alberta, Canada.¹⁵ Their unique 4-variable model (age, baseline GFR, presence of diabetes mellitus, and presence of congestive heart failure [Table 1]) provided a C-statistic (95% CI) of 0.80 (0.80–0.81) for the development of KDIGO serum creatinine–based AKI. The score performed slightly worse in the Kent cohort (n = 219,091), 0.71 (0.70–0.72), but a little better in the larger Canadian cohort (n = 1,173,607), 0.76 (0.75–0.76).¹⁵ This simple score, which combined outpatient data with demographics and comorbidities, remains one of the few models to be externally validated in a large-scale cohort.

Table 1.

Bell and Colleagues—AKI Risk Score¹⁵

	Point
Age
20–29	0
30–49	2
50–59	4
60–69	6
70–79	8
>80	9
eGFR
> 60 mL/min	0
45–59 mL/min	5
30–44 mL/min	8
< 30 mL/min	16
Diabetes mellitus	3
Heart failure	4

Abbreviations: AKI, acute kidney injury; eGFR, estimated glomerular filtration rate.

A score of ≤4 provides a positive predictive value (PPV) of 3.6% and negative predictive value (NPV) of 99.5% for any AKI. While ≤14 provides a PPV of 15.7% and NPV of 98.7% for any AKI.

Cardiac surgery–associated AKI has been a focus for traditional risk score studies due to high rates of morbidity and mortality.¹⁶ All seven of the cardiac surgery–associated AKI risk prediction models reviewed by Huen and colleagues used variables determined in the preoperative setting with prior renal insufficiency, prior heart surgery, age, and diagnosis of diabetes being the most frequently used.¹⁷ Of the seven models studied, only 2 used any variables beyond the preoperative period including increased operative time, inotrope requirement, postoperative volume overload, and low cardiac output.^18,19 Only four of the 7 models studied had external validation with areas under the receiver operator characteristic curve (AUC) ranging from 0.66 to 0.86. However, of these models, all 4 predicted AKI requiring dialysis that merely occurs in roughly 1% of patients undergoing cardiac surgery.²⁰ These initial cardiac surgery–associated AKI prediction models have significant limitations not only for their narrow definition of AKI but also for homogenous patient population, limited external validation, but perhaps most of all for their static view of risk. As patients progress through their perioperative course, their risk changes based not only on any changes in urine output or SCr but also due to their hemodynamics, administered medications, postoperative complications, as well as other interventions.

MODERN SURGICAL RISK MODELS—NO LONGER STATIC

As such, others have attempted to use acute risk factors to predict postoperative AKI risk knowing that intraoperative or early postoperative factors seem to impact the incidence and severity of AKI in the postoperative period.^21–23 Recently, Mathis and colleagues sought to determine the links between intraoperative hypotension and AKI and preoperative risk. Analyzing 138,021 cases of major noncardiac surgery from 8 hospitals from 2008 to 2015, they defined hypotension as the lowest mean arterial pressure range achieved for more than 10 minutes with ranges being defined in absolute and relative decreases. They defined risk based on the patient’s preoperative kidney function, American Society of Anesthesiology Physical Status score, anemia, and anticipated duration of anesthesia, and determined that in patients with the highest risk—mild hypotension (MAPs of 55–59) was associated with a 1.34 (1.16–1.56) increased adjusted odds of developing AKI.²⁴ Dividing their cohort into derivation and validation groups, they were able to demonstrate and replicate that relative hypotension (percent decrease) was not as predictive of impending AKI compared with absolute hypotension. Regardless, their use of big data to tease out the complicated interplay of preoperative risk, intraoperative hypotension, and development of AKI is part of a larger movement in incorporate real-time/intraoperative data into assessing AKI risk.²⁴

Lei and colleagues used data from 42,615 patients undergoing noncardiac surgery at 4 tertiary academic hospitals in the United States to develop several models to predict AKI.²⁵ Using logistic regression, gradient boosting machine (GBM) learning, and a random forest approach, they developed several models and assessed their performance through the addition of prehospitalization, preoperative and intraoperative variables. The inclusion of these successive time-based variables did lead to an increase in model performance. For example the GBM models predicted KDIGO AKI within the first 7 postoperative days with AUCs (95% CI) of 0.712 (0.69–0.73) for the prehospital data, 0.804 (0.79–0.82) for the inclusion of the preoperative variable and maxing out at 0.817 (0.80–0.83) for all the data.²⁵ The authors do not provide information about which features gained or loss importance in these progressive models. While the clinical significance of an increase in AUC of 0.013 between these last 2 modes remains unclear, it demonstrates that the inclusion of readily available intraoperative data (eg, intraoperative vitals, medications, blood products, etc.) improves AKI risk prediction.

Separately, Bihorac and colleagues have used single-center retrospective data in concert with machine learning and advanced analytic techniques to develop perioperative models to predict AKI in the first 3 and 7 postoperative days.²⁶ Using random forest classifiers, they used preoperative and intraoperative variables to predict outcomes in 2911 adults. They demonstrated that inclusion of intraoperative data improved the AUC from 0.84 to 0.86 with an improvement in accuracy from 0.76 to 0.78. In addition, including these data led to improvement of the net reclassification for patients (predominantly the false negatives) by 8% at 3 days and 7% at 7 days.²⁶

ASSESSING AKI RISK IN OTHER POPULATIONS

Building on these aforementioned static models that only attempted to forecast AKI risk based on preoperative or risk factors at the time of ICU admission, there has been an increasing number of published models that have attempted to add additional information to their models in the hopes of improving accuracy. Malhotra and colleagues conducted a multicenter prospective cohort study to develop and validate a risk score for predicting AKI within the first 7 days of ICU admission.²⁷ The model was developed in 573 patients from UCSD and validated in 144 patients from UCSD and 1300 patients from the Mayo Clinic. In total, 754 (37%) developed KDIGO-defined AKI. Their final model consisted of 10 independently weighted risk factors (Table 2) and provide an AUC of 0.81 (0.79–0.83) in the validation cohort (0.79 in the developmental cohort).²⁷ Although this has not been validated by others, it serves as a new and fairly parsimonious model based on the presence of baseline and a few acute risk factors.

Table 2.

Malhotra and Colleagues AKI Risk Prediction at ICU Admission²⁷

	Risk Factors	Points
Chronic	CKD	2
	Chronic liver disease	2
	Congestive heart failure	2
	Hypertension	2
	Atherosclerotic coronary disease	2
Acute	pH ≤ 7.30	3
	Nephrotoxin exposure	3
	Severe infection/sepsis	2
	Mechanical ventilation	2
	Anemia	1

Abbreviations: AKI, acute kidney injury; CKD, chronic kidney disease.

A cutoff value of greater than 5 points provided a positive predictive value of 32% and a negative predictive value of 95%.²⁷

The use of progressive data to enhance AKI risk prediction is not isolated to surgical cases. Moving away from these more traditional models, Flechet and colleagues developed the AKI predictor, a random forest machine learning risk algorithm for critically ill adults. Using data from the Early vs Late Parenteral Nutrition in Critically Ill Adults (EPaNIC) multicenter database (n = 4490), they developed and validated an AKI risk score using clinical information available (1) before and (2) on ICU admission as well as (3) data from ICU day 1.²⁸ The model was used to predict all 3 stages of AKI as well as just stage 2 or 3. Model performance for predicting any AKI outcome consistently improved with progressive data added to the model (eg, AUCs increasing from 0.77 (0.77–0.77) to 0.84 (0.83–0.84) for the prediction of stage 2 or 3 AKI. In addition, in their original validation cohort they demonstrated that their progressive model outperformed neutrophil gelatinase associated lipocalin for the development of stage 2 or 3 AKI (AUC 0.79 (0.79–0.79) compared with the aforementioned 0.84. Subsequently, these same authors further compared their machine learning algorithm’s ability to predict AKI to the treating physicians’ ability to predict the same outcomes.²⁹ In 252 patients from 5 ICUs from a single tertiary academic center, the AKIpredictor did not outperform physicians in its ability to predict stage 2–3 AKI but there was no statistical difference in their performance (AUC 0.75 (0.62–0.88) vs 0.80 (0.69–0.92), P = 0.25). Physicians tended to overestimate the risk of AKI, thus allowing the AKIpredictor to have a higher net benefit compared with physicians but in total this study was limited in that only 30 (12%) patients developed severe AKI.²⁹

In addition to these aforementioned models, there have been several models developed and published in the last 5 years that have sought to use both advanced AI techniques (eg, neural networks and machine learning) as well as harness the power of big data and the electronic medical records to predict AKI across all hospitalized patients.

Recently, Tomasev and colleagues published a seminal article on an AKI risk score using data from 703,782 adult patients in the United States Department of Veterans Affairs Healthcare System (172 inpatient and 1062 outpatient locations).³⁰ They used a recurrent neural network to develop a highly accurate model that could detect KDIGO-defined AKI. Using the full capacity of the electronic medical record including 620,000 base features (variables), they attempted to predict AKI with a lead time of 48 hours interpreting data in 6-h intervals. Their final model (33% precision) provided a sensitivity of 55.8% and specificity of 82.7%, based on a 2:1 false to true alert ratio. While this may be a higher than desired false positive rate, it is important to note that 25% of the false positives were in patients who eventually developed AKI; however, not within the first 48 hours. It seemed like prior CKD may have impacted these results. The model performed much better at predicting severe dialysis requiring AKI, providing 84.3% correct prediction of RRT within the next 30 days and over 90% accuracy at 90 days. However, this study was not without its limitations. Given that it used data from the Veteran’s administration, less than 7% of all subjects were female, limiting its generalizability. In addition, they used a randomly selected group of patients to serve as their test set, which has been shown to provide optimistic estimates of accuracy as opposed to external validation.³¹ In addition, they used deep learning and included over 300 features making the model fairly complex and it is unclear how pairing the model down with fewer features would impact discrimination and accuracy. While this model should eventually be validated in a more generalized global population, it remains the state-of-the-art model for AI detection of AKI.

This is not to say that there are no other AI models that have been developed and investigated. Koyner and Churpek have published a series of 3 articles using AI techniques to develop and validate AKI risk models. The first article, published in 2016, sought to predict ward based AKI in a cohort of 202,961 patients from 5 hospitals across Chicago.³² Owing to their retrospective data, they were not able to link to prehospitalization data and excluded all patients who were admitted with an SCr > 3.0 mg/dl as well as those receiving RRT within the first 48 hours of admission. Using a discrete time survival model (akin to the 6-h windows used by Tomasev and colleagues) the developed a cubic spline regression model using patient demographics, vital signs, and routine laboratory data (including blood urea nitrogen and SCr). In the 17,541 (8.6%) patients who developed any form of SCr-based AKI, the model provided an AUC of 0.74 (0.74–0.74) but the model performed significantly better at predicting stage 3 AKI (n = 1242 (0.6%)) with an AUC of 0.83 (0.83–0.84).³² Building on this regression model, they subsequently published a single-center GBM model that predicted SCr-based AKI and receipt of RRT in the next 48 hours, using a similar discrete time survival analysis.³³ In this tertiary academic center hospital cohort of 121,158 patients, 17,482 (14.4%) of whom developed KDIGO SCr-stage 1–based AKI and 821 (0.6%) of whom received RRT, over 53% of subjects were female and over 50% were Black.

This machine learning model which included over 130 features including demographics, vital signs, laboratory values, interventions, medications, and receipt of specific diagnostic tests (eg, radiology examinations and echocardiograms) provided excellent ability to detect AKI risk in both the derivation and validation cohorts.³³ In the validation cohort, the model predict all stages of AKI within the next 48 hours with increasing accuracy stage 1 (0.73 (0.72–0.73)), stage 2 (0.87 (0.87–0.87)), stage 3 (0.93 (0.93–0.93)), and receipt of RRT (0.96 (0.96–0.96)).³³ Importantly, this model performed equally well across patient locations (ward and ICU) as well as across admission SCr strata (<1.0, 1.0–1.9, 2.0–2.9). Most recently, they have published a follow-up article, where they have pared-down to 59 features and derived a new GBM data from a single center (University of Chicago). They then went on to validate this smaller GBM model in data from 2 separate health care systems (Loyola University Medical Center n = 200,613 and NorthShore University HealthCare n = 246,985), respectively. They were able to demonstrate training the model on one cohort, this simplified GBM model still provided AUCs of >0.85 for the prediction stage 2 and > 0.91 for stage 3 SCr-based AKI in the next 48 hours. The accuracy of this model was not impacted by differences across patient demographics and baseline kidney function. Similarly, this GBM model provided AUCs of >0.95 for the receipt of RRT in the next 48 hours in both cohorts. The authors did not provide information about which features were most informative across the 3 distinct hospitalized cohorts. While their model remains limited in that it does not define AKI using the urine output criteria and data restrictions limited the ability to use outpatient data for establishing a true baseline SCr, it remains the first large-scale externally validated machine learning AKI risk assessment algorithm. In the ensuing years we anticipate validation and refinement of these and other scores. Perhaps future risk scores will advance these pre-existing scores through the use of natural language processing or other AI techniques that will readily incorporate physician notes and the verbiage from diagnostic test reports/result.

IMPLEMENTATION OF RISK SCORES

Importantly just because you can predict AKI risk does not mean you can prevent AKI from happening. While several risk scores have been developed and validated, there is much less published data around the implementation and pitfalls of the modern risk scores. Much of what we know about implementation has been learned from clinical trials.^34–37

Several groups have used real-time risk scores or SCr-based alerts to further our understanding around the optimal care in patients with AKI. While some of these investigations have been randomized trials,^35,38 there have been several nonrandomized interventions which sought to pair some form of automated creatinine-based AKI Alert with regimented clinical care.^{34,35,39–41} Selby and colleagues performed a multicenter, pragmatic, step-wedge cluster randomized trial across 5 hospitals in the United Kingdom involving the implementation of early AKI e-alerts as well as AKI care bundles and a kidney-care focused educational program for all hospitalized patients, these alerts were all based on the standard consensus definition of AKI.³⁵ Others have used more complicated risk scores as their trigger for enrollment/intervention. Hodgson and colleagues performed a controlled before and after clinical trial across 2 British hospitals (one with the intervention and one without) in which 30,295 acute admission were flagged as being high risk based on the real-time calculation of their risk score which includes prior medical diagnoses, patient demographics but also real-time patient vitals as well as a nursing assessment score (Alert, Voice, Pain, Unresponsive AVPU Score).³⁴ Using a clinical prediction rule in the medical record, patients were flagged as being high risk for the development of AKI as well as flagging those with early-stage AKI. Doctors for the flagged patients also received alerts about their patient’s risks with attached AKI care bundles (Table 3). They demonstrated a minimal difference in before and after hospital-acquired AKI (HA-AKI) in the intervention hospital compared with the control site (odds ratio (95% CI) 0.99 (0.98–1.00); P = 0.049). There was also decreased inpatient mortality in the HA-AKI (0.73 (0.56–0.95); P = 0.021). There was no change in any outcomes for those with community acquired AKI (1.01 (0.79–1.29); P = 0.95).³⁴

Table 3.

Elements of the AKI Bundle Implemented in the Study by Selby and Colleagues³⁵

Assess volume status and optimize blood pressure Treat sepsis Medication reconciliation and stopping those contributing to AKI Perform urinalysis Referral (to nephrology or critical care) for stage 3 AKI or any AKI with complications Investigate the cause of AKI Manage hyperkalemia Alert patient to the presence of AKI

Abbreviation: AKI, acute kidney injury.

More recently, Wilson and colleagues have published data on an AKI prediction model developed from a retrospective data set of 170,000 hospital admissions.⁴² Subsequent to that they activated this model in real-time in their EHR, to determine the feasibility of prospective algorithm-directed AKI prediction.³⁷ They conducted a single center, prospective observational study over a 1.5 year period in which high-risk patients (risk of AKI within 24 hours was 15% as predicted by their algorithm) received a “pre-AKI” alert and had their risk reported in real time until they actually developed KDIGO AKI, at which point risk assessment stopped. Of the 2856 high-risk patients, 18.9% went on to develop AKI in 24 hours and this closely mirrored the 19.1% predicted risk at the time of enrollment.³⁷ To determine the feasibility of using this electronic risk score with other tests to improve risk stratification, enrolled patients then had urine and serum biomarkers of AKI measured along with urinalysis/microscopy.³⁷ Pairing electronic risk scores with other biomarkers may serve as a method to detect those at highest risk for the most severe forms of AKI. This group has recently developed and published a similar parsimonious, continuously updated risk score for the prediction of AKI in children, and so we would anticipate further investigations of similar projections around pediatric AKI.⁴³

Finally all of this work by the Wilson research group has been leading toward a larger, a double-blind, multicenter, parallel, randomized controlled trial across 6 hospitals to determine whether electronic health record alerts for AKI can improve patient outcomes (AKI progression, need for dialysis and mortality).⁴⁴ Patients with serum creatinine-based KDIGO stage 1 AKI were randomized to have their care providers receive an electronic alert which lasted 48 hours after the diagnosis of AKI and was linked to an AKI order (blood and urine tests, imaging, nursing, and dietary orders). Patients randomized to the alert group (n = 3059) were more likely to receive an order for intravenous fluids (absolute difference of 3.8% (1.4–6.2)) as well as have creatinine remeasured within 24 hour of randomization 87.2% vs 85.2%.⁴⁴ The primary outcome (progressive AKI, dialysis, and death) occurred in 21.3% of the alert group and 20.9% of the usual care (n = 2971); P = 0.67. Importantly there was a concerning signal for harm in the nonteaching (2 of 6) where 14-day mortality was higher in the alert group compared with usual care (15.6% vs 8.6%), relative risk 1.82 (1.22–2.72); P = 0.003. The relative risk for the same outcome in teaching hospitals was 0.89 (0.74–1.06); P = 0.18.⁴⁴ This study highlights 2 important tenants of AKI care, intervening whether with fluids, diuretics or other care plans in the setting of early AKI is not 100% benign and waiting until creatinine is already elevated is not the optimal timing for AKI interventions. Table 4 summarizes this trial as well as several others (discussed below) that have used AI and clinical decision support in the setting of AKI.

Table 4.

Summary of Recent Studies Investigating Electronic AKI Alerts Linked to Action Plans

Study	Study Design	Size	Results	Limitations
Kolhe et al.40	Prospective observational study of AKI care bundle coupled with an AKI alert.	2297 patients with 2500 episodes of AKI.	The care bundle was only completed in 12.2% of patients within the first 24 hours. Completion of the bundle was associated with significantly less severe AKI, less AKI progression and lower inpatient and 60-day mortality.	Observational, nonrandomized study.
Park et al.39	Before and after quality improvement study of AKI alert system with option for automated nephrology consults.	1884 patients in the before group and 1309 in the intervention (after) group.	AKI alert led to less overlooked AKI, defined as the absence of a follow-up SCr within 2 weeks of the original AKI. Despite being sicker, those in the intervention phase less likely to develop severe AKI and more likely to recover their SCr to within 20% of baseline.	Nonrandomized, single-center data, there were clear differences between the before and after groups. The AKI alert was not always real time, sometimes delayed several hours.
Hodgson et al.34	Controlled before and after study across 2 hospitals (one with intervention and one without).	30,295 acute medical admissions across 2 distinct 10-month periods.	In the intervention period a clinic prediction rule (CPR) flagged both patients at high risk for the future development of AKI as well as those with early AKI. Doctors for the flagged patients also received alerts with attached AKI care bundles. They demonstrated a difference in hospital-acquired AKI (HA-AKI) in the intervention hospital. There was also decreased inpatient mortality in the HA-AKI. There was no change in any outcomes for those with community acquired AKI.	Nonrandomized data. The CPR (risk assessment) was reliant on diagnosis codes.
Selby etal35	Multicenter, pragmatic stepped-wedge cluster randomized trial of a multi-faceted intervention (AKI e-alert, AKI care bundle and education program).	24,059 AKI episodes across 5 hospitals.	Over the intervention period there was no change in 30-day mortality. Hospital length of stay was reduced over the course of the intervention period. There was improved AKI recognition and AKI care care (eg, medication reconciliation).	Strict reliance on SCr-based AKI. Some portions of the intervention may not be generalizable outside of the National Health Services in England.
Menon et al41	Prospective nonrandomized study (quasi before-after) combining and electronic alert and standardized care pathway to evaluate AKI detection and progression in children.	239 episodes of AKI in 225 pediatrics.	Reduction in AKI stage by 71.4% for those getting the decision support tool compared to 64% and 55% in those without the tool. Higher GFRs at discharge in those getting the tool despite no difference at baseline.	Single-center, nonrandomized center with some patients coenrolled in the NINJA study.
Ugwuowo et al.37	Prospective observational cohort of high-risk algorithm predicted AKI in the next 24 hours with a subset of patients getting biomarkers measured.	2856 inpatient adult admissions in a single-center tertiary care hospital.	Patients flagged as high risk developed AKI in the next 24 hours 18.9% of the time. Based on the algorithm the predicted incidence should have been 19.1%. In a convenience sample of 100 patients with UAs and biomarkers lower fractional excretions of urea and presence of hyaline casts were associated with increased incidence of developing AKI.	Single-center study, with limited numbers of patients getting biomarkers measured.
Wilson et al.44	Prospective, double-blind, multicenter parallel group randomized controlled trial examining electronic alerts and order sets for adults with KDIGO stage 1 AKI.	6030 adults with creatinine-based stage 1 AKI across 6 separate hospitals (teaching and nonteaching).	There was no difference in AKI progression, need for dialysis or mortality comparing those with alerts (21.3%) vs usual care (20.9%). The alert group had a significantly higher mortality rate in nonteaching hospitals (15.6% vs 8.6%).	Did not use urine output definition of AKI. Alerts were not sent to pharmacists or nurses and had limited patient-specific recommendations.

Abbreviation: HA-AKI, hospital-acquired acute kidney injury.

Table updated and altered from the study by Koyner et al.⁵⁴

NINJA AND OTHER DECISION SUPPORT TOOLS

As nephrotoxin-based AKI is the most common form of hospital-acquired pediatric AKI, there have been several recent efforts to decrease/eliminate its incidence.⁴⁵ Goldstein and colleagues originally conducted a single-center, prospective, quality improvement project which implemented a decision support tool (DST) to limit the nephrotoxin exposure in noncritically ill pediatric patients already receiving aminoglycosides or 3 or more other nephrotoxins.^46,47 Utilizing the KDIGO AKI criteria, they demonstrated that surveillance, the DST trigger, and pharmacists recommended increased SCr monitoring led to lower AKI rates. In 1749 patients over 2358 admissions, the AKI rate decreased by 64% (2.96-1.06 AKI episodes/1000 patient days). This was in part due to a decrease in the exposure rate by 38% (11.63-7.24 exposures/1000 patient days).⁴⁷ The success of this trial project which was named Nephrotoxic Injury Negated by Just in time Action (NINJA) led to a multicenter quality initiative which demonstrated similar findings. In this nine-center study that included 638,695 inpatient hospital days, they replicated the same inclusion criteria and demonstrated a 23.8% reduction in AKI rates per exposure. This corresponded with a drop in AKI prevalence from 1.7 to 1.3 episodes of AKI per 1000 patient days.⁴⁸ Others have taken this concept and applied it outside the setting of the pediatric wards. Askenazi and colleagues published the Baby NINJA study over 1.5 years in a single center, level IV neonatal ICU and demonstrated a reduction in AKI incidence from 9.1 to 2.9 per 100 susceptible patient days (P < 0.001). This corresponded with 100 fewer episodes of AKI over an 18-month period.⁴⁹

Separately, Kellum and colleagues conducted a multicenter, sequential period analysis from 2012 to 2015 of 528,108 patients to determine the impact of a clinical decision support system to reduce length of stay and mortality for patients with AKI.⁵⁰ Over the study period, there was a modest decrease in AKI associated crude mortality from 10.2 to 9.4%. Compared with historical controls from the same institution, a multivariate mixed-effect model demonstrated a 1.2 day decrease in length of stay along with a 0.76 (0.70–0.83) adjusted odds of inpatient mortality.⁵⁰ Thus while the effect sizes were small they were sustained over the study period and demonstrate promise. In fact, this same group has continued this decision support system and published another 2 years of data, demonstrating over this period a 0.82 (0.75–0.89) odds of mortality and a 3.7 day decrease in length of stay for patients with AKI (with only a 0.8 day decrease in stay for those without AKI).⁵¹ Although these observational data/results cannot causally link the DST with these outcomes, taken together with the aforementioned other clinical support data, it certainly shows promise.^41,47,48 We hope that in the next few years, there will more data about the ability of decision support tools to help decrease nephrotoxin exposure in adults and all critically ill patients.

There are many issues to consider when implementing a risk score or decision support tool in real time. While score characteristics will help shape the thresholds that are chosen, these important test qualities should not be overlooked. As alarm fatigue has real consequences, alerting clinical teams to impending risk when that risk is low may create more problems than it solves. Thus, a risk assessment tool’s positive predictive value must be high enough to minimize the number of false positive who will likely benefit from any planned intervention. Enrolling large numbers of patients in trials who will not go on to meet end points, such as severe AKI, is fraught with potential complications.^52,53 In addition, the planned intervention will also impact the treating physicians desire to complete these actions. While automated care plans or care bundles such the one detailed in Table 3 may theoretically be ideal; there are limited data to support one specific intervention over another in preventing all forms of AKI.

There are many ways that AI is helping to move the care of patients with AKI forward. AI has been used to of state-of-the-art AKI risk scores and implement clinical decision support tools for patients at risk for or with early AKI. In the future, we can envision a seamless process, where on admission, risk scores identify patients who are at high risk and prompt clinicians to order follow-up risk stratification testing (eg, biomarkers of tubular injury). From there, those who remain at high risk will have their treating team alerted supplying them with improved/optimized care plans perhaps even helping to streamline the initiation of a nephrology consult. Although this may sound futuristic, the bedside clinician is still integral to this entire process as none of the scores or decision support tools discussed in this review are 100% accurate and all have their failings. This entire process remains dependent on the treating physicians. AI will never replace the clinical knowledge needed to ascertain which high-risk patients still need an aminoglycoside for or how to manage a high-risk patient with chronically low mean arterial pressures in the setting of heart failure in need of diuresis. As more and more AI tools are developed and validated it is important to remember that these tools are meant to help improve care and outcomes not replace the clinician.

CLINICAL SUMMARY.

• Dynamic risk models that incorporate real-time variables (eg, intraoperative and postoperative factors) have improved accuracy compared with static/fix models.

• Advanced artificial intelligence techniques (eg, machine-learning and neural networks) have helped develop highlight accurate acute kidney injury (AKI) risk prediction models; however, there is no data on their ability to improve outcomes after clinical implementation.

• The Nephrotoxic Injury Negated by Just in time Action (NINJA) and its associated follow-up studies have demonstrated a clear reduction in pediatric AKI after the implementation of a clinical decision support tool around the administration of nephrotoxic medications.

• Several clinical trials using serum creatinine and risk score–based trigger electronic alerts and kidney focused care bundles have demonstrated improved patient outcomes with earlier action in the setting of AKI and AKI risk.