Advances in artificial intelligence and deep learning systems in ICU related AKI

STRUCTURED ABSTRACT

Purpose of review:

Acute kidney injury (AKI) affects nearly 60% of all patients admitted to intensive care units (ICUs). Large volumes of clinical, monitoring, and laboratory data produced in ICUs allow the application of artificial intelligence (AI) analytics. The purpose of this article is to assimilate and critically evaluate recently published literature regarding AI applications for predicting, diagnosing, and subphenotyping AKI among critically ill patients.

Recent findings:

Among recent studies regarding AI implementations for predicting, diagnosing, and subphenotyping AKI among critically ill patients, there are many promising models but few had external validation, clinical interpretability, and high predictive performance. Deep learning techniques leveraging multimodal clinical data show great potential to provide continuous, accurate, early predictions of AKI risk, which could be implemented clinically to optimize preventative and early therapeutic management strategies.

Summary:

Use of consensus criteria, standard definitions, and common data models could facilitate access to machine learning-ready data sets for external validation. The lack of interpretability, explainability, fairness, and transparency of AI models hinders their entrustment and clinical implementation; compliance with standardized reporting guidelines can mitigate these challenges.

INTRODUCTION

Acute kidney injury (AKI) affects up to 60% of all patients admitted to intensive care units (ICUs).[1–3] Prevention, early diagnosis, and appropriate treatment are essential to mitigate the substantial burden of associated short- and long-term mortality and morbidity. AKI confers increased risk for chronic critical illness with poor long-term survival and quality of life, especially when recovery of renal function is delayed or incomplete.[4]

AKI in critical illness has complex pathophysiology that is difficult to represent with simple decision-making algorithms. Astute, highly trained critical care providers may be fully capable of performing an efficient, comprehensive diagnostic workup, but results from necessary tests often yield results hours later and confer substantial healthcare costs. Yet, the complex pathophysiology of AKI in critical illness can be accurately represented by artificial intelligence models (AI) using routinely collected clinical data. Peer-reviewed literature on this topic is expanding rapidly, presenting a mixture of high-quality evidence harboring the potential to improve care and outcomes, as well as variable-quality evidence that distracts and confuses investigators and clinicians.

The purpose of this article is to assimilate and critically evaluate recently published literature regarding artificial intelligence applications for predicting, diagnosing, and subphenotyping AKI among critically ill patients. We performed a systematic review of major, publicly available databases for salient peer-reviewed articles published in the prior 18 months.

METHODS

PubMed was systematically searched for articles related to the development or validation of Acute Kidney Injury prediction or clustering models published during an 18-month period ending June 2021. The search leveraged medical subject heading (MESH) terms and was batched into three groups: ICU, AKI, and prediction/modeling. Two-hundred and twenty-seven articles were identified by search criteria and were screened independently by two reviewers, with disagreements resolved by a third reviewer (Figure 1). Full texts of 46 articles that passed title/abstract screening underwent full text review, excluding articles that lacked a predictive component (11), did not use AKI as an outcome (5), used non-ICU patients (1), or were not an algorithm-based study (1). Information was extracted from each of the remaining 30 articles.

Figure 1.

Data Extraction Flow Diagram

AKI ALGORITHMS

We identified 30 studies published between January 2020 and July 2021 meeting inclusion criteria, with 27 studies describing the development or validation of AKI prediction models (Tables 1 and 2), two studies on AKI subphenotyping (Table 3), and one study on both.

Table 1.

Overview of the prediction models for acute kidney injury in ICU patients

Study	Study design	N	Region and time period	Patient Population	Outcome	Baseline sCr	Method	Prediction frequency	Prediction window	Prevalence
Deep Learning Models
Alfieri 2021[5**]	Retrospective Development	21,681	USA, Boston, MIMIC-III database 2001–2012 USA, eICU 2014–2015	ICU adult patients	AKIN stage 2 and AKIN stage 3 using sCr and Urine output	Lowest sCr in ICU	CNN	Every hour	18th hour from ICU admission until AKI stage 2/3 onset	3.0%
	Retrospective Validation	3,331								3.3%
	Retrospective Test	7,080								3.0%
	Retrospective Calibration	3,481								3.4%
Le 2021[17**]	Retrospective Development and 10-fold Cross Validation	12,347	USA, Boston, MIMIC-III 2001–2012	ICU adult patients	AKI KDIGO sCr and urine output	Smaller of MDRD sCr and the 20th percentile of sCr	CNN	Once	Within 24 h	Not reported
	Retrospective Development and 10-fold Cross Validation	6,821							Within 48 h	20.67%
Morid 2020[18]	Retrospective Development and 20-fold Cross Validation	38,579	USA, Boston, MIMIC-III 2001–2012	ICU adult patients	AKIN sCr	Lowest sCr during the ICU stay	RF, Kernel-based Bayesian Network, XGBoost, NN, SVM, kNN, LR, NB	Once	48 hours after ICU admission until hospital discharge	57%
Qian 2021[19**]	Retrospective Development	6,286	USA, Boston, MIMIC-III 2001–2012	ICU adult patients	KDIGO sCr and urine output	Minimum sCr within first 24 hours of ICU admission	LR, SVM, RF, XGBoost, LightGBM, CNN	Once	During ICU stay	50%
	Retrospective Validation	1,572								50%
Roy 2021[20**]	Retrospective Development	41,630	USA, Boston, MIMIC-III database 2001–2012	ICU adult patients	AKI KDIGO sCr	Not reported	Sequential subnetwork routing	Continuous, every hour	48 hours	28.7% for entire cohort
	Retrospective Validation	5,204
	Retrospective Test	5,204
Sato 2021[25**]	Retrospective Development and 5-fold Cross Validation	5,342	USA, eICU 2014–2015	ICU patients	AKI stage 1 or above	Lowest sCr within seven days before ICU admission	1D CNN	Each 15 min interval	24 to 48 hours from the admission	50%
	Retrospective Development and 5-fold Cross Validation	1,450			AKI stage 2 or above					50%
Vagliano 2021[22**]	Retrospective Development	38,201	USA, Boston, MIMIC-III database 2001–2012	ICU adult patients	AKI KDIGO sCr and urine output	Not reported	LSTM (RNN)	Continuous, every six hours	48 hours before the onset of AKI	78.6% for entire cohort
	Retrospective Validation	4,775
	Retrospective Test	4,775
Xu 2020[23**]	Retrospective Development and 5-fold Cross Validation	37,486	USA, Boston, MIMIC-III 2001–2012	ICU adult patients	KDIGO sCr, urine output and RRT criterion	Not reported	Memory networks	Once	Within 7 days after 24 hours / 48 hours of ICU admission	20.4%
Schwager 2021[26*]	Retrospective Development	78,776	USA, Rochester, Mayo Clinic 2005–2017	ICU adult patients	AKI KDIGO sCr and urine output	Mean of sCrs within 180–7 days before hospital admission	GBT, LR, RF, NN	Once	During ICU stay	39.92%
	Retrospective Validation	19,694								39.92%
Machine Learning Models
Gong 2021[16**]	Retrospective Development	9,958	USA, Boston, MIMIC-III database 2001–2012	ICU adult patients	AKI KIDGO sCr and RRT criterion	MDRD study or admission sCr for patients without or with CKD respectively	Ensemble model using LR and RF	Once	Next 48 hours after first 24 hours within ICU admission	14%
	Retrospective Validation	2,489								14%
Lee 2020[7**]	Retrospective Development	15,803	USA, Boston, MIMIC-III database 2008–2013	ICU patients	AKI receiving interventions and ICD codes	NA	GBT	Once for each prediction window	1) Next 24 hours 2) Entire ICU stay	50.2%
	Retrospective Validation	3,948								63.8%
	Retrospective Test	2,187								52.4%
	Retrospective Development	10,445	USA, Washington 2014–2016							72.7%
	Retrospective Validation	2,641								33.8%
	Retrospective Test	1,450								63.7%
Shawwa 2021[21**]	Retrospective Development	78,777	USA, Rochester, Mayo Clinic 2005–2017	ICU adult nonpregnant patients	AKI KDIGO sCr only, urine output only and both	Mean of sCrs within180–7 days before hospital admission	GBT	Once	During the ICU stay	39.9%
	Retrospective Validation	19,694								39.9%
	Retrospective External Validation	18,529	USA, Boston, MIMIC-III database 2001–2012							46%
Wang 2020[10**]	Retrospective Development	7,832	China, ICUC	ICU adult patients	KDIGO by sCr	Not reported	XGBoost	Once	24- / 48-hour prediction window before AKI onset	15.59% for entire cohort
	Retrospective Validation	5,221
	Retrospective External Validation	52,152	USA, Boston, MIMIC-III Database, 2001–2012							56.27%
Wong 2021[32*]	Retrospective Development and 5-fold Cross Validation	940	Singapore, Singapore General Hospital 2015–2017	Surgical ICU patients	AKI AKIN sCr	Lowest 90 days pre-ICU creatinine	Combination of DTs and gSEM	Once	First 48 h of ICU entry	21.4%
Logistic Regression Models
An 2020[34]	Retrospective Development	583	China, Tianjin January 2017 - December 2017	NICU patients with ABI	AKI KDIGO sCr and urine output	First sCr within 3 days of ICU admission	LR	Once	During hospital stay	12.17%
Coulson 2021[6]	Retrospective Development	17,048	Australian and New Zealand, ANZSCTS database September 2016 - December 2018	ICU patients with Cardiac Surgery	AKI KDIGO sCr	Immediate preoperative creatinine	LR	Once prior to surgery, once immediately after surgery	Surgery end to discharge	25.6% for entire cohort
	Retrospective Validation	5,683
Deng 2020[13]	Retrospective Development	2,042	USA, Boston, MIMIC-III database 2008–2012	ICU adult patients	AKI KDIGO sCr	KDIGO standards	LR	Once	24 hours within ICU admission	57.1%
	Retrospective Validation	875								53.6%
Fan 2021[14]	Retrospective Development	11,008	USA, Boston, MIMIC-III database 2001–2012	ICU adult patients with sepsis	AKI KDIGO sCr	First sCr after ICU admission	LR	Once	Within 48 hours after ICU admission	61%
	Retrospective Validation	4,718								61%
Fan 2021[15]	Retrospective Development	522	USA, Boston, MIMIC-III database 2001–2012	ICU patients with DKA	AKI KDIGO by sCr and urine output	KDIGO criteria	LR	Once	During ICU stay after DKA	42.9%
	Retrospective Validation	228								37.7%
Fu 2021[28]	Retrospective Development	157	China, Shenzhen January 2015-October 2018	ICU patients with septic shock	AKI KDIGO sCr	Not reported	LR	Once	Within 48 hours of ICU admission	53.5%
Ma 2021[29]	Retrospective Development	232	China, tertiary care hospital October 2014-April 2019	ICU septic patients	KDIGO by sCr and urine output	Pre-ICU creatinine based on hierarchy	LR	Once	Within 7 days of ICU admission	29.7%
	Retrospective Validation	126								19.8%
Matsuura 2020[8]	Retrospective Development	4,151	Japan, Tokushukai Medical Group 2012–2014	ICU adult patients	AKI (KIDGO sCr) stage 2/3 last over three days within one week of ICU admission	0.8 mg/dL	LR	Once	Within 7 days of ICU admission	12.9%
	Retrospective Validation	4,169								12.6%
Qiu 2021[30]	Prospective Observational Study	44	China, Pudong March 2017-August 2018	AKI patients with sepsis or septic shock	AKI (KDIGO sCr and urine output) lasting more than 48 hours	Lowest value in the previous 3 months	LR	Once	ICU stay	54.55%
Trongtrkul 2020[9]	Secondary Analysis from a Prospective Observational Study	3, 474	Thailand, THAI-SICU April 2010-January 2013	ICU adult patients underwent major non-cardiothoracic surgery before ICU admission	AKI KDIGO sCr criteria	Smaller of the lowest ICU admission sCr and MDRD sCr; Lowest 3-month pre-ICU sCr for patients with CKD	LR	Once	Within 7 days of ICU admission	9.6%
Vaara 2020[27]	Retrospective Development	780	USA, Austin Hospital 2011–2012	ICU adult patients	AKI KDIGO creatinine and RRT criteria	First point-of-care creatinine within 2 hours of ICU admission	LR	Once	Second day of ICU admission	18.97%
Wang 2020[31]	Prospective Development	628	China, Shangai July 2011- March 2013	Post cardiac surgery patients	AKI KDIGO sCr	Latest sCr after admission but before surgery	LR	Once	After the surgery	28.30%
Wu 2020[33]	Retrospective Development	84	China, a tertiary-care teaching hospital 2010 −2015	ICU patients with HS	AKI KDIGO sCr	Not reported	LR	Once	NA	35.7%
Yu 2021[11]	Retrospective Development	495	China, Fujian, 2013–2019	ICU adult patients with severe sepsis or septic shock underwent echocardiography	Presence of AKI criteria based on KDIGO sCr and urine output and consensus criteria for sepsis	Not reported	LR	Once	Within 24 hours of ICU admission	68.7%

Abbreviations: AKI, acute kidney injury; CKD, chronic kidney disease; ICU, intensive care unit; AKIN, Acute Kidney Injury Network; NICU, neurosurgery/neurological intensive care unit; ABI, acute brain injury; DKA, diabetic ketoacidosis; sCr, serum creatinine; GBT, gradient boosted tree; DT, decision tree; KDIGO, Kidney Disease Improving Global Outcomes; HS, hemorrhagic shock; MDRD, Modification of Diet in Renal Disease; SVM, support vector machine; XGBoost, extreme gradient boosting tree; NN, neural network; kNN, k-nearest neighbor; NB, naïve bayes; gSEM, generalized structural equation model; LR, logistic regression; CNN, convolutional neural network; LightGBM, light gradient boosting decision machine; LSTM, long-short term memory; RNN, recurrent neural network; ML, machine learning; RRT, renal replacement therapy.

Table 2.

Summary of the study design and performance of the algorithms

Study	Validation Type	Model specification	AUROC (95% CI or SD)	Other model performance (95% CI or SD)
Deep Learning Models
Alfieri 2021[5**]	Retrospective Internal Validation	CNN	0.89 (0.01)	Sp = 82.0% (3%), Se = 82.0%; Sp = 84.0% (3%) at Se = 80.0%
Le 2021[17**]	Retrospective Internal Validation	CNN, 48hr	0.856 (0.034)	Sp = 76.3% (5.70%), Se = 80.4% (0.00%), PPV = 23.6% (3.90%), F1= 0.361 (0.047)
		CNN, 24hr	0.863 (0.009)	Sp = 77.2%, Se = 80.3%, PPV = 22.1% (1.60%), F1 = 0.345 (0.019)
Morid 2020[18]	Retrospective Internal Validation	RF, local and global	0.809	Accuracy = 0.813
		NN	0.737	Accuracy = 0.735
Qian 2021[19**]	Retrospective Internal Validation	LightGBM	0.905	Se = 83.6%, PPV = 97.1% F1 = 0.897, Accuracy = 0.905
		CNN	0.718	Se = 73.0%, PPV = 71.5%, F1 = 0.722, Accuracy = 0.718
Roy 2021[20**]	Retrospective Internal Validation	SeqSNR, full feature set	0.793 (0.016)	AUPRC = 0.481 (0.444 – 0.513)
		SeqSNR, reduced feature set	0.782 (0.766–0.800)	AUPRC = 0.472 (0.435 – 0.514)
Sato 2021[25**]	Retrospective Internal Validation	1D CNN, 1st stage AKI	0.742 (0.010)	-
		1D CNN, 2nd stage AKI	0.844 (0.029)	-
Vagliano 2021[22**]	Retrospective Internal Validation	LSTM (RNN)	Continuous = 0.940	Brier Score = 0.101
Xu 2020[23**]	Retrospective Internal Validation	MN, 24hr	0.775 (0.013)	Se = 63.0%, PPV = 50.0%
		MN, 48hr	0.780 (0.013)	Se = 70.0%, PPV = 50.1%
Schwager 2021[26*]	Retrospective Internal Validation	PyTorch NN	-	PPV = 51.0%, NPV = 79.6% with either sCr or UO
Machine Learning Models
Gong 2021[16**]	Retrospective Internal Validation	VOTE (LR and RF)	0.773 (0.749 – 0.788)	Se = 64.5% (59.6% - 69.1%), PPV = 34.5%
		XGBoost	0.774 (0.748 – 0.789)	Se = 67.5% (62.5% - 72.8%), PPV = 32.6%
Lee 2020[7**]	Retrospective Internal Validation	GBT instance level (MIMIC-3)	-	Se = 62.2%, PPV = 43.1%, F1 = 0.509
		GBT instance level (UW-CDR)	-	Se = 50.7%, PPV = 43.8%, F1 = 0.470
		GBT patient level (MIMIC-3)	-	Se = 47.5%, PPV = 62.3%, F1 = 0.539
		GBT patient level (UW-CDR)	-	Se = 63.0%, PPV = 44.4%, F1 = 0.521
Shawwa 2021[21**]	Retrospective Internal Validation	GBT	0.690	Sp = 71.1%, Se = 56.2% F1 = 0.611, AUPRC = 0.585
	External (MIMIC III) Validation	GBT	0.656	Sp = 64.2%, Se = 58.3% F1 = 0.634, AUPRC = 0.602
Wang 2020[10**]	Retrospective Internal (ICUC) Validation, 24hr	Ensemble Time Series Model using XGBoost	0.810	Se = 75.0% AUPRC = 0.590, F1 = 0.580
	Retrospective internal (ICUC) validation, 48hr	Ensemble Time Series Model using XGBoost	0.780	Se = 68.0% AUPRC = 0.410, F1 = 0.440
	External (MIMIC III) Validation, 24hr	Ensemble Time Series Model using XGBoost	0.950	Se = 95.0% AUPRC = 0.980, F1 = 0.960
	External (MIMIC III) Validation, 48hr	Ensemble Time Series Model using XGBoost	0.950	Se = 98.0% AUPRC = 0.980, F1 = 0.980
Wong 2020[32*]	Retrospective Internal Validation	Combination of DTs and gSEM	Range 0.733–0.795	-
Logistic Regression Models
An 2020[34]	Retrospective Internal Validation	Multivariate LR	0.879	-
Coulson 2021[6]	Retrospective Internal Validation	Multivariate LR, preoperative AKI	0.67	Brier Score = 0.180
		Multivariate LR, postoperative AKI	0.70	Brier Score = 0.170
Deng 2020[13]	Retrospective Internal Validation	Multivariate LR	0.79 (0.03)	Sp = 69.4%, Se = 72.0%, PPV = 75.5%, NPV = 65.4%
Fan 2021[14]	Retrospective Internal Validation	Multivariate LR	-	PPV = 72.6%, NPV = 57.3%, C-Index = 0.712 (0.697 – 0.727)
Fan 2021[15]	Retrospective Internal Validation	Multivariate LR	0.712 (0.642 – 0.782)	Sp = 69.7%, Se = 66.3%, PPV = 57.0%, NPV = 77.3%, C-index = 0.733
Fu 2021[28]	Retrospective Development	Multivariate LR	0.686 (0.6 – 0.77)	Sp= 67.0%, Se = 63.0% PPV = 67.0%, NPV = 62.0%
Ma 2021[29]	Prospective Validation	Multivariable LR	0.784 (0.703 – 0.865)	cNRI = 0.660 (0.335 – 0.985)
Matsuura 2020[8]	Retrospective Internal Validation	Multivariable LR	0.79 (0.02)	Sp = 81.3%, Se = 63.8%, PPV = 33.0%, NPV = 93.9%, Brier Score = 0.084, CS = 1.06
Qiu 2021[30]	Retrospective Development	Univariate LR	0.710 (0.47 – 0.87)	Sp = 40.0%, Se = 83.3%
Trongtrakul 2020[9]	Retrospective Internal Validation	Multivariate LR	0.821 (0.797 – 0.845)	-
Vaara 2020[27]	Retrospective Development	Multivariate LR	0.830 (0.040)	cfNRI = 90.0 (74.9–106.1), IDI = 0.20 (0.160–0.240)
Wang 2020[31]	Retrospective Development	Multivariate LR	0.810 (0.770 – 0.850)	-
Wu 2020[33]	Retrospective Development	Multivariate LR	0.840 (0.730 −0.930)	Sp = 77.0%, Se = 82.0%
Yu 2021[11]	Retrospective Development	Multivariate LR	0.728 (0.680 – 0.777)

Abbreviations. LR, logistic regression; SVM, support vector machine; DT, decision tree; RF, random forest; NN, neural network; XGBoost, extreme gradient boosting; CNN, convolutional neural network; SeqSNR, sequential subnetwork routing; 1D-CNN, one-dimensional convolutional neural network; GB, gradient boosting; LightGBM, light gradient boosting decision machine; gSEM, generalized structural equation model; LSTM, long-short term memory; RNN, recurrent neural network; GBT, gradient boosting tree; MN, memory network; Sp, specificity; Se, sensitivity; F1, F Score; AUROC, area under the receiver operating characteristic; AUPRC, area under the precision-recall curve; NPV, negative predictive value; PPV, positive predictive value; cNRI, continuous net reclassification index; cfNRI, category-free net reclassifying index; IDI, incremental discrimination improvement; CS, calibration slope; CI, confidence interval; SD, standard deviation; AKI, acute kidney injury; sCr, serum creatinine; UO, urine output.

Table 3.

Overview of the subphenotyping models for ICU patients with acute kidney injury.

Study	Study design	N of AKI cases	Region and time period	Patient Population	AKI Definition	Baseline sCr	Method	Number of Subphenotypes
Xu 2020[23**]	Retrospective development	7,657	USA, Boston, MIMIC-III 2001–2012	ICU adult patients	KDIGO sCr, urine output and RRT criterion	Not reported	MN + K means	3
Wiersima 2020[12]	Prospective development	301	Finland, 17 Finnish ICUs, 1st September 2011 – 1st February 2012	Patients with sepsis and AKI during the first 48 h from admission to ICU	KDIGO sCr urinary output, the combination of sCr and urinary output, and RRT	Baseline sCr was available for 69% of patients and, if missing, estimated using the MDRD formula	LCA	2
Chaudbary 2020[24]	Retrospective development	4,001	USA, Boston, MIMIC-III 2001–2012	Patients older than18 with sepsis who developed AKI within 48 hours of ICU admission	KDIGO sCr, urine output	The lowest creatinine 7 days prior to ICU admission	Deep autoencoder+ K means	3

Abbreviations. AKI, acute kidney injury; ICU, intensive care unit; KDIGO, Kidney Disease Improving Global Outcomes; LCA, latent class analysis; MDRD, Modification of Diet in Renal Disease; MN, memory network; RRT, renal replacement therapy; sCr, serum creatinine.

Patient Populations

Twenty-two studies recruited patients from a single center; eight studies reported results from two or more centers.[5**,6,7**,8,9,10**,11–12] The most common source of patient data was Medical Information Mart for Intensive Care (MIMIC) III, which was used in 15 studies.[5**, 7**, 10**, 13–15,16**,17**,18,19**,20**,21**,22**,23**,24] While most studies (n=16) included mixed ICU populations,[5**, 7**, 8, 10**, 13, 16**,17**,18,19**,20**,21**,22**,23**, 25**,26*,27] there were seven studies focused on sepsis patients,[11, 12, 14, 24, 28–30] four focused on surgical patients,[6, 9, 31, 32*] one on diabetic ketoacidosis,[15] one on hemorrhagic shock,[33] and one on acute brain injury.[34]

Outcome Definitions

Among 30 studies reviewed, 26 studies defined AKI using Kidney Disease: Improving Global Outcomes (KDIGO) criteria; 13 used serum creatinine criteria alone and 13 used both urine output and serum creatinine criteria (Tables 1 and 3). Three studies used Acute Kidney Injury Network (AKIN) criteria[5**, 18, 32*] and one used International Classification of Diseases codes.[7**]

Outcomes included AKI stage ≥1 or ≥2,[25**] persistent AKI (defined as any stage AKI lasting more than 48 hours),[30] stage 2 or 3 AKI persisting for ≥3 days,[8] and presence of any stage of AKI for the remaining 25 studies. Windows used for outcome labeling ranged from 24 hours to 7 days or during hospital admission.

Baseline Serum Creatinine

Methods for determining the baseline serum creatinine (sCr) varied across studies, with some using the first sCr measured within 3 days of ICU admission[14, 19**, 27, 29, 31, 34] and others using the lowest value during ICU admission.[5**, 9, 18] Studies also established baseline sCr from the lowest sCr prior to admission (ranging from 365 days to one week prior[25**, 29, 30, 32*] or within 7 days prior[24]), mean sCr from 180 to 7 days prior to admission,[21**, 26*] preoperative sCr,[6, 31] or the Modified Diet and Renal Disease (MDRD) equation if a measured sCr was not available.[9, 12, 26*, 30] Eleven studies did not define baseline sCr.[7**, 10**,11–13, 15, 20, 22**, 23**, 28, 33]

PREDICTION MODELS

The most common modeling technique was logistic regression (LR). Twenty studies evaluated at least one LR model; fourteen evaluated LR models exclusively. Nine articles[5**, 17**,18,19**,20**, 22**, 23**, 25**] used deep learning models and five [7**, 10**, 16**, 21**, 26*, 32*] used other machine learning (ML) models: gradient boosting trees (GBT), random forest (RF), support vector machines (SVM), Naïve Bayes (NB), k-Nearest Neighbor (kNN), and decision trees (DT). Model performance was most commonly assessed by area under the receiver operating characteristic curve (AUROC), ranging from 0.52 to 0.98 (Table 2 and Supplemental Table 1).[10**] Other commonly used performance metrics included sensitivity, specificity, positive predicted value (PPV), negative predicted value (NPV), accuracy, and area under the precision and recall curves (AUPRC), a more informative metric for class-imbalanced predictive tasks.[25**] The highest performing models used ensemble models with dynamic, temporal components that analyze trends over time.[10**]

Deep Learning Models

Deep learning techniques are increasingly applied to large-scale, high-dimensional, multimodal ICU data, such as clinical note and physiological time series data. This approach shows great potential to provide continuous, accurate, early prediction of AKI risk.

Recurrent neural networks (RNN) and one-dimensional (1D) convolution neural networks (CNN) are particularly effective for parsing temporal data by allowing continuous predictions of patient risk as more data become available over time. Vagliano et al.[22**] developed a bidirectional long-short term memory (LSTM) network, which is a RNN subtype, to use temporal physiologic, laboratory, and therapeutic intervention variables to generate predictions updated every six hours; this approach outperformed RF, GBT, and LR models. Interpretability was measured via integrated gradients, demonstrating that creatinine and urine output in the past 6 hours were the most important risk factors. Alfier et al.[5**] developed a 1D CNN model to continuously predict the probability of developing AKI stage 2 or 3, achieving greater performance than LR. Similarly, Sato et al.[25**] generated a 1D-CNN to predict AKI every 15 minutes within 24–48 hours after ICU admission. Multiple time series of vital signs and serum creatinine values along with demographic information were used as model features, achieving greater performance than RNN, XGB, and LSTM. Using score-weighted class activation mapping, the system identified important time points and variables for each patient’s predictions.

Roy et al.[20**] proposed a multi-task model for hourly predictions of six outcomes: AKI, continuous renal replacement therapy, mechanical ventilation, vasoactive medications, length of stay, and mortality using hourly mean and standard deviations of laboratory values and vital signs. The model was compared with single-task and naïve multitask models, and yielded a statistically significant performance improvement for 4 of 6 tasks, demonstrated advantages for multitask modeling using EHR data, especially when training data are limited or endpoint labels are difficult to ascertain.

In contrast, Qian et al. and Morid et al. developed a CNN and NN, respectively, to predict AKI risk and achieved fair performance, where features from physiologic time series data were either transformed into summary statistics or other temporal patterns instead of feeding raw time series data into networks directly [18, 19**]. Schwager et al. [26*] compared GBT, LR, RF, and NN for predicting AKI classified by sCr alone, urine output alone, and either sCr or urine output. Interestingly, AKI classified by urine output alone was primarily associated with patient demographics, whereas AKI determined by Cr alone was primarily associated with patient comorbidities.

Applying natural language processing to clinical notes and extracting meaningful features can augment early AKI predictions.[35, 36] Xu et al.[23**] developed a memory network (MN) model to learn patient representations from multimodal data to predict future AKI risk. The model integrated useful information from unstructured clinical notes (extracted from a hierarchical LSTM model) and time-dependent, structured EHR data (such as laboratory measurements and vital signs), and yielded the best performance compared with both machine learning and other deep learning models that did not use multimodal data. Le et al.[17**] developed a multichannel, multiheaded attention model together with a CNN for time series data and a separate Doc2Vec network handling the document vector of clinical notes to extract useful information for AKI risk prediction. The model performed significantly better than XGBoost model and sequential organ failure assessment (SOFA).

Machine Learning Models

Several types of machine learning methods other than LR and deep learning were used to predict ICU-related AKI. Tree-based classifiers, GBT, and XGBoost were the most prevalent classifiers in AKI prediction, offering strong performance in clinical applications, high efficiency, and scalability.[7**, 10**, 16**, 21**, 26*] Gong et al.[16**] developed prediction models for AKI within 24 hours of ICU admission using LR, DT, SVM, RF, NN, XGBoost, and a voting scheme based on the combination of LR and RF. XGBoost and a voting scheme based on LR and RF yielded greatest predictive performance. To address the need for a flexible, robust interpretability tool, their framework was equipped with Shapley additive explanations. Wang et al. developed an ensemble time series model (ETSM) relying on XGBoost, processing vital signs and laboratory measurements to generate “explicit indicators”, and produced “implicit indicators” from medication information.[10**] ETSM demonstrated greater performance than NB, k-NN, AdaBoost, and RF models.

GBT is a supervised learning method and therefore requires labelled examples. Lee et al.[7**] employed “distant learning” to heuristically label data for training the GBT. Evidence of benefits for distant learning was not reported. Shawwa et al. employed GBT to predict AKI during ICU admission and did not achieve good performance when externally validated.[21**] Some variants of tree-based algorithms have been employed for identifying important features in AKI prediction. Wong et al. [32*] characterized important features by exploiting Chi-Square Automatic Interaction Detector and Classification and Regression Trees models. Their analyses via generalized structural equation model (gSEM) identified significant associations between hemoglobin levels and AKI.

COMPUTABLE PHENOTYPES OF AKI

Since AKI is a silent disease and delayed recognition and treatment are associated with worse clinical outcomes, it is essential to apply a consistent AKI definition and standardized definitions of baseline sCr to EHR datasets to accurately and rapidly identify AKI. While there are studies describing computable phenotypes for AKI or implementation of AKI alerts among ICU patients,[37–42] our search of literature from the past 18 months did not identify any relevant studies.

AKI SUBPHENOTYPING

There is substantial heterogeneity in the cause, severity, trajectory and outcomes of ICU-related AKI that are influenced by inherent disease mechanisms, the patient’s physiological response, and the care process, thereby concealing unique pathophysiologic processes specific to certain AKI populations. AKI subphenotyping is needed to identify subgroups of AKI patients with shared, modifiable, biological mechanisms, presenting differences in treatment responses or a high risk of mortality permitting early targeted interventions. Recent advances enable data-driven AKI subphenotyping using machine learning techniques. Bhatraju et al.[43] developed a latent class analysis (LCA) model using 29 variables including demographics, comorbidities, labs, ICU events, and biomarkers. The authors discovered two molecularly distinct subphenotypes that exhibit differences with regard to underlying pathophysiology, response to vasopressin therapy, and risk for adverse clinical outcomes. In a similar study by Wiersema et al.[12], a LCA model was developed using similar variables and identified two subphenotypes of sepsis-associated AKI patients, one of which was associated with a lower probability of short-term renal recovery and increased 90-day mortality. Chaudhary et al.[24] used deep learning algorithms to extract information from EHR data including laboratory measurements and vital signs and applied k-means clustering to identify 3 sepsis-associated AKI subphenotypes. These subphenotypes had different risks for dialysis and mortality. Xu at al.[23**] first developed a memory network model to learn patient representation, which can best predict the future AKI risk from structured and unstructured data, and then applied k-means clustering to representations of AKI patients, identifying three distinct AKI subphenotypes with unique laboratory values (such as glucose, albumin, and creatinine) and significant correlations with different stages of AKI.

DISCUSSION

Acute kidney injury (AKI) is a common condition with high mortality and resource utilization among critically ill patients. Timely identification of patients at risk for developing AKI is essential. While artificial intelligence applications have substantial potential to augment the prediction, diagnosis, and treatment of AKI among critically ill patients, several challenges must first be overcome. Despite success in using AI/ML in clinical research, progress is hindered by limited datasets and models with insufficient interpretability, fairness, and reproducibility that are difficult to share and implement across institutions. First, for models to be integrated across multiple EHRs at different institutions, data must be standardized; we suggest compliance with the Fast Healthcare Interoperability Resources framework, which describes standards for health information sharing across EHRs and cloud-based communications, and the Observational Medical Outcomes Partnership Common Data Model to map data from individual institutions to standard concepts. Second, the quality of model features and predictive performance must be analyzed systematically, optimized, and carefully monitored over time, initially in “silent” analytic-only phases and subsequently in clinical implementation phases, using processes similar to those of Phases 1, 2, and 3 for standard clinical trials.[44] Details regarding these processes are described in SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence) extension[45] and CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence)[46] guidelines. The Transparent Reporting of a multivariable prediction model for individual Prognosis or Diagnosis statement recommendations summarizes recommendations for reporting prediction models.[47] As part of feature optimization, it is necessary to ensure that training datasets are representative of the patients for whom model predictions and classifications are applied. For example, the standard use of race adjustment in estimating glomerular filtration rate and reference creatinine can underestimate the incidence of CKD and AKI among African Americans.[48–50] Artificial intelligence algorithms learn from data; when trained on racially imbalanced datasets, algorithms may produce racially biased results.[51, 52] Few studies in our review performed external validation, which is needed to establish generalizability. Most available AKI prediction models have moderate performance and lack interpretability. Finally, assignment of liability and accountability for errors associated with algorithms remains challenging and requires further attention among medical-judicial experts.

CONCLUSIONS

Among recently published AI implementations for predicting, diagnosing, and subphenotyping AKI among critically ill patients, there are many promising models but few had external validation, clinical interpretability, and high predictive performance. Deep learning techniques leveraging multimodal clinical data show great potential to provide continuous, accurate, early predictions of AKI risk, which could be implemented clinically to optimize preventative and early therapeutic management strategies. Use of consensus criteria, standard definitions, and common data models could facilitate access to machine learning-ready data sets for external validation. The lack of interpretability, explainability, fairness, and transparency of AI models hinders their entrustment and clinical implementation; compliance with standardized reporting guidelines can mitigate these challenges.

Supplementary Material

Supplemental Table 1

Supplemental Table 1. Summary of the study design and performance of the algorithms of main and comparison models

Key points:

• Large volumes of clinical, monitoring, and laboratory data produced in ICUs allow the application of artificial intelligence (AI) in predicting, diagnosing, and subphenotyping AKI among critically ill patients.

• Deep learning techniques leveraging multimodal clinical data show great potential to provide continuous, accurate, early predictions of AKI risk, which could be implemented clinically to optimize preventative and early therapeutic management strategies.

• Use of consensus criteria, standard definitions, and common data models could facilitate access to machine learning-ready data sets for external validation.

• The lack of interpretability, explainability, fairness, and transparency of AI models hinders their entrustment and clinical implementation; compliance with standardized reporting guidelines can mitigate these challenges.