Predicting 28-Day Mortality in Critically Ill Patients Receiving Continuous Renal Replacement Therapy: A Novel Interpretable Machine Learning Approach

Abstract

Purpose

This study aimed to develop and validate an interpretable machine learning (ML) model to predict 28-day all-cause mortality in critically ill patients undergoing continuous renal replacement therapy (CRRT), facilitating early risk stratification and clinical decision-making.

Patients and Methods

Data from 1362 CRRT patients were analyzed, including 1224 from the Medical Information Mart for Intensive Care IV database (training cohort) and 138 from a Chinese hospital (external validation cohort). Feature selection was performed using least absolute shrinkage and selection operator, support vector machine-recursive feature elimination, and Boruta algorithms. Nine machine learning models were constructed and compared, including Gaussian process (GP), ensemble methods (gradient boosting machine and eXtreme gradient boosting), and other classifiers. Model performance was assessed via the area under the receiver operating characteristic curve (AUC), decision curve analysis, and other metrics. The SHapley Additive exPlanation (SHAP) method was used to interpret the ML models.

Results

The GP model demonstrated consistent predictive performance across all cohorts, with training (AUC=0.841, accuracy=76.8%, sensitivity=65.5%), internal validation (AUC=0.794, accuracy=73.4%, sensitivity=60.0%), and external validation (AUC=0.780, accuracy=63.8%, sensitivity=39.0%) sets. Key predictors included red cell distribution width, age, lactate, septic shock, and vasoactive drug use. SHAP analysis provided transparent insights into feature contributions.

Conclusion

The GP-based model accurately predicts 28-day mortality in CRRT patients and demonstrates strong generalizability. By integrating SHAP explanations, it offers clinicians an interpretable tool to identify high-risk patients early, potentially improving outcomes.

Introduction

Continuous renal replacement therapy (CRRT) is a lifesaving treatment frequently used in critically ill patients. This technology is applicable not only for the treatment of acute kidney injury (AKI) but also for poisoning, sepsis, acute pancreatitis, and other conditions.^1–3 The literature indicates that approximately 250,000 patients worldwide receive CRRT annually, with approximately 15% of intensive care unit (ICU) patients receiving this therapy yearly.⁴ CRRT offers excellent solute clearance and hemodynamic stability, serving as a platform for various drugs and nutritional support while supporting multiple organ functions. Despite saving the lives of many critically ill patients, the mortality rate among CRRT patients remains high, ranging from 30% to 70%.¹ Consequently, identifying high-risk factors leading to death among CRRT patients and implementing early interventions are urgently needed to improve their prognosis.

While originally developed for AKI, CRRT is now widely applied to non-renal conditions in critical care. In severe burn patients with sepsis, CRRT can significantly reduce plasma levels of endotoxin and pro-inflammatory cytokines (eg, IL-6, TNF-α).⁵ Early initiation of CRRT can effectively remove pro-inflammatory cytokines such as TNF-α, thereby significantly reducing intra-abdominal pressure and improving clinical outcomes in patients with severe acute pancreatitis.⁶ Similar clinical benefits were observed in rhabdomyolysis patients, where earlier CRRT initiation was associated with significantly lower 90-day mortality rates.⁷ This expanded utilization underscores the need for mortality prediction tools applicable to all CRRT recipients, not just AKI patients.

With the development of data science and artificial intelligence technology, machine learning (ML) has been utilized to predict various clinical events, achieving remarkable results. Unlike traditional statistics, ML algorithms rely primarily on data to identify patterns and rules, whereas traditional statistics are typically based on a priori model assumptions. Consequently, ML algorithms exhibit superior predictive capabilities and adaptability to diverse data types and problems. Previous studies (eg, Hu et al⁸ and Fan et al⁹) have applied ML to AKI outcomes, but existing CRRT prognostic models^10,11 are limited by single-center data, lack of external validation, and a narrow focus on AKI patients. In contrast, CRRT has broader clinical applications, including sepsis, crush syndrome, and acute pancreatitis, highlighting the need for more comprehensive predictive models. Unlike previous studies focused exclusively on CRRT for AKI, our study is the first to develop an interpretable ML model for predicting mortality in all critically ill patients receiving CRRT, including those with sepsis, acute pancreatitis, and other non-renal indications. This broader scope addresses a critical gap in personalized risk stratification for heterogeneous CRRT populations.

Although ML demonstrates effective predictive capabilities, it has been criticized for its lack of interpretability, often referred to as the “black box” feature. To address this issue, the SHapley additive exPlanation (SHAP) method¹² was utilized to interpret ML models and visualize individual variable predictions.¹³ Unlike other interpretability techniques (eg, LIME or feature importance scores), SHAP provides both global and local explanations by quantifying each feature’s contribution to individual predictions based on game theory principles. This approach ensures consistency and aligns with clinical needs for transparent decision-making at both population and patient levels.

The objective of this study was twofold: first, to identify the optimal ML model for predicting 28-day all-cause mortality in patients undergoing CRRT; second, to quantitatively visualize the relationship between risk factors and outcomes via the SHAP method.

Materials and Methods

This study adhered to the updated TRIPOD+AI statement, providing guidance for reporting clinical prediction models that employ regression or ML methods.¹⁴

Training Cohort

The training cohort was sourced from the Medical Information Mart for Intensive Care IV (MIMIC-IV) database, version 2.0.¹⁵ The MIMIC-IV is a comprehensive, publicly accessible dataset comprising records of patients admitted to the ICU at Beth Israel Deaconess Medical Center in Boston, Massachusetts, USA, spanning the years 2008--2019. Access to this database was granted to the researcher after successful completion of an online ethics training and examination administered by the Massachusetts Institute of Technology, with data extraction permission provided (certification number: 46212703). The inclusion criteria included patients undergoing CRRT with modalities including slow continuous ultrafiltration (SCUF), continuous veno-venous hemofiltration (CVVH), continuous veno-venous hemodialysis (CVVHD), and continuous veno-venous hemodiafiltration (CVVHDF). The exclusion criteria were as follows: 1) patients aged younger than 18 years or older than 89 years; 2) patients with multiple hospital or ICU admissions; and 3) patients with a hospitalization duration of less than 48 hours. The MIMIC-IV-derived population constituted the training cohort, which was subsequently divided into a training set (80%) and an internal validation set (20%) via 5-fold cross-validation.

External Validation Cohort

The external validation cohort was sourced from the Department of Critical Care Medicine at the Fourth Hospital of Hebei Medical University. We conducted a systematic review of patients admitted between January 2020 and December 2021 who received CRRT and who adhered to the same inclusion and exclusion criteria as the development cohort did. Ethical approval for the study was obtained from the Ethics Committee of the Fourth Hospital of Hebei Medical University (Ethics Approval Number: 2021KS042).

Outcome

The outcome of this study is 28-day mortality among patients receiving CRRT.

Data Extraction

We utilized structured query language (SQL) for data extraction via Navicat software. The data extracted from the MIMIC database included the following: 1. Patient Demographics: Gender, age, height, and weight. 2. Admission diagnoses: Presence of chronic kidney disease (CKD), chronic heart failure (CHF), acute myocardial infarction (AMI), atrial fibrillation (AF), chronic liver failure (CLF), chronic obstructive pulmonary disease (COPD), chronic coronary syndrome (CCS), hypertension, diabetes mellitus, stroke, malignant neoplasm, sepsis, septic shock, cardiac arrest, AKI, and acute pancreatitis. 3. Vital signs: Heart rate, blood pressure, respiratory rate, body temperature, and oxygen saturation were measured 24 hours prior to the initiation of CRRT. 4. Laboratory tests: Blood routine, biochemistry, coagulation function, and blood gas analysis indicators were obtained 24 hours before CRRT. 5. Therapeutic Measures: Vasoactive drugs, mechanical ventilation, analgesics, sedatives, anticoagulants, diuretics, and β-blockers were used 24 hours before CRRT. 6. Criticality scores: Sequential Organ Failure Assessment (SOFA) score and Acute Physiology Evaluation III (APIII) score.

Data Handling

The initial step involved handling missing values. Variables with more than 30% missing data, such as albumin and fibrinogen, were excluded. For variables with less than 30% missing data, we employed the K-nearest neighbor (KNN) method for interpolation.¹⁶ Subsequently, outliers were addressed via the capping method, where values below the 1st percentile were set to the 1st percentile and values above the 99th percentile were set to the 99th percentile. Finally, we assessed and excluded variables with zero or near-zero variance and examined multicollinearity among the remaining variables.

Statistical Analysis

All the statistical analyses were conducted via R software (version 4.3.1). Continuous variables are presented as medians and interquartile ranges (IQRs) and were compared via the Mann‒Whitney U-test. Categorical variables are expressed as percentages (%) and were compared via either the chi-square test or Fisher’s exact test, as appropriate.

Feature Engineering

Feature engineering is a crucial step in model construction. We employed three complementary ML algorithms to filter variables: (1) least absolute shrinkage and selection operator (LASSO) regression for linear sparsity and model simplification through coefficient compression;¹⁷ (2) support vector machine recursive feature elimination (SVM-RFE) to capture nonlinear relationships through iterative elimination of the least important features;¹⁸ and (3) the Boruta algorithm, named after the Slavic tree method, which establishes statistical significance by comparing real features against randomly generated shadow variables.¹⁹

This multi-method approach addresses different aspects of feature selection: LASSO’s L1 regularization optimizes linear modeling, SVM-RFE’s recursive ranking preserves nonlinear interactions, while Boruta provides robustness against random noise. The intersection of variables selected by all three methods (eg, age, RDW, lactate) demonstrated both clinical relevance and statistical consistency with existing literature. We further validated the robustness of this feature set through Venn diagram visualization, which confirmed the stability of selected predictors across methodologies. This strategy enhanced reliability while minimizing the risks of overfitting (via LASSO), missing complex patterns (via SVM-RFE), or overlooking subtle but significant variables (via Boruta).

Model Construction and Comparison

After feature engineering, the most suitable variables for modeling were selected. Nine ML algorithms were utilized to construct the model, including the Gaussian process (GP), gradient boosting machine (GBM), eXtreme gradient boosting (XGBoost), multilayer perceptron (MLP), adaptive boosting (AdaBoost), k-nearest neighbors (KNN), support vector machine (SVM), logistic regression (LR), and naive Bayes (NB) algorithms. To mitigate overfitting, grid search and 10-fold cross-validation were employed to tune the hyperparameters of each ML model. The predictive performance of these models was compared with that of the SOFA and APIII scores, which are commonly used clinical indicators of criticality. The model with the highest discriminative ability was chosen on the basis of the area under the receiver operating characteristic (ROC) curve (AUC) in the internal validation set. The clinical utility of the models was assessed via decision curve analysis (DCA). Additional metrics, such as accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), recall, and F1 score, were also summarized. The same metrics were applied to evaluate the model in the external validation cohort.

Machine Learning Explainable Tool

While ML algorithms have enhanced predictive capabilities, their lack of interpretability, often termed the “black box” feature, has constrained their generalizability to clinical settings. The SHAP method, which is grounded in game theory,¹² addresses this limitation by providing interpretability for ML models. SHAP not only ranks the importance of each feature globally but also visualizes the impact of each feature on individual predictions via force plots. Consequently, SHAP significantly bolsters confidence in the ML models.

Model Visualization

For ease of clinical application, the final model will be presented in the form of a web-based calculator.

Results

A total of 1362 patients were included in the final analysis, comprising 1224 patients in the training cohort and 138 patients in the external validation cohort. The patient enrollment flowchart is presented in Figure 1.

Figure 1.

Study protocol flowchart.

Abbreviations: CRRT, Continuous Renal Replacement Therapy; ICU, intensive care unit; MIMIC, Medical Information Mart for Intensive Care.

Baseline Characteristics

According to the univariate analyses (Table 1), significant differences were observed between the survival and non-survival groups across several variables. Patients in the non-survival group were slightly older and presented a greater prevalence of chronic kidney disease (CKD), type 2 diabetes, sepsis, septic shock, cardiac arrest, and AKI. Additionally, there were differences in vital signs, including mean arterial pressure (MAP), respiratory rate, and temperature. Laboratory indices, such as lactate, pH, oxygenation index, platelet count, white blood cell count, international normalized ratio (INR), prothrombin time (PT), partial thromboplastin time (PTT), aspartate aminotransferase (AST), and total bilirubin, also differed significantly. Furthermore, the SOFA score and APIII score were significantly greater in the non-survival group than in the survival group.

Table 1.

Characteristics of Patients in the Training Set at Twenty-Four hours Prior to Initiation of CRRT

Variables	Survival (N=571)	Non-Survival (N=409)	p
Male, n (%)	352 (61.6)	249 (60.9)	0.86
Age (year)	63.22 (51.67, 71.86)	65.58 (53.56, 76.52)	0.004
Weight (kg)	86.51 (72.72, 103.50)	88.30 (73.40, 102.00)	0.921
Height (cm)	170.00 (163.36, 178.00)	170.00 (163.00, 177.91)	0.497
CKD, n (%)	308 (53.9)	153 (37.4)	<0.001
CHF, n (%)	111 (19.4)	100 (24.4)	0.071
AMI, n (%)	131 (22.9)	100 (24.4)	0.637
AF, n (%)	170 (29.8)	118 (28.9)	0.809
CLF, n (%)	74 (13)	70 (17.1)	0.085
COPD, n (%)	80 (14)	72 (17.6)	0.149
CCS, n (%)	160 (28)	117 (28.6)	0.898
Hypertension, n (%)	50 (8.8)	51 (12.5)	0.075
Diabetes_2, n (%)	224 (39.2)	133 (32.5)	0.037
Stroke, n (%)	46 (8.1)	40 (9.8)	0.409
Malignant_tumor, n (%)	61 (10.7)	60 (14.7)	0.076
Sepsis, n (%)	395 (69.2)	236 (57.7)	<0.001
Septic_Shock, n (%)	186 (32.6)	215 (52.6)	<0.001
Cardiac_arrest, n (%)	40 (7)	49 (12)	0.01
AKI, n (%)	456 (79.9)	367 (89.7)	<0.001
Acute_Pancreatitis, n (%)	42 (7.4)	31 (7.6)	0.993
HR_min (bpm)	77.00 (66.00, 89.50)	77.00 (66.00, 89.00)	0.922
HR_max (bpm)	104.00 (90.00, 119.50)	106.00 (93.00, 124.00)	0.073
MAP_min (mmHg)	58.00 (51.00, 65.00)	55.00 (48.00, 62.00)	<0.001
MAP_max (mmHg)	95.00 (83.00, 108.00)	93.00 (82.00, 107.00)	0.143
RR_min (cpm)	13.50 (10.00, 17.00)	14.00 (10.00, 18.00)	0.621
RR_max (cpm)	28.00 (24.00, 32.75)	30.00 (26.00, 34.00)	<0.001
T_min (°C)	36.50 (36.11, 36.78)	36.44 (36.00, 36.78)	0.07
T_max (°C)	37.28 (36.89, 37.80)	37.33 (36.94, 37.83)	0.214
Lactate_max (mmol/L)	2.30 (1.50, 3.56)	3.09 (1.94, 6.00)	<0.001
PH_min	7.29 (7.21, 7.35)	7.25 (7.17, 7.32)	<0.001
PH_max	7.37 (7.31, 7.42)	7.35 (7.29, 7.40)	<0.001
PaO2_Min (mmHg)	80.00 (66.00, 99.00)	77.00 (62.00, 92.00)	0.003
Oxygenation index_Min (mmHg)	168.00 (120.00, 218.11)	151.67 (96.25, 200.00)	<0.001
BE_Min (mmol/L)	−6.00 (−10.00, −2.05)	−8.00 (−13.00, −3.86)	<0.001
BE_Max (mmol/L)	−2.00 (−6.00, 0.00)	−4.00 (−7.00, 0.00)	<0.001
Calcium_Min (mmol/L)	1.05 (0.99, 1.09)	1.05 (0.98, 1.10)	0.995
Calcium_Max (mmol/L)	1.12 (1.07, 1.16)	1.11 (1.06, 1.18)	0.626
Hematocrit_Min (%)	26.60 (23.45, 30.65)	26.20 (22.60, 30.00)	0.08
Hematocrit_Max (%)	28.90 (26.05, 33.35)	29.60 (25.80, 33.60)	0.778
Hemoglobin_Min (g/dL)	8.80 (7.80, 10.20)	8.60 (7.40, 10.00)	0.03
Hemoglobin_Max (g/dL)	9.40 (8.50, 11.05)	9.60 (8.30, 11.10)	0.908
Platelets_Min (K/UL)	138.00 (75.50, 209.00)	108.00 (56.00, 179.00)	<0.001
Platelets_Max (K/UL)	162.00 (99.00, 241.00)	136.00 (77.00, 217.00)	<0.001
WBC_Min (*10^9/L)	11.20 (7.25, 16.10)	12.60 (8.30, 18.30)	0.007
WBC_Max (*10^9/L)	13.30 (9.40, 19.70)	15.80 (10.50, 22.00)	<0.001
RDW_Min (%)	16.00 (14.60, 17.60)	16.40 (15.10, 19.00)	<0.001
RDW_Max (%)	16.30 (14.90, 17.95)	17.30 (15.50, 19.60)	<0.001
Aniongap_Min (mmol/L)	18.00 (15.00, 21.00)	18.00 (15.00, 21.00)	0.026
Aniongap_Max (mmol/L)	20.00 (17.00, 25.00)	22.00 (19.00, 26.00)	<0.001
Bicarbonate_Min (mmol/L)	19.00 (15.00, 23.00)	17.00 (13.00, 21.00)	<0.001
Bicarbonate_Max (mmol/L)	21.00 (18.00, 25.00)	20.00 (17.00, 24.00)	<0.001
BUN_Max (mg/dl)	59.00 (39.00, 86.00)	62.00 (40.00, 92.00)	0.334
Chloride_Min (mmol/L)	99.00 (94.00, 103.00)	99.00 (94.00, 104.00)	0.551
Chloride_Max (mmol/L)	101.00 (97.00, 106.00)	102.00 (97.00, 108.00)	0.059
Creatinine_Max (mmol/L)	4.70 (3.30, 6.55)	3.90 (2.80, 5.30)	<0.001
Glucose_Min (mmol/L)	110.00 (90.50, 140.00)	111.00 (86.00, 146.00)	0.87
Glucose_Max (mmol/L)	145.00 (116.00, 198.00)	159.00 (118.00, 220.00)	0.018
Sodium_Min (mmol/L)	136.00 (132.00, 139.00)	136.00 (131.00, 140.00)	0.475
Sodium_Max (mmol/L)	138.00 (134.00, 141.00)	139.00 (134.00, 143.00)	0.047
Potassium_Min (mmol/L)	4.30 (3.80, 4.90)	4.40 (3.80, 4.90)	0.256
Potassium_Max (mmol/L)	4.80 (4.20, 5.70)	4.90 (4.30, 5.80)	0.066
INR_Max	1.50 (1.21, 2.00)	1.88 (1.40, 2.70)	<0.001
PT_Max (seconds)	16.40 (13.70, 21.75)	20.10 (15.40, 29.30)	<0.001
PTT_Max (seconds)	39.90 (31.35, 54.01)	47.60 (35.00, 70.10)	<0.001
ALT_Max (U/L)	78.00 (25.00, 329.66)	96.00 (32.00, 326.00)	0.059
ALP_Max (U/L)	107.17 (73.50, 154.65)	108.00 (77.00, 162.00)	0.323
AST_Max (U/L)	147.00 (51.00, 628.42)	192.23 (69.00, 659.63)	0.018
Bilirubin_Total_Max (μmol/L)	2.00 (0.80, 4.65)	2.90 (1.10, 6.93)	<0.001
Vasopressor, n (%)	337 (59)	339 (82.9)	<0.001
Mechanical ventilation, n (%)	359 (62.9)	332 (81.2)	<0.001
Sedative use, n (%)	370 (64.8)	297 (72.6)	0.012
Analgesic use, n (%)	360 (63)	308 (75.3)	<0.001
Anticoagulant, n (%)	230 (40.3)	145 (35.5)	0.142
Diuretic, n (%)	167 (29.2)	159 (38.9)	0.002
β-blocker, n (%)	63 (11)	29 (7.1)	0.048
APIII	84.00 (65.00, 103.00)	97.00 (79.00, 117.00)	<0.001
SOFA	11.00 (9.00, 14.00)	14.00 (12.00, 16.00)	<0.001

Abbreviations: AF, atrial fibrillation; AKI, acute kidney injury; ALP, alkaline phosphatase; ALT, alanine aminotransferase; AMI, acute myocardial infarction; APIII, Acute Physiology Evaluation III; AST, aspartate aminotransferase; BE, Base Excess; BUN, blood urea nitrogen; CCS, chronic coronary syndromes; CHF, congestive heart failure; CKD, chronic kidney disease; CLF, chronic liver failure; COPD, chronic obstructive pulmonary disease; HR, heart rate; INR, international normalized ratio; PT, prothrombin time; PTT, partial thromboplastin time; RDW, red cell distribution width; SOFA, Sequential Organ Failure Assessment.

Feature Engineering

We employed ML methods for feature engineering and selected the intersection of the features identified by all three methods.

The first ML method used was LASSO regression. Figure 2A depicts the coefficient path diagram of LASSO regression for variable screening. Figure 2B illustrates the 10-fold cross-validation process employed to select the optimal hyperparameter (λ) in LASSO regression. The dotted line on the left side of Figure 2B represents the log(λ) value that minimizes the error, resulting in λ = 0.008449. Using this hyperparameter, a total of 40 features were identified as significant (see supplementary material-1). The second ML method utilized was SVM-RFE, which selected the top 30 most important features (see supplementary material-1). The third ML method was the Boruta algorithm, and its screening process is shown in Figure 2C. This method identified 22 significant features (see supplementary material-1).

Figure 2.

Feature Engineering. Lasso coefficient path map of 75 features (A); Lasso cross validation curve (B); Feature engineering based on Boruta algorithm (C); The Venn diagram shows the intersection of three feature engineering ML algorithms (D).

Abbreviations: AF, atrial fibrillation; AKI, acute kidney injury; ALP, alkaline phosphatase; ALT, alanine aminotransferase; AMI, acute myocardial infarction; APIII, Acute Physiology Evaluation III; AST, aspartate aminotransferase; BE, Base Excess; BUN, blood urea nitrogen; CCS, chronic coronary syndromes; CHF, congestive heart failure; CKD, chronic kidney disease; CLF, chronic liver failure; COPD, chronic obstructive pulmonary disease; HR, heart rate; INR, international normalized ratio; LASSO, Least Absolute Shrinkage and Selection Operator; MBP, mean blood pressure; MI, myocardial infarction; PaO₂/FiO₂, arterial oxygen partial pressure to fractional inspired oxygen ratio; PT, prothrombin time; PTT, partial thromboplastin time; RDW, red cell distribution width; RFE, Recursive Feature Elimination; RR, respiratory rate; SOFA, Sequential Organ Failure Assessment; WBC, white blood cell.

To further refine the features, we utilized a Venn diagram to summarize and identify the intersection of variables screened by the three methods (Figure 2D). The final 15 features selected were age, CKD, septic shock, vasopressor use, mechanical ventilation, maximum respiratory rate, maximum lactate level, pH, maximum red cell distribution width (RDW), creatinine level, INR, prothrombin time (PT), partial thromboplastin time (PTT), total bilirubin level, and minimum RDW. Clinically, multiple covariances were observed between the maximum and minimum RDW, as well as between the INR and PT. Across all three ML feature engineering methods, the maximum RDW was found to be more important than the minimum RDW was, and the PT was more significant than the INR was. Therefore, the INR and minimum RDW were excluded from the final set of features. Ultimately, 13 features were included in the subsequent modeling stage.

Model Performance Comparisons

We constructed models using nine ML methods and employed grid search to select the best hyperparameters (see Supplementary Table 1). The optimal model was chosen based on the AUC comparison among all models. Figure 3A and B show the ROC curves for the nine ML models and the two traditional criticality scores for the training set and the internal validation set, respectively.

Figure 3.

ROC curves for the training (A) and internal validation sets (B), and DCA curves for the training (C) and internal validation sets (D).

Abbreviations: AdaBoost, Adaptive Boosting; APIII, Acute Physiology Evaluation III; AUC, Area Under the Curve; GBM, Gradient Boosting Machine; GP, Gaussian Process; KNN, K-Nearest Neighbors; LR, Logistic Regression; MLP, Multilayer Perceptron; NB, Naive Bayes; SOFA, Sequential Organ Failure Assessment; SVM, Support Vector Machine; XGBoost, eXtreme Gradient Boosting.

The performance evaluation on the training set demonstrated varying predictive accuracy across different ML models and critical illness scoring systems. Specifically, the AUC values for each model were as follows: GP 0.841, GBM 0.825, XGBoost 0.823, MLP 0.823, AdaBoost 0.818, KNN 0.794, SVM 0.785, LR 0.761, and Naive Bayes 0.732. In comparison, the traditional severity scores SOFA and APSIII achieved AUC values of 0.706 and 0.629, respectively. The AUC results on the internal validation set showed the following performance for each model: GP 0.794, GBM 0.763, XGBoost 0.760, MLP 0.792, AdaBoost 0.770, KNN 0.697, SVM 0.784, LR 0.781, and Naive Bayes 0.750. In comparison, the traditional severity scores, SOFA and APSIII, yielded AUC values of 0.665 and 0.641, respectively.

The GP model exhibited the best performance in both the training and internal validation sets. Table 2 and Table 3 present the additional evaluation metrics for the nine ML models in the training set and the internal validation set, respectively. Overall, the GP model demonstrated the best predictive performance. Figure 3C and D display the DCA results of the nine models in the training set and the internal validation set. The GP model achieved the highest net clinical gain in the training set and showed a slight decrease in the internal validation set.

Table 2.

Performance Comparison of ML Models in the Training Set

Models	AUC (95% CI)	Brier	Accuracy (95% CI)	Sensitivity	Specificity	PPV	NPV	F1 Score
GP	0.841 (0.816, 0.866)	0.167	0.768 (0.741, 0.795)	0.655	0.849	0.757	0.775	0.702
GBM	0.825 (0.799, 0.851)	0.186	0.759 (0.731, 0.786)	0.543	0.914	0.819	0.736	0.653
XGBoost	0.823 (0.796, 0.849)	0.199	0.763 (0.735, 0.790)	0.599	0.881	0.783	0.754	0.679
MLP	0.823 (0.796, 0.849)	0.167	0.752 (0.724, 0.779)	0.665	0.814	0.720	0.772	0.691
AdaBoost	0.818 (0.791, 0.844)	0.173	0.744 (0.715, 0.771)	0.628	0.827	0.722	0.756	0.672
KNN	0.794 (0.767, 0.820)	0.181	0.720 (0.691, 0.748)	0.570	0.828	0.704	0.729	0.630
SVM	0.785 (0.757, 0.814)	0.186	0.727 (0.698, 0.754)	0.548	0.855	0.730	0.725	0.626
LR	0.761 (0.731, 0.791)	0.194	0.710 (0.681, 0.738)	0.582	0.802	0.678	0.728	0.626
NB	0.732 (0.701, 0.764)	0.238	0.679 (0.648, 0.708)	0.567	0.758	0.627	0.710	0.596

Abbreviations: AUC, Area under curve; AdaBoost, Adaptive Boosting; CI, Confidence interval; GBM, Gradient Boosting Machine; GP, Gaussian Process; KNN, K-Nearest Neighbor; LR, Logistic Regression; MLP, Multi-Layer Perceptron; NB, Naive Bayes; NPV, Negative Predictive Value; PPV, Positive Predictive Value; SVM, Support Vector Machine; XGBoost, eXtreme Gradient Boosting.

Table 3.

Performance Comparison of ML Models in the Internal Validation Set

Models	AUC (95% CI)	Brier	Accuracy (95% CI)	Sensitivity	Specificity	PPV	NPV	F1 Score
GP	0.794 (0.738, 0.850)	0.186	0.734 (0.674, 0.788)	0.600	0.835	0.733	0.734	0.660
GBM	0.763 (0.703, 0.822)	0.202	0.697 (0.635, 0.754)	0.438	0.892	0.754	0.678	0.554
XGBoost	0.760 (0.700, 0.820)	0.210	0.705 (0.643, 0.761)	0.476	0.878	0.746	0.689	0.581
MLP	0.792 (0.735, 0.850)	0.180	0.730 (0.669, 0.784)	0.648	0.791	0.701	0.748	0.673
AdaBoost	0.770 (0.711, 0.828)	0.191	0.697 (0.635, 0.754)	0.514	0.835	0.701	0.695	0.593
KNN	0.697 (0.631, 0.762)	0.218	0.660 (0.597, 0.719)	0.543	0.748	0.620	0.684	0.579
SVM	0.784 (0.726, 0.841)	0.187	0.717 (0.656, 0.773)	0.533	0.856	0.737	0.708	0.619
LR	0.781 (0.724, 0.839)	0.187	0.725 (0.665, 0.780)	0.600	0.820	0.716	0.731	0.653
NB	0.750 (0.689, 0.810)	0.231	0.701 (0.639, 0.758)	0.581	0.791	0.678	0.714	0.626

Abbreviations: AUC, Area under curve; AdaBoost, Adaptive Boosting; CI, Confidence interval; GBM, Gradient Boosting Machine; GP, Gaussian Process; KNN, K-Nearest Neighbor; LR, Logistic Regression; MLP, Multi-Layer Perceptron; NB, Naive Bayes; NPV, Negative Predictive Value; PPV, Positive Predictive Value; SVM, Support Vector Machine; XGBoost, eXtreme Gradient Boosting.

Interpretability Analysis

The GP model was initially analyzed for global interpretability. Figure 4B shows the ranking of variable importance in the GP model. The results revealed that RDW, age, lactate, the presence of septic shock, and the use of vasoactive drugs were the five most significant factors influencing 28-day all-cause mortality in patients on CRRT. In Figure 4A, the horizontal axis represents the SHAP values, which indicate each feature’s contribution to the model’s prediction. Positive SHAP values increase the prediction, whereas negative values decrease it. The dots represent individual sample SHAP values, with yellow indicating larger feature values and purple indicating smaller values. The rightward dots signify a positive impact, whereas the leftward dots signify a negative impact, highlighting the influential features and their effects on the predictions. Specifically, our findings indicate that patients with higher RDW, ages, lactate levels, presence of septic shock, and use of vasoactive drugs have higher mortality rates.

Figure 4.

Summary plots of GP model for predicting 28-day all-cause mortality (A); Feature-ranking plots of GP model for predicting 28-day all-cause mortality (B); SHAP force plots for a survivor (C) and a non-survivor (D).

Abbreviations: CKD, chronic kidney disease; CRRT, continuous renal replacement therapy; PT, prothrombin time; PTT, partial thromboplastin time; RDW, red cell distribution width; SHAP, SHapley Additive exPlanation.

Furthermore, the SHAP method can be utilized to provide localized explanations for individual patients. Figure 4C and D displays force plots illustrating the local explanations generated by the model via the SHAP method. The length of each arrow signifies the magnitude of a feature’s influence on the prediction, with longer arrows denoting a greater impact. E[f(x)], the baseline expectation value, denotes the average model prediction when no feature information is present. The actual prediction f(x) for a given sample is obtained by adjusting this baseline with the sum of the SHAP values corresponding to all the features of that sample. This relationship is mathematically represented as follows:

If E[f(x)] exceeds f(x), it suggests that the combined influence of the sample’s features has a negative impact, lowering the prediction below the average. In contrast, if f(x) is greater than E[f(x)], the features have a positive effect, elevating the prediction above the baseline. Figure 4C and D depict patients with predicted outcomes of death and survival, respectively.

External Validation

To further assess the generalizability of this model to other cohorts, we conducted an external validation using a nonoverlapping cohort of patients who received CRRT from June 1, 2022, to June 1, 2023, at the Department of Critical Care Medicine of the Fourth Hospital of Hebei Medical University. The baseline characteristics of this external validation cohort are presented in Table 4. Significant differences were observed between the survival and non-survival groups in terms of age, maximum prothrombin time, peak blood creatinine level, presence of septic shock, use of vasoactive drugs, and requirement for invasive mechanical ventilation. The external validation of the GP model yielded an AUC of 0.78. Figure 5 displays the ROC curves, DCA curves, and various evaluation metrics for the external validation of the GP model, all of which demonstrated good predictive performance.

Table 4.

Characteristics of Patients in the External Validation Cohort at Twenty-Four Hours Prior to Initiation of CRRT

Variables	Survival (N=61)	Non-Survival (N=77)	p
Age (year)	64.00 (47.00, 71.00)	70.00 (59.00, 75.00)	0.004
Septic_Shock, n (%)	30 (49.2%)	58 (75.3%)	0.003
CKD, n (%)	7 (11.5%)	6 (7.8%)	0.658
RR_max (cpm)	22.00 (19.00, 29.00)	25.00 (20.00, 30.00)	0.158
RDW_Max (%)	14.30 (13.70, 15.20)	14.30 (13.60, 15.90)	0.771
PT_Max (seconds)	13.10 (12.00, 15.50)	15.40 (13.00, 20.60)	<0.001
PTT_Max (seconds)	34.30 (30.40, 39.20)	36.50 (31.00, 44.90)	0.169
Lactate_max (mmol/L)	1.30 (1.10, 2.70)	3.50 (1.40, 8.10)	<0.001
PH_max	7.35 (7.31, 7.40)	7.31 (7.21, 7.38)	0.005
Bilirubin_Total_Max (μmol/L)	1.14 (0.51, 2.60)	1.24 (0.56, 2.16)	0.709
Creatinine_Max (mmol/L)	3.41 (2.36, 4.75)	2.47 (1.71, 3.37)	0.008
Vasopressor, n (%)	33 (54.1%)	70 (90.9%)	<0.001
Mechanical ventilation, n (%)	42 (68.9%)	71 (92.2%)	<0.001

Abbreviations: CKD, chronic kidney disease; RR, respiratory rate; PT, prothrombin time; PTT, partial thromboplastin time; RDW, red cell distribution width.

Figure 5.

ROC curve (A), DCA curve (B) and other evaluation indicators (C) of the external validation cohort.

Abbreviations: AUC, Area under curve; CI, Confidence interval; GP, Gaussian Process; NPV, Negative Predictive Value; PPV, Positive Predictive Value.

Model Visualization

To facilitate the clinical application of our model, we developed a web-based calculator that can be used to estimate patient prognosis. The calculator is available at the following website: [hbykdxdsyyicu.shinyapps.io/CRRT-mortality/].

Discussion

In this study, we developed nine ML models to predict 28-day all-cause mortality in patients undergoing CRRT and compared their performance to that of traditional criticality scores. The GP model demonstrated the highest predictive accuracy, with an optimal cutoff value for the prediction probability of 0.424. To enhance the interpretability of the GP model, we applied the SHAP method, which provided insights into the model’s decision-making process and facilitated the practical application of its predictions. We subsequently conducted external validation in a separate study cohort with no overlap in time or space, achieving superior validation results. Finally, we developed a web-based calculator for easy clinical application.

We applied three ML algorithms for feature engineering and identified the 13 most important features. All laboratory parameters included in our model (RDW, lactate, PTT, PT, pH, total bilirubin, and creatinine) represent routinely available clinical tests. Compared to novel biomarkers, these conventional indicators offer greater accessibility and practicality for clinical implementation. On the basis of the SHAP global interpretation, the most significant feature among these was the RDW. Higher RDW were associated with poorer patient prognosis. Previous studies have shown that a high RDW is correlated with high mortality in patients undergoing peritoneal dialysis and intermittent hemodialysis.^20,21 A severe inflammatory response in critically ill patients can lead to impaired erythropoiesis and shortened erythrocyte survival, resulting in altered red blood cell volume. A high RDW often indicates impaired intravascular hemodynamics.²² Additionally, RDW is an indicator of immune dysregulation, which has been associated with the prognosis of numerous diseases and confirmed by relevant studies.^23,24

According to the SHAP analysis, patients with high lactate levels, septic shock, and the need for vasoactive drugs had a higher mortality rate. Approximately one-third of patients with sepsis are estimated to develop AKI,²⁵ and sepsis-associated AKI is associated with a greater risk of death than other types of AKI.²⁶ Therefore, early identification of high-risk patients is crucial for improving their prognosis. With the advancement of CRRT technology, there are increasing nonrenal indications for its use, such as sepsis and septic shock. As early as the last century, the use of CRRT to remove inflammatory mediators, including cytokines and complement, was proposed to improve the prognosis of patients with sepsis and multiple organ dysfunction syndrome (MODS).²⁷ Recent study results suggest that CRRT is recommended for hemodynamically unstable patients with septic shock and left ventricular insufficiency, even in the absence of AKI.²⁸ Compared with surviving patients, non-survival patients presented longer PT and PTT, indicating more severe coagulation abnormalities in deceased individuals. Numerous previous studies have demonstrated a strong association between coagulation abnormalities and death from sepsis, pyrexia, and AKI.^29–31 Non-survival patients also had a greater proportion of invasive mechanical ventilation and a faster respiratory rate, suggesting concomitant respiratory failure. As the number of organs that fail increases, the mortality rate of patients also increases. Tachypnea has been identified as a risk factor for increased mortality, and a study conducted in the emergency department revealed that persistent shortness of breath is strongly associated with increased mortality in patients admitted to this setting.³² The increased mortality associated with tachypnea may be attributed to lung injury resulting from hyperventilation.³³ Our study also revealed that patients with lower serum creatinine levels at CRRT initiation exhibited higher mortality rates. This phenomenon may be associated with “Augmented Renal Clearance”—a pathophysiological state characterized by abnormally increased creatinine clearance and accelerated renal elimination of medications, which often indicates critical illness severity.³⁴

With recent advances in artificial intelligence technology, an increasing number of medical studies are applying ML techniques to diagnose diseases and predict mortality.³⁵ However, the complexity of ML algorithms and the difficulty in visualizing the contribution of each variable to the outcome have hindered their effective application in clinical practice by clinicians. The development of interpretable ML techniques has gradually addressed this “black box” issue of machine learning.³⁶ In this study, we utilized SHAP techniques to conduct global and local interpretable analyses of the model, investigated the prognostic significance of each risk factor, and illustrated the model’s application to individual patients for predicting the risk of death. This approach significantly enhances the model’s credibility and supports clinicians in making more informed decisions.

Although previous studies have explored the prognosis of patients undergoing CRRT, these studies have focused exclusively on patients with AKI receiving CRRT.^10,11 With the advancement of CRRT technology, the current indications for CRRT have expanded beyond renal diseases to include sepsis, acute pancreatitis, heart failure, and numerous other nonrenal conditions.

This study has several strengths. First, this study is the first to explore interpretable ML for predicting 28-day all-cause mortality in all patients undergoing CRRT, not limited to those with AKI. Second, we employed three ML algorithms for feature engineering to identify the most relevant features for inclusion in the model. Third, we utilized multiple ML algorithms to construct the model. Compared with traditional statistical methods, ML algorithms demonstrate superior predictive performance. To prevent overfitting, we implemented a grid search method for hyperparameter tuning. The model exhibited good prediction accuracy in both internal and external validations, ensuring its generalizability.

Limitation

First, as a real-world retrospective study, selection bias and missing values are inevitable, which may affect the model’s predictive performance. However, we applied the KNN algorithm to impute the missing values, thereby minimizing their impact on the model.

Second, all indicators included in this study are clinically traditional, and some novel biomarkers were not incorporated.³⁷

Third, this study was limited to pre-CRRT single-timepoint data without longitudinal follow-up, and the validation cohort only included populations from the US and China. Despite employing rigorous methodologies, the model’s complexity may still carry overfitting risks due to the limited external validation sample size. Future multinational, multiethnic prospective studies involving broader populations are needed to enhance the model’s generalizability across diverse healthcare systems and demographic characteristics.

Conclusion

In this study, we constructed multiple ML models and compared them from various perspectives, ultimately selecting the GP model as the optimal choice for predicting 28-day all-cause mortality in patients receiving CRRT. We analyzed the interpretability of the model via SHAP technology to increase its credibility. Additionally, to evaluate the model’s generalizability, we conducted external validation in an independent cohort with no temporal or spatial overlap, achieving satisfactory results. Furthermore, we developed a web-based calculator for convenient clinical application, with future potential for integration into electronic medical records. However, further prospective studies are needed to thoroughly investigate the model’s performance.

Funding Statement

This work was supported by the Medical Science Research Project of Hebei (Grant No. 20221260) and the S&T Program of Hebei, China (Grant No. 223777104D).

Ethics Approval and Consent to Participate

The study was conducted in accordance with the Declaration of Helsinki. The study was approved by the Ethics Committee of the Fourth Hospital of Hebei Medical University (No. 2021KS042). Written informed consent was obtained from all participants.

Disclosure

The authors report no conflicts of interest in this work.