Explainable machine learning model for predicting short-term outcomes in sepsis- induced coagulopathy

Abstract

Background

Sepsis-Induced coagulopathy (SIC) is not only a common complication in the development process of sepsis but also related to poor prognosis of sepsis. We aimed to establish a machine learning (ML) model to predict the 28-day mortality risk of patients with SIC.

Methods

We collected data for model training from the Medical Information Mart for Intensive Care IV Database version 2.2 to establish the model. We extracted patient data from the First Affiliated Hospital of Wenzhou Medical University for the model’s external validation. We used Least Absolute Shrinkage and Selection Operator (LASSO) regression and logistic regression analysis to identify predictive factors for a 28-day mortality risk. Then, we built prognostic prediction models for SIC patients using multiple ML classification models. We evaluated predictive performance using Receiver Operating Characteristic (ROC) curves, calibration curves, and Decision Curve Analysis (DCA). We used Shapley Additive Explanations (SHAP) to interpret the models.

Results

We selected seventeen variables for model development, and the XGBoost model performed the best. The area under the curve (AUC) (95% CI) of the test set reached 0.840 (0.810–0.870), with an accuracy of 0.807, sensitivity of 0.836, and specificity of 0.798. The model also demonstrated excellent predictive performance in external validation, with an AUC (95% CI) of 0.864 (0.794–0.934).

Conclusion

We constructed an XGBoost model and provided model interpretability using the SHAP. This model provides a basis for assessing the 28-day mortality risk of patients with SIC, aiding in clinical decision support and the formulation of personalized treatment strategies.

Introduction

Sepsis is a severe systemic inflammatory response to infection that, if not promptly identified or treated, can rapidly progress to septic shock and multiple organ dysfunction, often leading to death [1]. Statistics show that sepsis causes 5.3 million deaths annually, with an overall mortality rate of approximately 30%, and an intensive care unit (ICU) mortality rate of 41.9% [1–3]. Research has found that 50% to 70% of sepsis patients develop coagulation dysfunction [4]. Coagulation dysfunction associated with sepsis is not only a common complication in the progression of sepsis, but also correlates with poor sepsis prognosis [5, 6]. Therefore, the evaluation and prediction of prognosis in SIC patients are crucial for early stratification and accurate treatment.

Recent studies have demonstrated the significant impact of coagulation dysfunction on the severity and prognosis of sepsis, primarily evident in two key aspects: 1. Exacerbation of inflammatory response: In sepsis, there is a close interaction between the coagulation and inflammatory systems. The development of systemic inflammatory response syndrome (SIRS) promotes the release of inflammatory mediators, which further activate the coagulation pathway and damage endothelial cells, intensifying the inflammatory response [2, 7]. Microcirculatory impairments: Coagulation acts as a defense mechanism, bacterial pathogens are constrained within the fibrin network at the site of infection, limiting their spread to adjacent tissues and systemic circulation. While fibrinolysis-induced damage is somewhat beneficial, but it also hampers the delivery of oxygen and nutrients, exacerbating tissue hypoxia, consequently leading to or exacerbating multiple organ dysfunction syndrome (MODS) [8].

The Scientific and Standardization Committee of the 20th International Society on Thrombosis and Hemostasis (ISTH) proposed the diagnostic criteria for SIC based on the Sepsis-3 guidelines, aimed at identifying sepsis-related coagulation abnormalities [9]. The SIC scoring system comprises three components: (1) prolonged prothrombin time (PT) or an elevated international normalized ratio (INR); (2) platelet count; (3) sequential organ failure assessment (SOFA) score. Patients with a SIC score of 4 or higher, following the exclusion of other causes of coagulation abnormalities, can be diagnosed with SIC [9]. The SIC score represents the first scoring system specifically tailored for coagulation disturbances in sepsis, exhibiting increased sensitivity in detecting sepsis-induced coagulation abnormalities compared to other disseminated intravascular coagulation (DIC) scoring system [10]. Since the introduction of the SIC score, studies from Asia have reported an incidence rate of 40% to 60% based on Sepsis 3.0 criteria, although European data is not yet available [9, 11–13]. Although the SIC score can serve as an early tool to identify coagulopathy, its independent prognostic value for mortality and disease progression remains limited, primarily due to the small evidence base, the static nature of the scoring system, the lack of incorporation of endothelial injury markers, and heterogeneity in clinical interventions. Further prospective, multicenter studies are needed to validate its predictive utility [14]. One study found that the SIC score is an independent risk factor for in-hospital mortality in septic shock patients [5]. A secondary analysis from a two-center randomized controlled trial in Germany reported a 90-day mortality rate for SIC ranging from 26.8% to 53.3% [6]. Research by Iba T. and colleagues revealed that the 28-day mortality rate for patients with a SIC score of 4 was around 30%, but this escalated dramatically to over 45% for patients with a score of 6 or higher [13]. Studies comparing the SIC score with other commonly used prognostic assessment tools like the SOFA score and the Acute Physiology and Chronic Health Evaluation II (APACHE II) score have shown that the SIC score has the smallest area under the ROC curve (0.658), the highest sensitivity (74.3%), and the lowest specificity (34.1%) [15].

Given the high incidence and mortality rates of SIC, developing a reliable predictive model is essential. Currently, prognostic models for sepsis-induced coagulation dysfunction are receiving extensive attention and research. A predictive model for the 28-day prognosis of SIC patients was developed using traditional logistic regression analysis based on the Medical Information Mart for Intensive Care III database (MIMIC-III). On one hand, logistic regression is mainly used to deal with linear relationships, and its performance may not be as good as some machine learning(ML) models when the relationship between the dependent and independent variables is nonlinear. However, this model lacked external validation, making it less reliable [16]. Zhou et al. developed a predictive model for SIC using the XGBoost algorithm based on the MIMIC-III, MIMIC-IV, and eICU Collaborative Research Databases, with good predictive performance. Nevertheless, this study focused primarily on Western patients and did not include a validation cohort of Chinese patients, limiting the generalizability of the model [17].

This study aims to develop an accurate predictive model based on the MIMIC-IV database and validate it within our ICU to predict the 28-day mortality risk of SIC patients in the ICU. We used a variety of ML algorithms and SHAP for model interpretability, helping clinicians better understand the reasons behind specific predictions and offering individualized treatment plans.

Materials and methods

Study subjects

The study utilized data from the MIMIC-IV 2.2, an open-access critical care database containing comprehensive clinical data for patients admitted to Beth Israel Deaconess Medical Center between 2008 and 2019. The database includes demographic information, vital signs, laboratory results, and hourly physiological data validated by ICU nurses. We extracted data of patients with SIC entering the ICU within 24 h from the MIMIC-IV database as the training and validation sets. An author (Chenglong Liang) was approved to extract data from the database for research purposes (Certification No. 58898095). This database was approved by the Institutional Review Boards (IRB) of the Massachusetts Institute of Technology (MIT).

According to the Sepsis-3 definition, sepsis is characterized by infection-induced organ dysfunction caused by a dysregulated host response, with organ dysfunction being defined as an increase of 2 or more points in the SOFA score due to infection. The definition of SIC was proposed by the DIC Scientific and Standards Committee and the International Society on Thrombosis and Hemostasis (ISTH) in 2017. It classifies patients with sepsis and coagulation disorders, defined as infection-induced organ failure combined with coagulation dysfunction. The diagnostic criteria for SIC include platelet count, PT-INR, and SOFA score, with a total score of ≥ 4. The study excluded participants based on the following criteria: (1) Patients under 18 years of age; (2) Pregnant patients; (3) Long-term anticoagulant use or congenital coagulation disorders; (4) 4. Patients with coagulation dysfunction due to chemotherapy; (5) Patients who died or were discharged within 24 h after ICU admission.

The data from these patients served as training data for model development. The First Affiliated Hospital of Wenzhou Medical University collected external validation data from 2020 to 2023. The data extraction process is shown in Fig. 1. The hospital provided external validation sets from 2020 to 2023. The data extraction procedure is illustrated in Fig. 1.

Fig. 1.

Flowchart of screening

Data extraction

We used structured query language (SQL) to obtain the required data fields, and we relied on Navicat Premium software as the database management interface to execute SQL queries, handle multi-table joins, and export the raw datasets.

We collected the following variables: (1) Demographic data: age, gender, ICU stay duration, and admission/discharge times from ICU (documented in-hospital and non-in-hospital deaths). (2) Average vital signs within 24 h of entering ICU: temperature, mean arterial pressure, heart rate, systolic and diastolic blood pressure, respiratory rate, and oxygen saturation (SpO₂). (3) Laboratory Parameters: the worst values of complete blood count, liver function, renal function, arterial blood gases, and coagulation indicators within 24 h after entering the ICU. (4) Advanced Life Support Records: The team recorded mechanical ventilation, renal replacement therapy, SOFA scores, medications, and comorbidities.

To address missing data, we performed data visualization using R version 4.3.2 with the vim package. We excluded variables with over 30% missing observations, such as D-dimer, total bilirubin, direct bilirubin, and indirect bilirubin, to ensure the accuracy of the study. We imputed the remaining missing values using multiple imputations with chained equations (via the mice package).

Statistical analysis and methods

We used the Mann-Whitney U test to compare continuous variables and the chi-square test for categorical variables. Patients were categorized into “survival” and “death” groups, according to their survival status within 28 days. We randomly divided the dataset into a training set and a testing set. After selecting features, we applied multiple ML classification models to comprehensively analyze and compare the importance of different indicators in both sets. We evaluated and validated the results using the optimal model. Additionally, we developed an overall SHAP model and provided single-sample explanations. The specific steps included: (1) Data Partitioning: We used Python (scikit-learn 0.22.1) to randomly allocate the data extracted from MIMIC-IV 2.2, assigning 60% of patients to the training set (n = 2,797), 20% to the validation set (n = 700), and 20% to the test set (n = 875). (2) Feature Selection: Least Absolute Shrinkage and Selection Operator (LASSO) regression analysis was conducted using R software (glmnet) to prevent overfitting and severe collinearity issues, resulting in the selection of significant variables. Subsequently, multiple logistic regression analysis was conducted using SPSS version 26 to further control confounding factors and identify variables with p < 0.05.(3) Comprehensive Analysis of Multiple cCassification Models: This study utilized Python(sklearn version 0.22.1, XGBoost version 1.2.1, LightGBM version 3.2.1) to develop and validate nine ML models on the training set, including XGBoost, Logistic Regression, LightGBM, Random Forest, AdaBoost, Multi-layer Perceptron Classifier (MLP), Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Complement Naive Bayes (GNB). We evaluated the models using Area Under the Curve (AUC) values, Decision Curve Analysis (DCA), and compared accuracy, sensitivity, specificity, F1 score, Positive Predictive Value (PPV), and Negative Predictive Value (NPV). We built and validated the final predictive model on the testing set, including the corresponding calculation of the AUC curve. (4) Training, Validation, and Testing of the Optimal Predictive Model: We trained XGBoost as the baseline model and then optimized it using hyperparameters.We determined the XGBoost model’s hyperparameters using GridSearchCV with ten-fold cross-validation on the training set. We evaluated the model on the testing set and plotted learning curves using scikit-learn version 0.22.1. We also assessed the model’s reliability and practicality using external validation. (5) Explanation of ML Predictive Models: We used the Python package SHAP version 0.430 to demonstrate both the overall interpretability of the model and single-sample explanations.

Results

We included 4,505 SIC patients, with 4,372 allocated to the training data set (2,797 in the training set, 700 in the validation set, and 875 in the testing set) and 133 assigned to the external validation cohort. We categorized patients into the Survival and Death groups based on their 28-day ICU survival status. The mortality rates were 26.1% based on the MIMIC-IV database and 27.0% through the external validation using FAHWMU.

Baseline characteristics

Table 1 shows the overall baseline characteristics, vital signs, and laboratory parameters of the training data. In the univariate analysis, we considered factors such as continuous renal replacement therapy (CRRT), total carbon dioxide (TCO₂), PO2, hydrogen ion concentration (pH), anion gap (AG), glucose, total calcium, potassium, sodium, hematocrit, red cell distribution width (RDW), hemoglobin, platelet count, white blood cell count (WBC), red blood cell count (RBC), comorbidities like Type 2 Diabetes, heart failure, myocardial infarction, hyperlipidemia, acute renal failure, and vital signs like respiratory rate (RR), diastolic blood pressure (DBP), systolic blood pressure (SBP), and heart rate (HR) as significant between the two groups.

Table 1.

Basic characteristics of population

Variable	Survival, N = 3,230	Death, N = 1,142	p
Age, years	72.000(61.000–83.000)	68.000(56.000–80.000)	< 0.001
SOFA	6.000(4.000–9.000)	9.000(6.000–12.000)	< 0.001
ICU stay time	3.470(1.980–7.098)	5.530(3.183–9.853)	< 0.001
Creatinine	1.400(0.900–2.400)	1.600(1.000–2.700)	< 0.001
Urea nitrogen	32.000(20.000–52.000)	32.000(21.000–53.000)	0.402
INR	1.400(1.200–1.700)	1.500(1.200–2.200)	< 0.001
PTT	35.000(29.500–50.000)	39.150(30.400–63.525)	< 0.001
PT	15.300(13.400–18.700)	16.300(13.500–23.100)	< 0.001
Totalco2	27.000(23.000–30.000)	25.000(21.000–28.000)	< 0.001
Lactate	2.100(1.500–3.500)	3.100(1.800–5.900)	< 0.001
PO2	66.000(42.000–95.000)	61.000(42.000–87.000)	< 0.001
PCO2	45.000(39.000–54.000)	46.000 (39.000–54.000)	0.943
PH	7.330(7.260–7.390)	7.280(7.180–7.370)	< 0.001
Anion gap	16.000(13.000–20.000)	18.000(15.000–22.000)	< 0.001
Glucose	152.000(120.000–206.000)	169.000(130.000–235.750)	< 0.001
Chloride	107.000 (102.000–111.000)	107.000(101.250–112.000)	0.423
Calcium total	7.900(7.400–8.500)	7.800(7.200–8.400)	< 0.001
Potassium	4.500(4.100–5.000)	4.600(4.100–5.200)	< 0.001
Sodium	137.000 (134.000–140.000)	137.000(134.000–141.000)	0.046
Hematocrit	28.000(23.800–32.900)	29.300 (24.700–35.100)	< 0.001
RDW	15.700(14.400–17.675)	15.400(14.200–17.400)	< 0.001
Hemoglobin	9.200(7.900–10.800)	9.700 (8.100–11.500)	< 0.001
Platelet count	149.500(96.000–216.000)	143.500(80.000–211.000)	0.008
WBC	13.200 (9.200–18.875)	15.300(10.500–21.000)	< 0.001
RBC	13.200(9.200–18.875)	3.150(2.630–3.840)	< 0.001
Temperature	36.810 (36.560–37.128)	36.800(36.470–37.218)	0.665
RR	29.000(25.000–33.000)	29.000(25.000–33.000)	< 0.001
SpO2	92.000(89.000–94.000)	92.000(87.000–94.750)	0.078
MBP	55.000(48.000–63.000)	56.000(48.000–65.000)	0.162
DBP	43.000(36.000–50.000)	45.000(36.000–52.000)	0.003
SBP	88.000(78.000–99.000)	87.000(76.000–98.000)	0.021
HR	106.000(92.000–122.000)	110.000 (95.250–125.000)	< 0.001
Ventilation, n (%)			0.271
NO	657 (20)	215 (19)
YES	2,573 (80)	927 (81)
CRRT, n (%)			< 0.001
NO	3,192 (99)	1,104 (97)
YES	38 (1.2)	38 (3.3)
Hypertension, n (%)			0.803
NO	1,999 (62)	702 (61)
YES	1,231 (38)	440 (39)
Type 2 diabetes, n (%)			< 0.001
NO	2,087 (65)	846 (74)
YES	1,143 (35)	296 (26)
Type 1 diabetes, n (%)			0.499
NO	3,166 (98)	1,123 (98)
YES	64 (2.0)	19 (1.7)
Heart failure, n (%)			< 0.001
NO	2,066 (64)	795 (70)
YES	1,164 (36)	347 (30)
Myocardial infarction, n (%)			< 0.001
NO	3,005 (93)	1,003 (88)
YES	225 (7.0)	139 (12)
Liver cirrhosis, n (%)			0.144
NO	2,697 (83)	932 (82)
YES	533 (17)	210 (18)
Hepatitis, n (%)			0.123
NO	3,024 (94)	1,054 (92)
YES	206 (6.4)	88 (7.7)
Hyperlipemia, n (%)			0.005
NO	2,230 (69)	839 (73)
YES	1,000 (31)	303 (27)
Chronic bronchitis, n (%)			0.923
NO	2,953 (91)	1,043 (91)
YES	277 (8.6)	99 (8.7)
Chronic nephrosis, n (%)			< 0.001
NO	2,443 (76)	926 (81)
YES	787 (24)	216 (19)
Acute renal failure, n (%)			< 0.001
NO	1,566 (48)	398 (35)
YES	1,664 (52)	744 (65)
Gender, n (%)			0.454
NO	1,413 (44)	485 (42)
YES	1,817 (56)	657 (58)
Vasopressor, n (%)			< 0.001
NO	1,791 (55)	176 (15)
YES	1,439 (45)	966 (85)
Cardiotonic, n (%)			< 0.001
NO	3,010 (93)	885 (77)
YES	220 (6.8)	257 (23)

Note: Categorical data is represented as frequencies (percentages) and compared using the chi-square test. Non-normally distributed variables are reported as median (interquartile range) and compared using the Kruska Wallis test

Abbreviations: WBC: White blood cell count, RBC: Red blood cell count., RDW: Red cell distribution width, INR: International normalized ratio, PT: Prothrombin time, PTT: Partial thromboplastin time, BUN: Blood urea nitrogen, PO₂: Oxygen partial pressure, PCO₂: Carbon dioxide partial pressure, PH: Acidity-alkalinity, SpO₂: Oxygen saturation, RR: Respiratory rate, HR: Heart rate, SBP: Systolic blood pressure, DBP: Diastolic blood pressure, MBP: Mean arterial pressure, SOFA: Sequential organ failure assessment score, CRRT: Continuous renal replacement therapy

Selection of risk factors for 28-day mortality in SIC patients

The present study aimed to utilize the 28-day survival status in the ICU as the dependent variable and conducted LASSO regression analysis on the remaining independent variables. LASSO (Least Absolute Shrinkage and Selection Operator) is a data mining technique that incorporates penalty functions into traditional multiple regression models to continuously shrink coefficients, thereby addressing issues of multicollinearity and overfitting. As shown in Fig. 2, the initial 48 independent variables were reduced to 22, including Type 2 diabetes. To further control for confounding factors, we performed a multiple logistic regression analysis on the 22 independent variables mentioned above, identifying significant variables with p < 0.05. Ultimately, we included age, SOFA, vasopressor, cardiotonic, ICU stay time, creatinine, PT, TCO₂, lactate, AG, sodium, RDW, platelet count, SBP, Type 2 diabetes, heart failure, and acute renal failure in the predictive model (see Table 2).

Fig. 2.

Using the LASSO regression analysis method to select feature factors. (A) Use 10-fold cross-validation to draw a vertical line at the selected value, where the optimal lambda produces 22 non-zero coefficients. (B) In the LASSO model, the coefficient distribution of 48 texture features is plotted from the log (λ) sequence. Draw a vertical dashed line at the minimum mean square error (λ = 0.002) and the standard error of the minimum distance (λ = 0.011)

Table 2.

Multivariate logistic regression analysis

Variables	OR	95%CI	P
Age	0.989	0.980–0.998	< 0.001
SOFA	1.135	1.104–1.167	< 0.001
ICU stay time	0.972	0.962–0.982	< 0.001
Creatinine	1.002	1.000-1.004	< 0.001
PT	1.011	1.004–1.019	0.004
TotalCO2	0.971	0.957–0.984	< 0.001
Lactate	1.038	1.001–1.076	0.043
Anion gap	1.028	1.005–1.051	0.015
Sodium	1.021	1.008–1.035	0.002
RDW	0.947	0.916–0.979	0.001
Hematocrit	1.025	0.980–1.072	0.288
Hemoglobin	1.05	0.914–1.205	0.488
Platelet count	1.002	1.001–1.003	< 0.001
MBP	1.005	0.990–1.021	0.540
DBP	1.001	0.988–1.015	0.846
SBP	1.007	1.000-1.014	0.040
Type 2 diabetes	1.002	0.989–1.016	< 0.001
Heart failure	1.012	1.005–1.020	0.027
Myocardial infarction	1.255	0.958–1.639	0.097
Acute renal failure	1.213	1.018–1.446	0.031
Vasopressor	4.672	3.804–5.759	< 0.001
Cardiotonic	1.939	1.525–2.465	< 0.001

Abbreviations: OR: Odds ratio, SOFA: Sequential organ failure assessment score, PT: Prothrombin time, RDW: Red cell distribution width, SBP: Systolic blood pressure, DBP: Diastolic blood pressure, MBP: Mean arterial pressure

Comprehensive analysis of multi-model classification

In this study, we developed and validated nine ML models, including XGBoost, Logistic Regression, LightGBM (LGBM Classifier), Random Forest, AdaBoost, MLP, SVM, KNN, and GNB. We evaluated the models using AUC values. The results showed that in the training set, Random Forest had the highest AUC (95% CI) of 1, performing the best, followed by XGBoost with an AUC (95% CI) of 0.921 (0.912–0.930). In the internal validation set, XGBoost demonstrated a slightly higher AUC compared to other models, with an AUC (95% CI) of 0.823 (0.780–0.867) (Fig. 3). The sensitivity, specificity, and accuracy of XGBoost were 0.782, 0.732, and 0.758, respectively (Table 3), whereas Random forest exhibited signs of overfitting, highlighting the stability of XGBoost. An analysis of the nine ML models was conducted using DCA, calibration curves, and PR curves The DCA results of the nine predictive models (Fig. 3) indicated that both XGBoost and Random forest had significantly higher net benefits compared to the other models, showcasing better clinical applicability. The calibration curves demonstrated high predictive accuracy for XGBoost and Logistic Regression models (Fig. 3). Overall, the comprehensive analysis suggested that XGBoost could be considered the optimal model.

Fig. 3.

Comprehensive analysis of ML models. (A) ROC curves and AUCs in the training set. (B) ROC curves and AUCs in the validation set. (C) Decision curve analysis (DCA) in the validation set. The red dashed line assumes survival of all SIC patients within 28 days, and the black dashed line assumes death of all patients. Solid lines indicate different models. (D) Calibration curves in the validation set. The diagonal dashed line represents the reference. The x-axis shows mean predicted probabilities, and the y-axis shows observed event probabilities. The closer the fitted lines are to the reference, the smaller the values in parentheses, indicating greater predictive accuracy. Colors represent different models, with values shown as mean and 95% CI

Table 3.

Comprehensive analysis of multi-model classification

Multi-Model Classificatin	AUC	Cutoff	Accuracy	Sensitivity	Specificity
XGBoost	0.823	0.339	0.758	0.782	0.732
logistic	0.757	0.256	0.699	0.66	0.758
Light GBM	0.634	0.632	0.742	0.624	0.73
Random forest	0.823	0.531	0.8	0.76	0.742
AdaBoost	0.813	0.487	0.74	0.808	0.698
KNN	0.620	0.4	0.723	0.633	0.559
SVM	0.493	0.327	0.6	0.367	0.705
MLP	0.752	0.261	0.683	0.69	0.708
CNB	0.644	0.9	0.688	0.492	0.763

Abbreviations: Light GBM: LGBM Classifier, MLP: Multilayer perceptron, SVM: Support Vector Machine, KNN: K-Nearest Neighbors, GNB: Complement NB

Construction and evaluation of the optimal model

XGBoost is used as the baseline model, and the training cohort is subjected to ten-fold cross-validation with optimized hyperparameters (learning_rate = 0.3, reg_lambda = 0.5, max_depth = 4, min_child_weight = 4). The results show that the AUC (95% CI) of the training set is 0.902 (0.896–0.918), the AUC (95% CI) of the validation set is 0.820 (0.772–0.868), and the AUC (95% CI) of the test set is 0.840 (0.810–0.870). The AUC of the training, validation, and test sets eventually stabilize around 0.85, indicating an accurate model prediction. As the performance of the validation set in terms of the AUC metric is lower than the test set or the ratio is lower than 10%, it can be considered that the model fits successfully. Furthermore, the learning curve indicates that the training set and validation set have strong fitting and high stability (Fig. 4). These results suggest that the XGBoost model can be used for classification modeling tasks on the dataset.

Fig. 4.

Training, validation, and testing of the XGBoost model. (A) Training set ROC and AUC, (B) Validation set ROC and AUC. Perform ten-fold cross-validation on the training dataset. Different colors of solid lines represent ten different outcomes. (C) Testing set ROC and AUC, (D) Learning curve. The red dashed line represents the training set, the blue dashed line represents the validation set, displaying the net benefit curve of the predictive model. The X-axis represents the threshold probability of 28-day mortality risk, and the Y-axis represents net benefit, these values are presented as averages and with a 95% CI

External validation of the final model

We performed validation using the external validation cohort from the First Affiliated Hospital of Wenzhou Medical University, covering the years 2020 to 2023. The AUC (95% CI) of the final model for external validation was 0.864 (0.794–0.934) (Fig. 5), which is consistent with the results of the internal validation. This suggests that the final model demonstrated robust performance in both internal and external validations.

Fig. 5.

(A) External validation set ROC and AUC, (B) External validation set DCA

Explanatory analysis

We applied the SHAP method to interpret the final predictive model. We estimated feature importance using the entire training set and ranked features by their SHAP values to better illustrate their relationships with model predictions. Because this study included the worst values of vital signs and laboratory parameters upon ICU admission, the top six risk factors associated with 28-day mortality in SIC patients were vasopressor use, ICU length of stay, RDW, lower SBP, higher SOFA score, and prolonged PT, all of which contributed to increased mortality risk. Figure 6A shows the feature ranking based on mean absolute SHAP values, identifying vasopressor use, ICU stay, RDW, SBP, SOFA score, and PT as the six most influential predictors.

Fig. 6.

(A) SHAP-ranked feature importance. The matrix plot illustrates the relative importance of each covariate in developing the final predictive model. (B) Force plot visualizing individual model predictions as the cumulative contribution of features. The baseline represents the average prediction. Feature values and names are listed at the bottom of the plot. Bold numbers denote the predicted probability (f(x)), whereas the baseline corresponds to the prediction without model inputs. f(x) represents the log-odds ratio for each observation. Red features indicate increased mortality risk, while blue features indicate reduced risk. Arrow length intuitively reflects the magnitude of feature influence, with longer arrows representing stronger effects

In addition, Fig. 6B illustrates how the SHAP method explains individual model predictions. The force plot begins at the baseline value, representing the average of all predictions. Each predictor (with its corresponding Shapley value) is shown as an arrow that increases (red) or decreases (blue) the prediction relative to the baseline. The arrow length indicates the predictor’s importance, with longer arrows representing stronger effects. Feature values are listed at the bottom of the plot. The model’s final output is represented at the intersection of the red and blue arrows.

Discussion

We developed and validated nine ML models for SIC patient outcome prediction. XGBoost demonstrated more stable AUCs across validation and test sets, with reduced overfitting, making it a robust choice for clinical prediction. XGBoost is a tree-based ensemble ML algorithm known for its high computational efficiency and ability to handle various types of data (including numerical, categorical data), making it suitable for dealing with the complexity of medical data [18]. Additionally, XGBoost’s advantages include its ability to handle heterogeneous medical data and complex nonlinear relationships. Importantly, it offers built-in feature importance metrics, which were found to be largely consistent with SHAP-derived rankings. This dual approach to interpretability supports the identification of key risk factors and potential biomarkers, enhancing clinical applicability [19].

We used the SHAP method to explain the prediction results of the XGBoost model, revealing how each feature influenced the model’s predictions. The study correlates with the use of vasoactive drugs with the 28-day mortality risk in SIC patients, followed by RDW values. Reflecting tissue perfusion parameters (SBP, total carbon dioxide, AG, blood lactate levels) assists in evaluating the short-term mortality risk of SIC patients. Septic shock patients often receive vasoactive drug treatment, revealing a hierarchy of severity in septic patients [20]. Most studies demonstrate that the SIC score is a significant prognostic indicator for sepsis-associated coagulopathy, yet there is limited research on the relationship between SIC and the prognosis of septic shock. This study indicates that patients with septic shock experiencing coagulation abnormalities have unfavorable short-term prognoses, aligning with previous research findings. In a multicenter observational database study in Japan, some observed that the SIC score holds good prognostic value and can identify lethal coagulopathy in septic patients requiring vasopressors. Among septic patients receiving vasoactive drugs, the incidence of SIC was 66.4%, while in those not requiring vasoactive drug treatment, the incidence was 42.2%. It suggests that in septic patients utilizing vasoactive drugs, the risk of SIC-related mortality significantly increases (HR = 1.39, 95%CI: 1.13–1.70, P < 0.01) [11]. In a European study, the incidence of SIC and the necessity for vasoactive drugs were calculated, with Julie Helms et al. reporting a positive SIC score in 84.2% of patients [21]. Simultaneously, in a Chinese study assessing the prognostic value of SIC in septic patients, Zhu Weimin et al. found that the use of vasoactive drugs was an independent factor affecting 28-day mortality in septic patients (HR = 3.66, 95%CI: 1.53–8.75, P < 0.05) [22].

The RDW value is a parameter in standard blood tests that reflects the heterogeneity of red blood cell sizes. It has been established as a significant predictor for various diseases, including sepsis [23–25], acute pancreatitis [26], and acute kidney injury [27]. High RDW values in sepsis patients have been linked to systemic inflammation and are considered an independent risk factor for unfavorable outcomes in sepsis [28]. A meta-analysis incorporating 11 studies by Helms J et al. reported a correlation between increased RDW values and sepsis mortality rate (HR = 1.14, 95% CI 1.09–1.20, Z = 5.78, P < 0.01) [21]. Yi W. Fan and colleagues highlighted that RDW-SD and RDW increase rate (RDW fluctuation during hospitalization) serve as strong predictors for the incidence of sepsis-associated disseminated intravascular coagulation (DIC) [28]. Regarding the relationship between RDW growth rate, DIC incidence rate, and 28-day mortality rate, RDW shows a sustained increase of approximately 6% [25]. It is currently believed that during sepsis onset, the host’s inflammatory response and oxidative stress excessively activate coagulation factors, leading to widespread microthrombi in the capillaries [29]. Consequently, the excessive consumption of coagulation factors and impairment in the fibrinolytic system may contribute to the development of sepsis-associated DIC. Nonetheless, research exploring the relationship between the critical RDW value, sepsis-related coagulopathy incidence, and prognosis is limited and warrants further investigation.

Tissue perfusion refers to the circulation and exchange of blood within tissues, which is essential for maintaining organ function and cellular metabolism. Inadequate tissue perfusion, particularly in septic shock, can lead to tissue ischemia, hypoxia, ultimately resulting in organ dysfunction or even organ failure [20]. Patients with sepsis-associated coagulopathy often present with extensive microvascular damage and abnormal coagulation function, affecting the efficiency of tissue perfusion. Parameters such as SBP, TCO₂, AG, and lactate levels can be utilized for the indirect assessment of tissue perfusion efficiency and the severity of a patient’s condition. SBP serves as a crucial parameter for assessing a patient’s hemodynamic status, with low SBP potentially indicating low blood volume or impaired cardiac pumping function, both of which can result in inadequate tissue perfusion. A reduction in TCO₂ levels may suggest metabolic acidosis, often correlated with elevated blood lactate levels, reflecting tissue hypoxia. Total carbon dioxide can mirror a patient’s acid-base balance status, with abnormalities potentially indicating inadequate tissue perfusion. For instance, an elevation in blood lactate leading to lactic acidosis may cause a decrease in TCO₂ levels. An increased AG may signify lactic acidosis induced by hypoxia or inadequate tissue perfusion, leading to an escalated mortality rate [30]. Under conditions of tissue hypoxia, cellular metabolism shifts towards anaerobic glycolysis, resulting in increased lactate production. Therefore, elevated blood lactate levels usually signify inadequate tissue perfusion and severe hypoxia. Blood lactate serves as an independent prognostic factor for septic patients, with a threshold of 16% identified as the optimal value for prognostic analysis of sepsis [25]. Early monitoring of lactate concentration in septic patients can lower mortality rates [31]. Thus, understanding and monitoring these tissue perfusion indicators carry significant importance in evaluating the severity and prognosis of patients with SIC. Timely recognition and improvement of inadequate tissue perfusion in SIC patients are crucial for enhancing treatment success and reducing the risk of mortality.

Advantages and limitations

Compared to previous studies, this research project has several significant advantages. First, we selected the training cohort from the MIMIC-IV database, with patients mainly from Western countries, and conducted external validation with patients from the First Affiliated Hospital of Wenzhou Medical University. This demonstrated the model’s satisfactory performance in both internal and external validation cohorts. Second, the study utilized multiple ML models for predictive analysis, which allowed for a comprehensive evaluation of the strengths and weaknesses of different models, enhancing prediction accuracy and reliability. Third, the use of the SHAP method to explain the model could reveal the extent of each feature’s impact on the prediction outcomes, aiding in understanding the model’s prediction process and results.

However, the research project still has significant limitations. First, though multiple imputation was applied to handle missing data, deleting variables due to over 30% data loss might lead to information loss, affecting the integrity of the analysis results. Second, despite the internal and external validations, the model in the project mainly relied on one database (MIMIC-IV) and a specific external validation cohort, limiting the generalization ability of the model, requiring future validation in broader populations and regions.Third, the database was not comprehensively studied and lacked some key variables, such as heparin treatment, and the study only collected the worst laboratory indicators and the average vital signs within 24 h of patients’ early admission to the ICU, lacking continuous dynamic assessment, which could lead to selection bias. We further excluded patients who died or were discharged within 24 h of admission to ensure complete 24-h baseline data; this exclusion inevitably removed the most unstable cases and may have led to underestimation of early mortality risk. Finally, some models in the project (e.g., Random forest) exhibited signs of overfitting, indicating the models might be overly sensitive to the training data, necessitating focus on improving the model’s generalization performance.

Conclusion

This study successfully established a model to predict the risk of death in SIC patients in the ICU within 28 days. The model not only demonstrated good predictive performance but also provided deeper insights into clinical decision-making through its explanatory feature analysis. This can help clinical doctors to understand which factors have a greater impact on patient prognosis, potentially guiding more accurate treatment strategies and resource allocation.

Abbreviations

AG

Anion gap

AUC

Area under the curve

CRRT

Continuous renal replacement therapy

DCA

Decision curve analysis

DIC

Disseminated intravascular coagulation

DBP

Diastolic blood pressure

GNB

ComplementNB

HR

Heart rate

ICU

Intensive care unit

ISTH

International society on thrombosis and hemostasis

KNN

K-nearest neighbors

LASSO

Least absolute shrinkage and selection operator

LightGBM

LGBM classifier

MLP

Multilayer perceptron

NPV

Negative predictive value

PCO2

Carbon dioxide partial pressure

PH

Acidity-alkalinity

PPV

Positive predictive value

PT

Prothrombin time

RBC

Red blood cell

RDW

Red cell distribution width

ROC

Receiver operating characteristic

RR

Respiratory rate

SBP

systolic blood pressure

SIC

Sepsis-induced coagulopathy

SIRS

Systemic inflammatory response syndrome

SOFA

Sequential organ failure assessment

SPO2

Oxygen saturation

SVM

Support Vector Machine

TCO2

Total carbon dioxide

MIMIC

Medical information mart for intensive care

ML

Machine learning

SHAP

Shapley additive explanations

WBC

White blood cell count

Author contributions

Jingye Pan: conceptualization, methodology, Jinmei Wu, Xianwei Zhang: data curation, writing-original draft preparation, Chenglong Liang, Baoxin Wang: visualization, investigation, Xiangyuan Ruan, Yihau Dong, Xueyang Xu: writing-reviewing and editing.

Funding

National Natural Science Foundation of China (Grant No.82272204). Key Clinical Specialty of Zhejiang Province (Critical Care Medicine, Y2022). “Pioneer” and “Leading Goose” R&D Program of Zhejiang (2023C03084). Wenzhou major science and technology innovation project (ZY2023005).

Data availability

The datasets during and/or analysed during the current study available from the corresponding author on reasonable request.

Declarations

Ethical approval and consent to participate

All procedures performed in the present study were in accordance with the principles outlined in the 1964 Helsinki Declaration and its later amendments.The establishment of MIMIC-IV was approved by the institutional review boards of the Beth Israel Deaconess Medical Center (Boston, MA) and Massachusetts Institute of Technology (Cambridge, MA), thus, this study was granted a waiver of informed consent. The data of external validation cohort were de-identified, and informed consent was not required. This study was reviewed and approved by the Ethics Committee in Clinical Research of the First Affiliated Hospital of Wenzhou Medicine University (KY2023-R061).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.