Development of Machine Learning–Based Models to Predict Mortality in Intensive Care Unit Patients with Sepsis

Abstract

Objective

This study aimed to develop and validate a predictive model using machine learning to estimate mortality among intensive care unit patients.

Methods

The medical records of 874 sepsis patients hospitalized at the Affiliated Hospital of Chengde Medical University from 2021 to 2024 were retrospectively analyzed. Sepsis patients were randomly divided into training and validation sets in a 7:3 ratio. We constructed mortality prediction models for sepsis patients using machine learning algorithms, including extreme gradient boosting (XGBoost), logistic regression (LR), random forest (RF), adaptive boosting (AdaBoost), k-nearest neighbors (KNN), support vector machine (SVM), multilayer perceptron (MLP), and Gaussian naive Bayes (GNB). The predictive performance of the machine learning models was assessed using receiver operating characteristic (ROC) curves, calibration curves, and decision curve analysis (DCA). This study was reported in accordance with the RECORD (REporting of studies Conducted using Observational Routinely collected health Data) guidelines.

Results

A total of 874 patients with sepsis were included, of whom 338 died and 536 survived. Significant differences were observed between the mortality and survival groups with respect to platelet count (PLT), platelet distribution width (PDW), platelet distribution width to count ratio (PCR), mean platelet volume (MPV), monocyte count (Mono), albumin (Alb) level, total bilirubin (Tbil), alanine aminotransferase (ALT), aspartate aminotransferase (AST), lactate dehydrogenase (LDH), serum creatinine (Scr), blood urea nitrogen (BUN), fibrinogen (Fib) level, D-dimer level, lactate (Lac) level, respiratory system infection, gastrointestinal system infection, and APACHEII score. The ROC curve analysis showed that the areas under the curve (AUC) of the XGBoost, LR, RF, AdaBoost, KNN, SVM, MLP and GNB models in predicting the in-hospital mortality rate of sepsis patients were 0.954, 0.880, 0.951, 0.901, 0.859, 0.910, 0.836 and 0.854, respectively. Among the nine algorithms, the XGboost model performed significantly better than the others, achieving an accuracy of 0.879, a sensitivity of 0.734, and an F1 score of 0.821. The calibration curve demonstrated that the XGBoost model showed the best performance among the eight algorithms, with predictions closely matching the observed outcomes. Decision curve analysis indicated that the XGBoost model outperformed both extreme strategies.

Conclusion

Machine learning models provide a reliable approach for predicting in-hospital mortality in patients with sepsis. Among them, the XGboost model demonstrated the highest predictive performance, aiding clinicians in identifying high-risk patients with sepsis and implementing early interventions to reduce mortality.

Introduction

Sepsis is an inflammatory syndrome caused by a dysregulated host response to infection, which can result in life-threatening multiple organ dysfunction.¹ Despite notable advances in antibiotic management and medical technology, sepsis continues to have a high incidence, and its mortality rate remains unacceptably high. In the United States, sepsis is the leading cause of death among hospitalized patients, with annual medical costs exceeding tens of billions of dollars, making it a persistent public health challenge.²

In recent years, machine learning has emerged as a powerful tool in medicine for the analysis of large-scale data. With the development of critical illness databases and advances in machine learning, researchers have increasingly proposed models to predict disease prognosis and complications.^3–5 Although existing studies report promising results, these prediction models often lack sufficient external validation and real-world assessment to support clinical application. This study aimed to develop machine learning models to predict mortality risk in ICU patients with sepsis using clinical data from a Chinese cohort, and to identify the model with the highest predictive performance. Accordingly, this study is better tailored to the healthcare settings of developing countries, as it employs readily available admission data and early laboratory indicators to predict mortality risk in patients with sepsis, thereby improving its clinical utility for physicians in China.

Materials and Methods

Patients and Collected Variables

A total of 874 patients who met the Sepsis-3.0 criteria and were admitted to the ICU of the Affiliated Hospital of Chengde Medical University between 2021 and April 2024. Each cohort was classified into survival and non-survival groups according to in-hospital mortality outcomes. All patients received management in accordance with international guidelines. In this study, patient screening was conducted according to the Sepsis-3 criteria established in 2026 to identify individuals with sepsis. Subsequent organ dysfunction was defined as an acute increase of ≥2 points in the SOFA score within the first 24 hours of ICU admission. We initially selected patients who met either of the following two criteria within 24 hours after ICU admission: (a) a diagnosis of sepsis based on the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3); (b) age ≥18 years; (c) patients admitted to the ICU with complete clinical data. The exclusion criteria were as follows: (1) presence of autoimmune diseases; (2) diagnosis of malignant tumors; (3) pregnancy. A flowchart of the study is shown in Figure 1. After applying the criteria, 874 eligible sepsis patients were included. The study was approved by the Ethics Committee of the Affiliated Hospital of Chengde Medical University (Approval number: CYFYLL2023401). The need for written informed consent was waived because of the retrospective nature of the study.

Figure 1.

Flowchart of sepsis selection.

Clinical Data Collection

The clinical data of all enrolled ICU patients with sepsis were obtained from hospital medical records within the first 24 hours: (1) sex and age; (2) common comorbidities, including hypertension, cerebrovascular disease (CVD), diabetes, coronary artery disease (CAD) and chronic pulmonary disease; (3) site of infections, including respiratory, gastrointestinal, genitourinary, hepatobiliary, skin and soft tissue and other site; (4) biochemical indicators on admission, including C-reactive protein (CRP), white blood cell count (WBC), hemoglobin, platelet count (PLT), platelet distribution width (PDW), platelet distribution width to count ratio (PCR), mean platelet volume (MPV), neutrophil count (Neu), lymphocyte count (Lym), monocyte count (Mono), fibrinogen (Fib) level, albumin (Alb) level, D-dimer, total bilirubin (Tbil), alanine aminotransferase (ALT), aspartate aminotransferase (AST), lactate dehydrogenase (LDH), serum creatinine (Scr), blood urea nitrogen (BUN), procalcitonin (PCT) and lactate (Lac) levels; (5) A standardized severity scoring system, including the APACHE II score, was utilized to assess disease severity at ICU admission, based on routinely collected physiolo gical and laboratory variables.

Statistical Methods

All data were analyzed using R Studio (version 4.2.3) and python (version 3.11.4). Sepsis patients were randomly assigned to training and testing sets in a 7:3 ratio. Normally distributed continuous variables were presented as mean ± SD and compared between groups using Student’s t-test. Continuous variables with skewed distributions were expressed as medians and interquartile ranges and compared using the Mann–Whitney test. Categorical variables were expressed as percentages and compared using the chi-square test or Fisher’s exact test, as appropriate.

This study utilized multiple machine learning algorithms to analyze the data, including XGBoost, logistic regression (LR), random forest (RF), AdaBoost, K-nearest neighbors (KNN), support vector machine (SVM), multilayer perceptron (MLP), and Gaussian Naive Bayes (GNB). The predictive performance of each model was assessed using the area under the receiver operating characteristic curve (AUC). Additionally, decision curve analysis (DCA) was performed to evaluate the clinical utility of the models by quantifying their net benefit across a range of threshold probabilities.

Results

Patient Characteristics

A total of 874 adult patients diagnosed with sepsis were included in the final cohort, following predefined inclusion and exclusion criteria. The patients were randomly divided into a training set (n = 611) and a validation set (n = 263) in a 7:3 ratio. Statistically significant differences were observed in PLT, PDW, PCR, MPV, Mono, Alb, Tbil, ALT, AST, LDH, Scr, BUN, Fib, D-dimer level, lactate level, respiratory system infection, gastrointestinal system infection, and APACHEII score between the death group and the survival group (all P < 0.05). Detailed baseline data for both sets are provided in Table 1.

Table 1.

Baseline Characteristics of Patients with Sepsis Between in Training and Validation Sets

Variables	Training Set (n=611)			Testing Set (n=263)			P-value Among Sets
	Survival group (n=372)	Death group (n=239)	P-value	Survival group (n=164)	Death group (n=99)	P-value
Age(year)	66 (57, 74)	69 (58, 77)	0.050	69 (57, 77)	69 (61, 76)	0.671	0.184
Sex, n (%)			0.071			0.088	0.638
Female	177 (47.58)	112 (46.86)		79 (48.17)	37 (37.37)
Male	195 (52.42)	127 (53.14)		85 (51.83)	62 (62.63)
CRP (mg/L)	138.42 (80.67, 212.52)	154.27 (72.73, 211.77)	0.707	122.14 (65.16, 194.91)	125.72 (56.85, 210.43)	0.924	0.350
WBC (×109/L)	11.25 (6.99, 15.92)	11.29 (6.21, 17.41)	0.693	11.69 (7.67, 17.40)	11.50 (7.33, 17.08)	0.054	0.935
Hemoglobin (g/L)	106.00 (90.00, 122.00)	104.00 (86.00, 122.00)	0.577	104.00 (91.00, 124.00)	103.00 (84.00, 124.00)	0.509	0.774
PLT (×109/L)	102.00 (74.00, 144.00)	58.00 (34.00, 110.00)	< 0.001	96.00 (73.00, 152.00)	42.00 (26.00, 68.00)	< 0.001	0.452
PDW (%)	13.30 (11.20, 16.50)	15.00 (12.50, 17.90)	< 0.001	13.90 (11.50, 16.20)	15.60 (12.40, 17.70)	< 0.001	0.574
PCR (%)	13.51 (8.33, 21.24)	28.75 (11.45, 48.52)	< 0.001	13.19 (8.85, 20.36)	26.90 (11.18, 51.29)	< 0.001	0.689
MPV(fL)	10.80 (10.10, 12.00)	13.10 (11.30, 14.10)	< 0.001	10.90 (10.10, 11.60)	12.90 (10.90, 13.80)	< 0.001	0.171
Neu (%)	89.20 (83.30, 92.60)	88.80 (79.80, 92.60)	0.415	89.40 (83.34, 92.70)	89.20 (84.10, 92.70)	0.941	0.762
Lym (%)	5.50(3.30, 9.20)	6.10 (3.50, 13.00)	0.060	5.50(3.40, 9.10)	5.80 (3.50, 10.00)	0.573	0.698
Mono(%)	4.10 (2.64, 6.20)	3.20 (2.10, 5.80)	0.005	4.20 (2.50, 6.10)	3.30 (2.20, 5.30)	0.031	0.838
Alb (g/L)	26.80 ± 5.89	24.91 ± 5.30	< 0.001	26.89 ± 5.90	25.53 ± 5.82	< 0.001	0.420
Tail(µmol/L)	23.11 (12.50, 36.44)	34.58 (20.50, 58.41)	< 0.001	23.21 (14.07, 35.10)	34.15 (20.32, 74.07)	< 0.001	0.981
ALT (U/L)	32.00 (20.00, 74.00)	59.00 (29.00, 151.00)	< 0.001	34.50 (22.00, 79.00)	65.00 (31.53, 233.00)	< 0.001	0.110
AST (U/L)	47.72 (28.27, 85.00)	115.86 (48.29, 287.16)	< 0.001	54.99 (30.72, 89.77)	127.98 (49.79, 469.54)	< 0.001	0.257
LDH (U/L)	291.00 (222.75, 418.74)	566.40 (328.21, 1200.00)	< 0.001	305.00 (227.00, 404.18)	538.74 (285.47, 1249.47)	< 0.001	0.731
Scr (µmol/L)	139.27 (113.70, 178.04)	219.00 (149.87, 301.00)	< 0.001	137.21 (109.70, 171.12)	196.10 (145.50, 297.20)	< 0.001	0.466
BUN (mmol/L)	12.09 (9.04, 15.35)	17.90 (11.21, 27.53)	< 0.001	11.64 (9.03, 15.91)	16.17 (11.47, 25.99)	< 0.001	0.858
Fib (g/L)	4.80 (2.98, 5.77)	5.33 (3.01, 6.94)	< 0.001	4.63 (2.92, 5.70)	5.51 (2.82, 7.03)	< 0.001	0.202
D_dimer (µg/mL)	5.36 (2.43, 8.96)	13.98 (5.90, 23.99)	< 0.001	4.70 (1.92, 7.88)	12.06 (5.65, 21.92)	< 0.001	0.159
PCT (ng/mL)	17.44 (3.77, 50.41)	17.71 (3.82, 64.91)	0.270	16.93 (4.35, 49.27)	17.29 (3.28, 63.03)	0.095	0.853
Lac (mmol/L)	3.50 (2.60, 4.70)	7.50 (3.69, 12.30)	< 0.001	3.40 (2.50, 4.78)	7.60 (3.99, 11.60)	< 0.001	0.491
Site of infection
Respiratory, n (%)	103 (27.69)	116 (48.54)	< 0.001	39 (23.78)	39 (39.39)	0.025	0.384
Gastrointestinal, n (%)	118 (31.72)	37 (15.48)	< 0.001	41 (25.00)	13 (13.13)	0.005	0.355
Genitourinary, n (%)	52 (13.98)	30 (12.55)	0.614	20 (12.20)	11 (11.11)	0.495	0.844
Hepatobiliary, n (%)	36 (9.68)	22 (9.205)	0.240	36 (21.95)	20 (20.20)	0.737	0.170
Skin, n (%)	32 (8.58)	18 (7.53)	0.516	4 (2.44)	2 (2.02)	0.089	0.320
Other, n (%)	31 (8.33)	20 (8.37)	0.189	24 (14.63)	15 (15.15)	0.108	0.634
Comorbidity
Hypertension, n (%)	137 (36.83)	76 (31.80)	0.203	71 (43.29)	37 (37.37)	0.344	0.081
CAD, n (%)	79 (21.24)	53 (22.18)	0.301	30 (18.29)	23 (23.23)	0.333	0.454
Diabetes, n (%)	94 (25.27)	55 (23.01)	0.526	41 (25.00)	24 (24.24)	0.890	0.917
CVD, n (%)	65 (17.47)	50 (20.92)	0.287	40 (24.39)	24 (24.24)	0.978	0.064
COPD, n (%)	16 (4.30)	13 (5.44)	0.518	3 (1.83)	3 (3.03)	0.527	0.088
Scoring system
APACHE II score	21.00 (18.00, 27.00)	27.00 (22.00, 33.00)	< 0.001	22.00 (18.00, 28.00)	28.00 (22.00, 35.00)	< 0.001	0.635

Comparison of Predictive Efficacy Among Machine Learning Models

Receiver operating characteristic (ROC) curve analysis was conducted to evaluate the predictive efficacy of nine machine learning models—including XGBoost, LR, RF, AdaBoost, KNN, SVM, MLP and GNB—for in-hospital mortality in sepsis patients. The ROC curve analysis shows that among the nine models, the XGboost algorithm demonstrates the best predictive performance for in-hospital mortality in sepsis patients (AUC = 0.954), followed by the LR (AUC = 0.880), RF (AUC = 0.951), AdaBoost (AUC = 0.901), KNN (AUC = 0.859), SVM (AUC = 0.910), MLP (AUC = 0.836) and GNB (AUC = 0.854), as shown in Figure 2.

Figure 2.

The ROC curves of 8 machine learning models for predicting the mortality rate of sepsis patients.

Comparison of Predictive Efficacy Among Machine Learning Models

A multi-model comparison showed that XGBoost performed best. The cut-off value, sensitivity, specificity, accuracy, positive predictive value, negative predictive value, F1 score, and kappa value were 0.915, 0.734, 0.970, 0.879, 0.944, 0.855, 0.821 and 0.733 in the validation set, as shown in Table 2. In addition, The calibration curve illustrates the accuracy of each model’s predictions; the closer the fitted line is to the reference line, the smaller the value in brackets, as shown in Figure 3. The clinical decision curve demonstrates the net benefit of each model, as shown in Figure 4.

Table 2.

Multi-Model Classification Results in the Validation Set

Model	AUC (%)	Cut off	Sensitivity (%)	Specificity (%)	Accuracy	Positive Predictive Value	Negative Predictive Value	F1 Score	Kappa
XGBoost	0.954	0.915	0.734	0.970	0.879	0.944	0.855	0.821	0.733
LR	0.880	0.420	0.763	0.858	0.822	0.777	0.852	0.768	0.623
RF	0.951	0.490	0.858	0.927	0.900	0.885	0.914	0.869	0.789
AdaBoost	0.901	0.497	0.802	0.845	0.828	0.769	0.873	0.783	0.641
KNN	0.859	0.400	0.769	0.791	0.783	0.698	0.847	0.731	0.549
SVM	0.910	0.417	0.799	0.882	0.850	0.825	0.878	0.805	0.684
MLP	0.836	0.351	0.701	0.836	0.784	0.736	0.815	0.716	0.542
GNB	0.854	0.039	0.728	0.867	0.813	0.779	0.836	0.750	0.602

Figure 3.

Calibration curves of eight machine learning models for predicting mortality in sepsis patients.

Figure 4.

The DCA of eight machine learning models for predicting the mortality rate of sepsis patients.

The Importance of Features in the XGBoost Model

In the XGBoost model, the top 10 features include lactate level, PCR, Scr, APACHEII score, D-dimer level, LDH, Fib, MPV, Alb, and respiratory system infection, as shown in Figure 5.

Figure 5.

The importance of features in the XGboost model.

Discussion

According to the Sepsis-3 definition, sepsis emphasizes the strong association between infection and life-threatening organ dysfunction.^6,7 Therefore, early identification and prognostic assessment of sepsis are crucial, as they provide clinicians with valuable information to evaluate patient status and to initiate timely and appropriate treatment, ultimately improving survival outcomes.^8,9 This study applied machine learning methods to clinical data collected within the first 24 hours of ICU admission and demonstrated their ability to predict in-hospital mortality in patients with sepsis. In accordance with the RECORD guidelines, this study used routinely collected de-identified clinical data and reported all key study design and analysis details to ensure reproducibility. Validation results confirmed that the XGBoost model exhibited the highest predictive performance among all tested models, with an AUC of 0.954 in the validation set.

In recent years, artificial intelligence has been increasingly used to predict mortality risk across a wide range of diseases, including sepsis. Machine learning, a subset of artificial intelligence, has been widely applied in the clinical diagnosis, prognostic assessment, and precision treatment of sepsis.^10,11 It has shown significant advantages over traditional scoring systems commonly used for critically ill patients. Extreme Gradient Boosting (XGBoost), a decision tree–based ensemble algorithm, has gained widespread recognition for its high predictive accuracy and computational efficiency. It has consistently achieved top performance in machine learning competitions on platforms such as Kaggle and has been increasingly applied in clinical research, particularly for risk stratification and outcome prediction.^12,13

In this study, multiple machine learning models were developed, with the XGboost model outperforming the other eight models in predicting in-hospital mortality in sepsis patients. The XGboost model developed in this study identified the top five most important features, which include lactate level, PCR, serum creatinine, APACHE IIscore and D-dimer. Lactate level has traditionally been regarded as a marker of tissue hypoxia and is commonly used to assess disease severity, monitor treatment response, and predict prognosis.^14,15 The platelet count-to-distribution width ratio (PCR), which combines two readily available platelet indices—platelet distribution width (PDW) and platelet count (PLT)—has been proposed as a prognostic biomarker in sepsis. Previous studies have demonstrated its utility in assessing disease severity in pediatric sepsis and in predicting clinical outcomes.¹⁶ Continuous renal replacement therapy (CRRT) is commonly required in patients with sepsis-associated acute kidney injury (AKI). In our model, serum creatinine emerged as the most influential variable, as it reflects renal function and helps assess the severity of sepsis. Studies have demonstrated that higher APACHE IIscore is associated with increased hospital-acquired infection rates and in-hospital mortality in patients with sepsis, indicating that this score has prognostic value.^17,18 In addition, sepsis is frequently complicated by disseminated intravascular coagulation (DIC). This condition is marked by widespread platelet activation, resulting in microvascular thrombus formation, elevated D-dimer levels, and reduced fibrinogen concentrations.^19,20

This study has several limitations. First, This study is a single-center investigation utilizing data from one of China’s top-tier hospitals. No external database validation was performed, and the model’s performance warrants further verification through prospective studies. Second, due to significant regional variations in ICU settings, healthcare resources, and laboratory testing capabilities across China, the findings of this study may be more applicable to sepsis patients admitted to ICUs within the country. Finally, due to the limitations in data types available from the single-center database, certain imaging and ultrasound indicators related to infection or sepsis were not included in the model. Moving forward, we plan to integrate the existing model with local multidisciplinary resources to enhance both its performance and clinical applicability.

Conclusion

Machine learning models provide a reliable and accurate approach for predicting in-hospital mortality in ICU patients with sepsis. The XGBoost model developed in this study demonstrated the highest predictive performance, with an AUC of 0.954 in the validation set, and identified Lac level, PCR, Scr, APACHEII score, and D-dimer level as the key predictive factors. This study was reported in accordance with the RECORD guidelines, and the XGBoost model is based on readily available clinical indicators, making it a practical clinical decision support tool for identifying high-risk sepsis patients and implementing early targeted interventions to reduce mortality. Future multi-center prospective studies are needed to further validate the model’s performance and generalizability.

Funding Statement

The study was supported by the Medical Science Research Project of Hebei (Approval number: 20241528).

Data Sharing Statement

The authors confirm that all data supporting the findings of this study are available from the corresponding author upon reasonable request. Prof. Wang had full access to all data in the study and takes full responsibility for the integrity of the data and the accuracy of the data analysis.

Ethical Consideration

The study was approved by the Ethics Committee of the Affiliated Hospital of Chengde Medical University and was performed in accordance with the tenets of the Declaration of Helsinki.

Author Contributions

Tingting Wang and Yi are first co-authors. Wang Tingting: Conceptualization, Methodology, Writing – Original Draft, Project Administration; Sun Yi: Investigation, Validation, Writing – Review & Editing; Zhang Mengna: Formal Analysis, Supervision, Writing – Review & Editing; Cao Shuchen: Conceptualization, Supervision, Writing – Review & Editing; Wang Yanfei: Conceptualization, Funding Acquisition, Project Administration, Supervision, Writing – Review & Editing. All authors gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this study.