Machine learning prediction of in-hospital mortality risk among hospitalized patients with secondary bloodstream infection: a retrospective cohort study

Abstract

Background

Secondary bloodstream infection (SBSI) carries substantial mortality burden. Dedicated mortality prediction models with interpretable predictions remain limited.

Objective

To develop an interpretable machine learning model for predicting in-hospital mortality in SBSI patients and establish a clinically applicable web-based tool.

Methods

Data from 340 SBSI patients (January 2020–December 2024) were analyzed. Candidate variables were selected using the union of multivariate Cox regression and LASSO-Cox regression. Five prognostic models were constructed including Cox regression, LASSO-Cox regression, Random Survival Forest (RSF), Gradient Boosting Machine (GBM), and XGBoost, with 10 selected variables as predictors. Model performance was evaluated using time-dependent Area Under the Receiver Operating Characteristic Curve (AUC) at 7, 14, and 28 days as the primary metric. Internal validation was conducted using bootstrap resampling with 1,000 iterations. The optimal RSF model was interpreted using SHapley Additive exPlanations (SHAP) analysis. An interactive web-based prediction tool was subsequently developed.

Results

In-hospital all-cause mortality rate was 34.41% (117/340). Among the five models, RSF demonstrated superior discriminative ability with AUC values of 0.869 (95% CI: 0.824–0.914), 0.897 (95% CI: 0.856–0.937), and 0.878 (95% CI: 0.822–0.933) at 7, 14, and 28 days, respectively, substantially outperforming traditional Cox regression and LASSO-Cox models. RSF exhibited excellent calibration with corrected C-index of 0.8809. Decision curve analysis confirmed RSF’s clinical superiority, demonstrating the greatest net benefit across a wide range of threshold probabilities. SHAP analysis identified log Lactate Dehydrogenase, Multidrug-Resistant Organism infection, and log creatinine as the top three predictive features. An interactive web-based prediction tool was successfully developed enabling real-time risk assessment at 7, 14, and 28 days.

Conclusion

A interpretable machine learning model (RSF) for SBSI mortality prediction was successfully established. SHAP-based transparency and the interactive web-based tool facilitate clinical implementation and support personalized risk stratification for improved patient outcomes.

Supplementary information

The online version contains supplementary material available at 10.1186/s12879-026-13375-7.

Introduction

Bloodstream infection is a systemic infectious syndrome resulting from the invasion and proliferation of pathogenic microorganisms within the bloodstream and represents a major category of nosocomial infections [1]. Based on the source of infection, bloodstream infection is classified as primary or secondary. Secondary bloodstream infection (SBSI) is defined as bloodstream infection secondary to an infectious focus at another anatomical site [2]. Evidence indicates that patients with SBSI experience higher all-cause mortality, particularly in the presence of Multidrug-Resistant Organism (MDRO) [3, 4]. The pathogenesis of SBSI is multifactorial, including pathogen dissemination from the primary focus, host immune competence, and antimicrobial therapy strategies [5]. The likelihood of SBSI differs markedly by primary infection site, with lower respiratory tract, intra-abdominal, and urinary tract infections constituting the most frequent sources [3]. Accurate assessment of mortality risk in patients with SBSIs is essential for informed clinical decision-making. In recent years, machine learning approaches have demonstrated notable strengths in medical predictive modeling by accommodating complex nonlinear associations and capturing interactions among variables [6]. Prior studies have shown that prediction models based on machine learning and artificial intelligence yield favorable performance in risk stratification and therapeutic evaluation of sepsis [7, 8]. Nevertheless, such models are frequently regarded as “black boxes” because the underlying prediction mechanisms lack interpretability, thereby constraining their adoption in routine clinical practice [9]. SHAP, a game theory–based interpretability framework, enables quantitative attribution of individual feature contributions to model outputs and supports interpretable assessment at both global and individual levels [10]. The SHAP approach has been successfully applied in predictive modeling across domains including cardiovascular disease and oncology [11, 12]. Additionally, online risk calculators based on machine learning have been successfully developed and validated for predicting mortality in patients with sepsis and bloodstream infections, demonstrating the clinical utility of web-based prediction models in guiding therapeutic decision-making [13].

Although machine learning–based mortality risk prediction models have been developed for general sepsis and bloodstream infections, dedicated models for secondary bloodstream infections are limited, and clinically implementable tools with interpretable predictions are particularly lacking. In this study, a mortality risk prediction model for patients with SBSI was constructed using multiple machine learning algorithms derived from real-world data, the SHAP method was applied to achieve model interpretability, and a web-based prediction platform was developed to enhance clinical applicability. This approach provides an evidence-based foundation for early identification of high-risk patients and supports the formulation of individualized treatment strategies in clinical settings.

Participants and methods

Study design and participants

Study design

A single-center retrospective observational cohort study was conducted. Data on patients with SBSI were obtained from the Xinglin Nosocomial Infection Surveillance System at the First Affiliated Hospital of Nanjing Medical University (a 4,500-bed tertiary comprehensive hospital) between January 2020 and December 2024, including both hospital-acquired and community-acquired cases. This prediction model study was conducted and reported according to the TRIPOD+AI statement [14].

Study population

We retrospectively collected data from 408 patients identified using a parallel case-ascertainment strategy: patients labeled as SBSI and all patients labeled as bacteremia were exported, and bacteremia cases were independently cross-checked by two reviewers to capture potential misclassified SBSI. Inclusion criteria comprised: (1) a confirmed diagnosis of SBSI; and (2) age ≥ 18 years. Exclusion criteria included: (1) the blood culture was obtained > 3 days earlier than the onset of the primary infection(n = 17); (2) multiple primary infection sites with an indeterminate secondary source(n = 34); (3) ≥2 matched pathogens identified between the primary infection site and the bloodstream (n = 16); and (4) hospital stay > 300 days after SBSI onset (n = 1). After exclusions, 340 patients were included in the final analysis. According to clinical outcomes, patients were categorized into a survival group (N = 223) and a mortality group (N = 117). See Fig. 1. The study protocol was approved by the Ethics Committee of the First Affiliated Hospital with Nanjing Medical University and was conducted in adherence to the Declaration of Helsinki.

Fig. 1.

Flowchart of patient exclusion and inclusion

Diagnostic criteria and case definition

The diagnosis of SBSI was established in accordance with the 2025 edition of the National Healthcare Safety Network Patient Safety Manual [15] and required simultaneous fulfillment of the following conditions: (1) satisfaction of the diagnostic criteria for bloodstream infection; (2) documentation of a defined site infection; and (3) identification of at least one matched pathogen between blood culture and the corresponding site infection specimen, with blood samples obtained within 3 days before or 14 days after onset of the site infection. Diagnostic criteria for hospital-acquired infection and site-specific infection were based on the Diagnostic Criteria for Hospital-Acquired Infections (Trial Version) [16]. Pathogen matching was defined as consistency between the microorganism isolated from blood cultures and that identified from the site infection specimen. Site-specific infection referred to primary infections such as lower respiratory tract infection, intra-abdominal infection, and urinary tract infection.

Data collection

Outcome and time variables

The study outcome variable was in-hospital mortality, defined as either in-hospital death or treatment withdrawal/discharge against medical advice due to deterioration. The time variable is defined as the period of time from the collection of the blood culture sample to the patient’s death or discharge. The temporal distribution of mortality outcomes is presented in Supplementary Figure 1.

Covariate variables

The following candidate predictors were included: (1) baseline characteristics (age, sex, malignancy, diabetes, hypertension, and coronary heart disease); (2) infection-related characteristics, including MDRO infection, infections at other site (concomitant infections at sites other than the primary focus), and primary infection site (abdominal infection, lower respiratory tract infection, urinary tract infection, and other infections), Infection setting (community-acquired infection, healthcare-associated infection), pathogen type[gram-positive cocci（G+), gram-negative bacillus（G-), candida albicans)]; (3) treatment-related variables during the index hospitalization,including In-hospital surgery(surgery during the current hospitalization), and ICU admission; and (4) laboratory parameters obtained from the first routine laboratory tests at admission, including white blood cell (WBC), Platelets, creatinine, Lactate Dehydrogenase (LDH), D-dimer, Albumin-Bilirubin Grade (ALBI) grade, Neutrophil-to-Lymphocyte Ratio (NLR) score, Systemic Immune-Inflammation Index (SII) score, and Controlling Nutritional Status (CONUT) grade.

Scoring calculation formulas and classification

(1) NLR [17] score: applied to evaluate systemic inflammatory status and calculated as absolute neutrophil counts divided by absolute lymphocyte counts (unit: ×10⁹ /L). (2) SII score: applied to characterize the inflammatory–immune–coagulation state and calculated as neutrophil counts (×10⁹ /L) multiplied by platelet counts (×10⁹ /L) and divided by lymphocyte counts (×10⁹ /L). (3) CONUT [18] grade: applied to assess nutritional status and calculated as the sum of the albumin score, lymphocyte score, and total cholesterol score. Grading criteria were defined as follows: normal nutritional status (0–1 points, not observed in this cohort), mild malnutrition (2–4 points), moderate malnutrition (5–8 points), and severe malnutrition (≥9 points). (4) ALBI grade: applied to evaluate liver function and calculated as ALBI = log10 (total bilirubin [μmol/L]) × 0.66 + albumin [g/L] × (−0.085); grading criteria included ALBI grade 1 (≤−2.60), ALBI grade 2 (−2.60 to −1.39), and ALBI grade 3 (>−1.39).

Types of MDRO infection

In this study, MDRO infections included Carbapenem-resistant Enterobacterales (CRE), Carbapenem-resistant Acinetobacter Baumannii (CRAB), Carbapenem-resistant Pseudomonas Aeruginosa (CRPA), Methicillin-resistant Staphylococcus Aureus (MRSA), and Vancomycin-resistant Enterococcus (VRE) [19, 20].

Data preprocessing

A standardized preprocessing pipeline was applied to ensure data quality and analytical validity before statistical and machine learning analyses.

Missing data imputation

Missing values were identified across variables including SII score, LDH, Creatinine, D-dimer, CONUT grade, ALBI grade, and ICU admission status. Missing data were handled using multiple imputation by chained equations (MICE) via the mice package in R.The imputation parameters were as follows: m = 5 imputed datasets, maxit = 50 iterations, method = predictive mean matching (PMM), and seed = 123. Baseline characteristics (Table 1) were derived from the original dataset prior to imputation.

Table 1.

Comparison of baseline characteristics between survival and the mortality group

Variables	Total (n = 340)d	Survival group (n = 223)	Mortality group (n = 117)	p
Sex, n(%)
Male	238(70.000)	153(68.610)	85(72.650)	0.517
Female	102(30.000)	70(31.390)	32(27.350)
Malignancy, n(%)
No	265(77.941)	169(75.785)	96(82.051)	0.236
Yes	75(22.059)	54(24.215)	21(17.949)
Diabetes, n(%)
No	272(80.000)	179(80.269)	93(79.487)	0.977
Yes	68(20.000)	44(19.731)	24(20.513)
Hypertension, n(%)
No	226(66.471)	149(66.816)	77(65.812)	0.948
Yes	114(33.529)	74(33.184)	40(34.188)
Coronary heart disease, n(%)
No	301(88.529)	208(93.274)	93(79.487)	<0.001
Yes	39(11.471)	15(6.726)	24(20.513)
Primary infection site, n(%)
Intra-abdominal infection	135(39.706)	98(43.946)	37(31.624)	<0.001
Lower respiratory tract infection	105(30.882)	43(19.283)	62(52.991)
Urinary tract infection	68(20.000)	61(27.354)	7(5.983)
Other infections	32(9.412)	21(9.417)	11(9.402)
Infection setting, n(%)				0.672
Community-acquired infection	236 (69.412%)	157 (70.404%)	79 (67.521%)
Healthcare-associated infection	104 (30.588%)	66 (29.596%)	38 (32.479%)
In-hospital surgery, n(%)
No	168(49.412)	103(46.188)	65(55.556)	0.127
Yes	172(50.588)	120(53.812)	52(44.444)
MDRO infection, n(%)
No	225(66.176)	171(76.682)	54(46.154)	<0.001
Yes	115(33.824)	52(23.318)	63(53.846)
Pathogentype, n(%)				0.244
G+	70 (20.588%)	50 (22.422%)	20 (17.094%)
G-	254 (74.706%)	165 (73.991%)	89 (76.068%)
Candida albicans	16 (4.706%)	8 (3.587%)	8 (6.838%)
Infection at other sites, n(%)
No	226(66.471)	154(69.058)	72(61.538)	0.203
Yes	114(33.529)	69(30.942)	45(38.462)
Polymicrobial bacteremia, n(%)				0.248
No	304 (89.412%)	203 (91.031%)	101 (86.325%)
Yes	36 (10.588%)	20 (8.969%)	16 (13.675%)
ICU admission, n(%)				0.003
No	229 (67.353%)	163 (73.094%)	66 (56.410%)
Yes	111 (32.647%)	60 (26.906%)	51 (43.590%)
CONUT grade, n(%)
Mild malnutrition	67(20.743)	46(21.296)	21(19.626)	0.384
Moderate malnutrition	169(52.322)	117(54.167)	52(48.598)
Severe malnutrition	87(26.935)	53(24.537)	34(31.776)
ALBI grade, n(%)
Grade1	48(14.243)	36(16.216)	12(10.435)	0.185
Grade2	224(66.469)	148(66.667)	76(66.087)
Grade3	65(19.288)	38(17.117)	27(23.478)
Age,Mean (SD)	61.27 (14.98)	59.44 (14.51)	64.76 (15.30)	0.002
WBC(×109 /L), Median (IQR)	8.735 (5.858,13.845)	8.630 (5.740,12.915)	9.190 (6.360,14.930)	0.202
Platelets(×109 /L), Median (IQR)	171.000 (97.750,227.000)	176.000 (112.500,240.000)	150.000 (75.000,200.000)	0.005
NLR score, Median (IQR)	7.510[(3.270,18.750)	6.520(3.230,15.850)	12.770(3.790,22.000)	0.008
SII score, Median (IQR)	979.170(469.240,2724.660)	889.120(460.460,2497.880)	1068.510(478.270,3007.710)	0.255
D-dimer(mg/L), Median (IQR)	2.540(0.790,6.600)	2.010(0.640,5.650)	3.450(1.420,8.030)	0.003
Creatinine(μmol/L ), Median (IQR)	74.100(54.800,133.500)	69.200(52.800,97.900)	93.000(64.800,224.200)	<0.001
LDH(,U/L ), Median (IQR)	253.000 (192.500,395.500)	230.000 (186.000;335.000)	311.000 (227.000;511.500)	<0.001

Normality assessment and log-transformation

The distribution of all continuous variables was evaluated using the Lilliefors test (Kolmogorov-Smirnov test with estimated parameters). Continuous variables with marked right-skewed distributions were subjected to log transformation (log[x+1]) to reduce the influence of skewness and extreme values on regression modeling. Age was approximately symmetrically distributed and was retained in its original scale.

Assessment of multicollinearity

Variance inflation factors (VIF) were calculated to detect multicollinearity among candidate predictors. Variables with VIF ＞5 were excluded from subsequent analyses. All retained variables had VIF values＜5, indicating acceptable levels of multicollinearity.

Final analytical dataset

After the complete preprocessing pipeline, the final analytical dataset comprised 340 patients and 20 variables, including 4 log-transformed continuous variables. The primary outcome was all-cause in-hospital mortality, recorded in 117 patients (34.4%), with 223 survivors (65.6%).

Statistical and machine learning methods

Statistical analysis

Continuous variables were assessed for normality and homogeneity of variance. Variables approximating a normal distribution were summarized as mean ± standard deviation and compared using independent-samples t tests, whereas non-normally distributed variables were reported as median (interquartile range) and compared using the Mann–Whitney U test. Categorical variables were presented as CONUTs (%) and compared using the χ² test or Fisher’s exact test, as appropriate.

Cox proportional hazards regression

Univariate Cox proportional hazards regression（Cox regression） was performed on all 20 preprocessed variables. Nine variables with statistical significance (p < 0.05) were subsequently entered into a multivariate Cox proportional hazards regression model. Through multivariate analysis, six independent risk factors were identified. Hazard ratios (HRs) with 95% confidence intervals (CIs) were calculated for each variable.

Variable selection strategy

To identify key prognostic factors, we employed both LASSO-Cox regression and traditional Cox regression analyses. LASSO-Cox regression with ten-fold cross-validation identified nine significant variables, while traditional Cox regression identified six independent risk factors. The union of these two methods yielded ten final prognostic variables for model development. This dual-method strategy combines the strengths of penalized regression (LASSO) with full multivariate analysis, ensuring robust variable selection.

Model development

Five prognostic models were developed using the ten selected variables as predictors: Cox regression, LASSO-Cox regression, Random Survival Forest (RSF), Gradient Boosting Machine (GBM), and XGBoost.

Model validation and performance evaluation

To assess model generalizability and adjust for optimism bias, we conducted internal validation using bootstrap resampling with 1,000 iterations. The apparent C-index and bootstrap-corrected C-index were calculated for each model to quantify discrimination ability. Model discriminative ability was evaluated using time-dependent Area Under the Receiver Operating Characteristic Curve (AUC) at fixed time points of 7, 14, and 28 days. Sensitivity, specificity, accuracy, positive predictive value (PPV), negative predictive value (NPV), and Youden’s index were calculated at the optimal cutoff threshold determined by maximizing the Youden index. Decision Curve Analysis (DCA) was performed to assess the clinical net benefit of each model across a range of threshold probabilities, comparing the benefit of using each model versus treating all patients or none. Models were evaluated at the three predefined time points (7, 14, and 28 days) to capture early, intermediate, and extended mortality prediction horizons.

SHAP-based model interpretability

To enhance model interpretability, SHapley Additive exPlanations (SHAP) values were computed for the RSF model using the iml (Interpretable Machine Learning) R package. Shapley values were calculated for all patients using the Shapley function with a sample size of 50 coalition samples per observation. Each patient’s ensemble mortality prediction was decomposed into contributions from the 10 predictor variables, providing both direction (increasing or decreasing mortality risk) and magnitude of impact.

Web-based prediction model development

An interactive web-based prediction tool was developed using Shiny, a web framework that enables the creation of interactive web applications directly from R code. The application integrates the trained RSF model with a user-friendly interface for real-time mortality risk prediction.

All Statistical analyses were performed using R version 4.4.2. p < 0.05 was considered as a significant difference. Detailed statistical methods are described in the”Supplementary Methods”.

Results

Comparison of baseline characteristics between survival and mortality group

A total of 340 patients were analyzed, including 223 in the survival group and 117 in the mortality group. Compared with the survival group, the mortality group demonstrated significantly different distributions of primary infection site (χ² = 48.694, p<0.001) and a markedly higher prevalence of MDRO infection (53.85% vs. 23.32%, χ² = 31.951, p<0.001). A greater proportion of coronary heart disease was observed in the mortality group (20.51% vs. 6.73%, χ² = 14.363, p<0.001), accompanied by older age (64.76 ± 15.30 vs. 59.44 ± 14.51 years, p = 0.002).

Laboratory parameters differed significantly between groups. The mortality group exhibited lower platelet counts (150.000 vs. 176.000, p = 0.005], higher NLR (12.77 vs. 6.52, p = 0.008), elevated D-dimer (3.45 vs. 2.01, p = 0.003), and markedly increased LDH levels (311.0 vs. 230.0, p < 0.001). Additionally, the mortality group showed higher creatinine (93.0 vs. 69.2, p < 0.001). Furthermore, ICU admission rate was significantly higher in the mortality group (43.59% vs. 26.91%, p = 0.003).

No statistically significant differences were identified between groups with respect to sex, in-hospital surgery, Infection at other sites, malignancy, diabetes, hypertension, CONUT grade, ALBI grade, pathogen type, polymicrobial bacteremia, and infection setting (all p>0.05). Detailed results are presented in Table 1.

Cox regression analysis for mortality risk factors among patients with SBSI

Univariate Cox regression analysis identified nine significant risk factors associated with mortality, including lower respiratory tract infection (HR: 2.545, p < 0.001), MDRO infection (HR: 2.295, p < 0.001), coronary heart disease (HR: 2.1, p = 0.001), ALBI grade 3 (HR: 2.172, p = 0.026), ICU admission (HR: 1.57, p = 0.016), age (HR: 1.022, p = 0.001), log LDH (HR: 1.727, p < 0.001), log creatinine (HR: 1.684, p < 0.001), and log D-dimer (HR: 1.191, p = 0.012) (Table 2).

Table 2.

Univariate and multivariate cox regression analysis of mortality risk factors in SBSI patients

Variable	Univariate cox		Multivariate cox
	HR(95 CI%)	P value	HR(95 CI%)	P value
Sex	1
Male	0.961 (0.639,1.444)	0.848
Female
Primary infection site
Intra-abdominal infection	1		1
Lower respiratory tract infection	2.545 (1.689,3.835)	＜0.001	2.003 (1.222,3.282)	0.006
Urinary tract infection	0.467 (0.208,1.049)	0.065	0.497 (0.214,1.155)	0.104
Other infections	1.42 (0.722,2.79)	0.309	1.28 (0.64,2.562)	0.485
Infection setting	0.924 (0.625,1.364)	0.689
Community-acquired infection
Healthcare-associated infection
Polymicrobial bacteremia
No	1
Yes	1.399 (0.824,2.375)	0.214
In-hospital surgery
No	1		1
Yes	0.625 (0.433,0.901)	0.012	0.804 (0.521,1.239)	0.322
Pathogen type
G+	1
G-	1.343 (0.827,2.182)	0.233
Candida albicans	2.18 (0.958,4.965)	0.063
MDRO infection
No	1		1
Yes	2.295 (1.594,3.303)	＜0.001	1.653 (1.097,2.49)	0.016
Infection at other sites
No	1
Yes	0.979 (0.673,1.425)	0.913
Malignancy
No	1
Yes	0.781 (0.486,1.254)	0.306
Diabetes
No	1
Yes	0.98 (0.625,1.537)	0.929
Hypertension
No	1
Yes	1.059 (0.723,1.552)	0.768
Coronary heart disease
No	1		1
Yes	2.1 (1.339,3.293)	0.001	1.551 (0.943,2.549)	0.084
CONUT grade
Mild malnutrition	1
Moderate malnutrition	0.91 (0.556,1.489)	0.707
Severe malnutrition	1.302 (0.77,2.201)	0.325
ALBI grade
Grade1	1		1
Grade2	1.784 (0.97,3.281)	0.062	1.57 (0.811,3.037)	0.181
Grade3	2.172 (1.099,4.292)	0.026	2.665 (1.181,6.011)	0.018
ICU admission
No	1		1
Yes	1.57 (1.089,2.264)	0.016	0.842 (0.536,1.324)	0.457
Age	1.022 (1.009,1.034)	0.001	1.023 (1.009,1.037)	0.001
log SII	1.04 (0.888,1.217)	0.629
log LDH	1.727 (1.389,2.148)	＜0.001	1.611 (1.213,2.141)	0.001
log Creatinine	1.684 (1.362,2.082)	＜0.001	1.322 (1.017,1.717)	0.037
log D-dimer	1.191 (1.039,1.365)	0.012	0.966 (0.804,1.161)	0.712

To identify independent prognostic factors, multivariate Cox regression analysis was conducted incorporating the significant variables from univariate analysis. Lower respiratory tract infection (HR: 2.003, p = 0.006), MDRO infection (HR: 1.653, p = 0.016), ALBI grade 3 (HR: 2.665, p = 0.018), age (HR: 1.023, p = 0.001), log LDH (HR: 1.611, p = 0.001), and log creatinine (HR: 1.322, p = 0.037) emerged as independent predictors of mortality (Table 2).

Variable selection for the mortality risk prediction model in SBSI patients

LASSO-Cox regression identified nine variables associated with mortality from the initial 20 candidate variables, including MDRO infection, coronary heart disease, log LDH, log creatinine, ALBI grade, pathogen type, age, infection at other sites, and in-hospital surgery (Fig. 2).

Fig. 2.

LASSO-Cox regularization path and cross-validation curves

To identify the most robust predictors, we employed both traditional multivariate Cox regression and LASSO-Cox regression for variable selection. The Venn diagram in Fig. 3 depicts the overlap between variables retained by both methods. Five variables were consistently selected by both multivariate Cox regression and LASSO-Cox regression, demonstrating high reproducibility and reliability. These common variables include primary infection site, MDRO infection, log LDH, log creatinine, and ALBI grade.

Fig. 3.

Venn diagram of variables selected by multivariate Cox regression and LASSO-Cox regression. The red circle represents variables selected by multivariable Cox regression analysis (n = 6 variables: 1 unique to multivariable Cox, 5 overlapping). The blue circle represents variables selected by LASSO-Cox regression (n = 9 variables: 4 unique to LASSO-Cox, 5 overlapping). The overlapping region (n = 5) represents variables consistently selected by both methods, indicating robust prognostic factors for mortality prediction

Additionally, multivariate Cox regression identified one unique variable (in-hospital surgery), while LASSO-Cox regression identified four unique variables (coronary heart disease, pathogen type, age, and infection at other sites). The final prediction model was constructed by integrating the union of all variables identified by either method, incorporating primary infection site, MDRO infection, coronary heart disease, log LDH, log creatinine, ALBI grade, pathogen type, age, infection at other sites, and in-hospital surgery. This comprehensive approach ensures that the selected variables capture all statistically significant predictors identified through rigorous variable selection procedures, providing enhanced clinical applicability for predicting patient mortality risk.

Prognostic performance comparison across prediction models

Machine learning approaches demonstrated superior prognostic performance compared to traditional statistical methods across all time points. RSF achieved the highest discriminative ability, with AUC values of 0.869 (95% CI: 0.824–0.914), 0.897 (95% CI: 0.856–0.937), and 0.878 (95% CI: 0.822–0.933) at 7, 14, and 28 days, respectively (Table 3, Fig. 4). Notably, RSF also exhibited the highest corrected C-index (0.8809) among all models, indicating excellent calibration after accounting for optimism bias. In contrast, traditional Cox regression and LASSO-Cox models exhibited substantially lower AUC values, ranging from 0.723–0.772 and 0.723–0.769, respectively, with corrected C-indices of 0.7480 and 0.7455 (Tables 3, 4).

Table 3.

Performance comparison of statistical and machine learning models at 7, 14, and 28 Days

Model	Time	AUC(95 CI%)	Sensitivity	Specificity	Accuracy	PPV	NPV	Youden_Index
Cox	7-day	0.725 (0.649–0.801)	0.8125	0.5719	0.6059	0.2378	0.9489	0.3844
	14-day	0.772 (0.711–0.834)	0.9167	0.4219	0.5441	0.3422	0.9391	0.3385
	28-day	0.767 (0.692–0.841)	0.9266	0.3074	0.5059	0.387	0.8987	0.234
LASSO-Cox	7-day	0.723 (0.648–0.798)	0.7708	0.5719	0.6	0.2284	0.9382	0.3428
	14-day	0.769 (0.708–0.831)	0.8452	0.5703	0.6382	0.3923	0.9182	0.4156
	28-day	0.759 (0.683–0.835)	0.9266	0.3203	0.5147	0.3915	0.9024	0.247
RSF	7-day	0.869 (0.824–0.914)	1	0.4521	0.5294	0.2308	1	0.4521
	14-day	0.897 (0.856–0.937)	0.9643	0.5234	0.6324	0.399	0.9781	0.4877
	28-day	0.878 (0.822–0.933)	0.9541	0.5498	0.6794	0.5	0.9621	0.5039
GBM	7-day	0.781 (0.713–0.849)	0.9375	0.4486	0.5176	0.2184	0.9776	0.3861
	14-day	0.829 (0.775–0.883)	0.9405	0.5039	0.6118	0.3835	0.9627	0.4444
	28-day	0.814 (0.746–0.882)	0.9174	0.5714	0.6824	0.5025	0.9362	0.4889
XGBoost	7-day	0.793 (0.731–0.855)	0.9375	0.4521	0.5206	0.2195	0.9778	0.3896
	14-day	0.830 (0.776–0.884)	0.9405	0.543	0.6412	0.4031	0.9653	0.4834
	28-day	0.833 (0.767–0.898)	0.8899	0.6883	0.7529	0.574	0.9298	0.5782

Fig. 4.

Time-dependent ROC curves comparing prognostic performance of statistical and machine learning approaches at 7, 14, and 28 Days in SBSI patients

Table 4.

Discrimination and calibration performance metrics of five prognostic models for SBSI mortality prediction

Model	Apparent Cindex	Mean Boot Cindex	Optimism	Corrected Cindex	95% CI
Cox	0.7357	0.7234	−0.0123	0.7480	0.7041–0.7361
LASSO-Cox	0.7343	0.7232	−0.0112	0.7455	0.7029–0.7361
RSF	0.8469	0.8128	−0.034	0.8809	0.7929–0.8347
GBM	0.7885	0.8342	0.0457	0.7428	0.8095–0.8584
XGBoost	0.7941	0.7672	−0.0269	0.8210	0.7446–0.7865

RSF maintained superior sensitivity (>0.95) across all prediction horizons while preserving reasonable specificity (0.45–0.54), achieving consistently high overall accuracy (Table 3),. Decision curve analysis further confirmed RSF’s clinical superiority, demonstrating the greatest net benefit across a wide range of threshold probabilities compared to other models (Fig. 5). These findings collectively support RSF as the optimal model for mortality prediction and risk stratification in SBSI patients.

Fig. 5.

Time-dependent decision curve analysis of five prognostic models for SBSI mortality prediction at 7, 14, and 28 Days

SHAP-based model interpretation and feature contribution

SHAP analysis revealed the relative importance ranking of features in the RSF model. Log LDH, MDRO infection, and log creatinine were the top three most influential predictors for mortality risk stratification（Fig. 6). The SHAP dependence plots demonstrated the non-linear relationships between these key variables and predicted mortality (Fig. 7).

Fig. 6.

SHAP beeswarm plot illustrating feature contributions to RSF mortality predictions. Features are ranked by mean absolute SHAP values. Each dot represents a patient sample; horizontal position indicates SHAP value (impact on mortality prediction), and color intensity represents feature value magnitude (blue = low, red = high). Points extending rightward indicate increased mortality risk; points extending leftward indicate protective effects

Fig. 7.

SHAP dependence plots of top 10 variables associated with mortality in SBSI patients. Each point represents an individual patient, with the color gradient indicating the value of another feature. The red smoothing curve represents the overall trend. SHAP values greater than 0 indicate increased risk of mortality, while values less than 0 indicate decreased risk. Binary variables are coded as: 0 = no, 1 = yes. Primary infection site: 1 = intra-abdominal infection, 2 = lower respiratory tract infection, 3 = urinary tract infection, 4 = other infections. Pathogen type: 1 = gram-positive bacteria (G+), 2 = gram-negative bacteria (G-), 3 = Candida albicans. ALBI grade: 1 = grade 1, 2 = grade 2, 3 = grade 3

Clinical prediction tool

To facilitate clinical implementation, an interactive web-based prediction tool was developed based on the RSF model (Fig. 8). The tool provides time-dependent mortality risk assessment at three clinically relevant timepoints (7, 14, and 28 days), enabling dynamic risk stratification throughout the patient’s hospital course. Users input key clinical and laboratory parameters including demographic data (age), biochemical markers (LDH, creatinine), liver disease severity (ALBI grade), infection characteristics (primary infection site, pathogen type, MDRO status), and procedural factors (in-hospital surgery, infection at other sites). The model automatically calculates predicted mortality probability and stratifies risk into three categories: low risk (<30%), intermediate risk (30–60%), and high risk (≥60%).

Fig. 8.

Web-based clinical prediction tool for in-hospital mortality risk assessment in SBSI patients

The tool provides an intuitive interface with real-time risk visualization and input summaries, enabling clinicians to rapidly assess individual patient risk profiles across multiple timepoints, guide personalized treatment intensification, and support informed discussions regarding prognosis and resource allocation. The interactive prediction tool is publicly accessible at: https://zhanzhanouba.shinyapps.io/moxing/

Discussion

Through comprehensive analysis of 340 patients with SBSI, a machine learning–based mortality risk prediction model was developed, and SHAP interpretability analysis was applied to enhance transparency. A web-based prediction platform was subsequently established to translate these findings into accessible clinical decision support, offering novel perspectives for advancing patient care in this domain. The results indicated an overall in-hospital mortality rate of 34.4% among patients with SBSI, consistent with previously reported data [3].

In this study, we employed both multivariable Cox regression and LASSO-Cox regression to identify independent risk factors for in-hospital mortality in patients with SBSI. The convergence of these two statistical approaches yielded 10 variables that formed the foundation of our machine learning model, a methodological strategy that enhances the robustness and biological plausibility of our predictions. Among these variables, five were consistently identified by both multivariable Cox regression and LASSO-Cox regression, namely primary infection site, MDRO infection, log LDH, log creatinine, and ALBI grade, warranting particular attention as they represent the most stable and reproducible prognostic factors. Log LDH, MDRO infection, and log creatinine emerged as the top three predictors with the highest SHAP importance scores, suggesting their paramount role in mortality risk stratification.

Log LDH emerged as the most potent predictor of mortality in our model. Serum LDH is an independent predictor of 28-day mortality in septic patients (AUC = 0.783), serving as a sensitive biomarker of tissue injury in sepsis [21]. Furthermore, failure of LDH to improve within 48 hours of admission strongly predicts mortality [22]. Serum creatinine serves as a critical marker of organ dysfunction, specifically reflecting the severity of renal impairment. In our model, creatinine demonstrated pronounced threshold effects, with mortality risk increasing steeply beyond a critical value threshold. Elevated creatinine concentration indicates the progression toward multiple organ dysfunction syndrome (MODS), a hallmark of severe sepsis with substantially increased mortality. The ALBI grade represents an objective measure of hepatic synthetic function, capable of detecting more subtle changes in liver function compared to traditional scoring systems such as the Child-Pugh and MELD scores. Higher ALBI grades were significantly associated with increased in-hospital mortality, 30-day mortality, and 90-day mortality among septic patients in the intensive care unit [23, 24].

Primary infection site emerged as a key predictor of secondary bloodstream infection, reflecting the heterogeneity in clinical outcomes across different anatomical locations of infection. Our model demonstrated that this variable contributed significantly to mortality risk prediction, consistent with findings from previous studies [25]. The anatomical site of primary infection influences the rate of bacterial translocation, the magnitude and quality of immune activation, and the severity of organ dysfunction, all of which collectively determine patient prognosis in SBSI. This observation offers meaningful insights for optimizing the management of SBSI. Identification of the primary infection site as the dominant predictive factor highlights the central importance of source control in infection management. Previous research [26] has shown that early screening to identify the infection site and timely management of the primary infectious focus can improve outcomes in patients with bloodstream infection. In particular, patients with bloodstream infection secondary to lower respiratory tract infection may warrant intensified therapeutic interventions and closer clinical monitoring [27]. MDRO infection can limit the choice of effective antimicrobial agents, thereby increasing treatment difficulty and elevating the risk of mortality [28]. Our model ranked MDRO infection as the second most influential predictor, underscoring the critical role of pathogen antimicrobial resistance in the prognosis of SBSI and emphasizing the necessity for clinicians to exercise judicious antimicrobial selection and implement rigorous infection control measures [29].

This study demonstrates that ML approaches substantially outperform traditional logistic regression and Cox proportional hazards models for predicting in-hospital mortality in SBSI patients. The RSF model achieved superior discriminative ability with area under the receiver operating characteristic curve (AUC) values of 0.869, 0.897, and 0.878at 7, 14, and 28 days, respectively, compared to Cox regression (0.725, 0.772, 0.767) and LASSO-Cox models (0.723, 0.769, 0.759). RSF demonstrated excellent calibration after optimism correction (corrected C-index: 0.8809), substantially outperforming traditional Cox regression and LASSO-Cox models (C-indices: 0.7480 and 0.7455, respectively).

SHAP-based model interpretation provided unprecedented transparency in understanding RSF predictions. Calculation and visualization of SHAP values enabled elucidation of both global prediction patterns and individualized risk factor contributions for each patient [10]. The non-linear relationships revealed through SHAP dependence plots underscore why traditional linear models fail to capture complex prognostic patterns in critical illness. This interpretability is crucial for clinician acceptance and regulatory approval of ML models in clinical practice, addressing long-standing concerns about the “black box” nature of ensemble methods [9]. From a clinical application standpoint, the prediction model developed in this study demonstrates notable practical strengths. The development of an interactive web-based prediction tool facilitates clinical translation of the RSF model. Real-time risk assessment at multiple clinically relevant timepoints (7, 14, and 28 days) enables dynamic risk stratification throughout the hospital course, supporting personalized treatment decisions and resource allocation

Nevertheless, several limitations warrant consideration. As a single-center retrospective investigation, sample representativeness and external validity may be constrained. Multicenter prospective studies are therefore required to verify the broader applicability of the model [30]. In addition, the cohort of patients with secondary bloodstream infection included in this study may still be incomplete, as some secondary bloodstream infections may have been merged into primary infection diagnoses and thus could not be identified in the hospital infection surveillance system. In future work, we will further disentangle and refine the data to provide a more comprehensive and complete dataset.

Conclusion

In summary, this study developed a machine learning–based RSF model for predicting in-hospital mortality in SBSI patients. Through convergent statistical approaches, we identified ten robust prognostic variables, with log LDH, MDRO infection, and log creatinine demonstrating the highest predictive importance. SHAP-based interpretability analysis provided unprecedented transparency in model predictions, addressing concerns about clinical implementation. The interactive web-based prediction platform enables real-time risk stratification and personalized clinical decision-making for SBSI patients.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Abbreviations

ALBI

Albumin-Bilirubin

AUC

Area Under the Curve

CI

Confidence Interval

CONUT

Controlling Nutritional Status

CRAB

Carbapenem-Resistant Acinetobacter baumannii

CRE

Carbapenem-Resistant Enterobacterales

CRPA

Carbapenem-Resistant Pseudomonas aeruginosa

GBM

Gradient Boosting Machine

LASSO

Least Absolute Shrinkage and Selection Operator

MDRO

Multidrug-Resistant Organism

MRSA

Methicillin-Resistant Staphylococcus aureus

NLR

Neutrophil-to-Lymphocyte Ratio

RSF

Random Survival Forest

ROC

Receiver Operating Characteristic

SBSI

Secondary Bloodstream Infection

SHAP

SHapley Additive exPlanations

SII

Systemic Immune-Inflammation Index

VRE

Vancomycin-Resistant Enterococcus

XGBoost

eXtreme Gradient Boosting

Author contributions

Zhanjie Li, Jidan Zhang, Zhijie Zhang, and Liyun Wang conceived and designed the study. Zhanjie Li collected and curated the data. Zhanjie Li and Jidan Zhang performed the data analysis and interpretation. Zhanjie Li and Jidan Zhang drafted the manuscript. Zhijie Zhang and Liyun Wang critically revised the manuscript for important intellectual content. All authors (Zhanjie Li, Jidan Zhang, Zhijie Zhang, and Liyun Wang) read and approved the final manuscript and agree to be acCONUTable for all aspects of the work.

Funding

This work was supported by Chinese Preventive Medicine Association Hospital Infection Department Development Youth Talent Promotion Project (CPMA-HAIsC-2024012900108); Jiangsu Provincial Association for Science and Technology Young Science and Technology Talent Support Project (Health Field) (JSTJ-2023-WJ006); Jiangsu Province Hospital Management Innovation Research Project (JSYGY-3-2023-559); Project of Chinese Hospital Reform and Development Institute, Nanjing University (NDYG2023039); The third Outstanding Young and Middle-aged Talents Training Program of Jiangsu Provincial People’s Hospital (YNRCQN0314); Young Scholars Fostering Fund of the First Affiliated Hospital with Nanjing Medical University (PY2022017).

Data availability

The data used to support the findings of this study are available from the corresponding author upon request.

Declarations

Human ethics and consent to participate

This study was conducted in accordance with the revised Declaration of Helsinki and was approved by the Ethics Committee of the First Affiliated Hospital with Nanjing Medical University. The requirement for written informed consent was waived by the Ethics Committee because this study was a retrospective analysis using anonymized data.Clinical trial number: not applicable.

Consent for publication

This study was conducted as a retrospective analysis of existing medical records and did not involve direct interaction with patients. Therefore, informed consent for publication was not required. The data used in this research were anonymized to protect patient confidentiality, and all procedures were in accordance with ethical standards.

Competing interests

The authors declare no competing interests.