Development and validation of an interpretable machine learning model for predicting sepsis risk in pleural effusion patients

Abstract

Background

Intensive care unit (ICU)-acquired sepsis in patients with pleural effusion presents a formidable clinical challenge with high mortality. We aimed to develop and externally validate an interpretable machine learning (ML) framework for early risk prediction.

Methods

This multicenter retrospective study utilized data from the Medical Information Mart for Intensive Care IV (MIMIC-IV) and eICU Collaborative Research Database (eICU databases). A robust tri-algorithm intersection strategy [Boruta, least absolute shrinkage and selection operator (LASSO), and recursive feature elimination (RFE)] was employed to extract core predictors. Nine supervised ML algorithms were systematically evaluated. SHapley Additive exPlanations (SHAP) and restricted cubic spline (RCS) analyses were integrated to demystify the “black-box” decision-making process. The optimal model was deployed as a dynamic nomogram and validated via decision curve analysis (DCA).

Results

We identified a parsimonious subset of 10 critical predictors. During comprehensive algorithmic competition, the logistic regression (LR) model demonstrated superior discriminative performance and external generalizability. Global SHAP analysis identified baseline Sequential Organ Failure Assessment (SOFA) score, pneumonia, and preemptive sedative use as top-tier risk drivers, while highlighting the prognostic value of metabolic markers (anion gap and albumin). Crucially, RCS analysis revealed a distinct non-linear, “U-shaped” dose-response relationship between baseline white blood cell (WBC) counts and sepsis risk, cautioning against the deceptive reassurance of sepsis-associated leukopenia. DCA confirmed that the nomogram provided substantial net clinical benefit.

Conclusions

We successfully developed a highly transparent, data-driven ML framework to predict ICU-acquired sepsis risk in pleural effusion patients. The resulting web-based nomogram offers intensive care physicians an actionable bedside tool for personalized risk stratification and timely intervention.

Highlight box.

Key findings

• We developed and externally validated a highly interpretable logistic regression model based on 10 critical predictors to predict intensive care unit (ICU)-acquired sepsis in patients with pleural effusion.

What is known and what is new?

• ICU-acquired sepsis in patients with pleural effusion is a formidable clinical challenge with high mortality, yet traditional predictive models often lack specific tailoring and interpretability.

• We integrated Shapley Additive exPlanations and restricted cubic spline analyses to demystify the machine learning “black-box”, revealing a distinct “U-shaped” dose-response relationship between baseline white blood cell counts and sepsis risk.

What is the implication, and what should change now?

• The resulting web-based nomogram offers intensive care physicians an actionable bedside tool for personalized risk stratification and timely intervention, alerting clinicians to the deceptive reassurance of sepsis-associated leukopenia.

Introduction

Sepsis is a devastating syndrome defined as life-threatening organ dysfunction caused by a dysregulated host response to infection (1). Despite significant advances in critical care medicine, it remains a leading cause of morbidity and mortality in intensive care units (ICUs) worldwide, imposing an enormous epidemiological and economic burden on global healthcare systems (2,3). Among the highly vulnerable critically ill populations, patients presenting with pleural effusion face uniquely elevated risks. In the ICU setting, pleural effusion is rarely a benign, passive accumulation of fluid; rather, it frequently signifies underlying severe pneumonia, complicated parapneumonic infections, or profound systemic inflammation (4). The physical presence of massive effusions further severely compromises respiratory mechanics and venous return, creating a vicious cycle of hypoxia and hemodynamic instability that predisposes patients to septic shock (5).

The insidious onset of sepsis in these patients often overlaps with their primary pulmonary or hemodynamic symptoms, making early and precise identification extremely challenging. It is well established that delayed administration of targeted antimicrobial therapy is inextricably linked to exponential increases in mortality rates (6). Traditionally, intensive care physicians have relied on generalized prognostic scoring systems, such as the Sequential Organ Failure Assessment (SOFA) or the Acute Physiology and Chronic Health Evaluation (APACHE) II, to stratify risk. However, these tools were fundamentally designed to assess broad, reactive illness severity rather than proactively predicting specific infectious trajectories (7,8). Furthermore, while traditional circulating biomarkers like procalcitonin (PCT) and C-reactive protein (CRP) are widely utilized in clinical routines, they exhibit significant limitations in the complex ICU environment. Their diagnostic specificity is often compromised, as they can be markedly elevated in response to sterile inflammation, major surgery, or trauma, thereby leading to false-positive alarms and the overuse of broad-spectrum antibiotics (9,10).

In recent years, machine learning (ML) has revolutionized clinical predictive modeling. By excelling at identifying complex, multidimensional patterns within massive electronic health records (EHRs), ML has achieved remarkable success in predicting various critical care complications, such as acute kidney injury and acute respiratory distress syndrome (11,12). Recently, ML algorithms applied to large-scale critical care databases, such as Medical Information Mart for Intensive Care IV (MIMIC-IV) and eICU, have also demonstrated exceptional capability in predicting sepsis trajectories and risk stratification (13). Despite their superior discriminative performance, the widespread clinical adoption of advanced ML models [e.g., ensemble trees or deep neural networks (NNs)] has been severely hindered by their inherent “black-box” nature (14). In critical care scenarios where life-and-death decisions are made, clinicians require not only an accurate probability score but also a transparent, pathophysiologically sound explanation of why the algorithm has reached a specific conclusion—a paradigm known as explainable artificial intelligence (XAI) (15).

To address these critical clinical and methodological gaps, this multicenter retrospective study leveraged high-resolution data from both the MIMIC-IV and eICU databases. The primary aim of this study was to construct and externally validate an optimal, highly interpretable predictive model for ICU-acquired sepsis specifically tailored for patients with pleural effusion. We employed a rigorous feature selection pipeline [intersecting Boruta, least absolute shrinkage and selection operator (LASSO), and recursive feature elimination (RFE) algorithms] and conducted a comprehensive performance competition among nine distinct algorithms. Crucially, we integrated Shapley Additive exPlanations (SHAP) and restricted cubic spline (RCS) analyses to explicitly demystify the algorithmic decision-making process and unveil vital non-linear dose-response relationships (16). Finally, the best-performing model was translated into an intuitive nomogram and a dynamic web-based calculator. This translation of complex computational algorithms into user-friendly clinical tools has been increasingly advocated in the latest literature to facilitate real-time decision-making at the bedside (17). The core clinical message we intend to convey is that leveraging such a highly transparent, data-driven tool can assist critical care physicians in identifying high-risk patients early, thereby optimizing personalized bedside interventions and resource allocation. We present this article in accordance with the TRIPOD reporting checklist (available at https://jtd.amegroups.com/article/view/10.21037/jtd-2026-0824/rc).

Methods

Study design and ethical considerations

This multicenter retrospective cohort study was conducted using data extracted from two large, publicly accessible critical care databases: the MIMIC-IV (version 3.1, https://physionet.org/content/mimiciv/) (18) and the eICU Collaborative Research Database (eICU-CRD, https://physionet.org/content/eicu-crd/) (19). The study was conducted in strict accordance with the Declaration of Helsinki and its subsequent amendments. Regarding institutional review board (IRB) authorization, given that both databases consist of de-identified health information, the requirement for individual patient consent and formal ethical approval was waived by the IRBs of the PhysioNet platform. Furthermore, one of the authors has successfully completed the Collaborative Institutional Training Initiative (CITI) program course “Data or Specimens Only Research” and fulfilled the credentialing requirements set by PhysioNet to officially access both databases.

Study population and data preprocessing

Adult patients (age ≥18 years) diagnosed with pleural effusion upon ICU admission were screened for eligibility. To ensure baseline immunological homogeneity and avoid data confounding, patients were excluded based on the following strict criteria: (I) non-first ICU admission (to prevent within-patient correlation bias); (II) ICU length of stay less than 24 hours (as this prevents adequate observation for our fixed 24-hour baseline feature extraction window); and (III) pre-existing severe immune-compromising conditions, including human immunodeficiency virus (HIV) infection, leukemia, lymphoma, and multiple myeloma. Following the application of these criteria, the final sample size comprised 4,649 patients in the internal development cohort (MIMIC-IV) and 2,870 patients in the external validation cohort (eICU).

Prior to algorithm training, a rigorous clinical feature engineering step was applied using tabular data extracted within the first 24 hours of ICU admission. Variables with missing rates exceeding 20% were considered highly incomplete and subsequently excluded. For the remaining variables, missing values were imputed using the multiple imputation method to generate complete datasets, preserving statistical power and reducing bias (20). Notably, to mitigate multicollinearity and prevent algorithmic overfitting, variables exhibiting high clinical redundancy—such as blood urea nitrogen (BUN), which heavily overlaps with the renal sub-score of the SOFA score—and noise-susceptible indicators like red blood cell (RBC) count, were a priori excluded from the candidate feature pool.

Optimal feature selection via ML intersections

To overcome the inherent biases and instability of single-algorithm feature selection, a highly complementary, tri-algorithm hybrid strategy was employed. First, the Boruta algorithm was applied to holistically capture all relevant attributes by comparing them with randomized shadow features. Subsequently, LASSO regression with 10-fold cross-validation was utilized to perform dimensionality reduction and eliminate multicollinearity and enforce model sparsity via L1 regularization. Finally, RFE was conducted to systematically rank feature importance and verify algorithmic stability. Only the independent variables uniformly identified by the intersection of these three algorithms were retained to form the final parsimonious optimal subset.

Model development and performance evaluation

Nine supervised ML algorithms were developed to predict the onset of sepsis: logistic regression (LR), support vector machine (SVM), gradient boosting machine (GBM), NN, K-nearest neighbors (KNN), eXtreme gradient boosting (XGBoost), AdaBoost, LightGBM, and CatBoost. Model training and hyperparameter tuning were systematically executed via Grid Search combined with 10-fold cross-validation within the training cohort to identify the optimal parameters and ensure broad generalizability.

To deeply explore the clinical landscape and inter-variable relationships of the optimal feature subset, a correlation matrix was constructed using Spearman’s rank correlation. Furthermore, an unsupervised hierarchical clustering heatmap was generated to visualize the standardized expression patterns (Z-scores) of these features across sepsis and non-sepsis patients. Beyond standard metric evaluations (AUC, accuracy, sensitivity, and specificity), the predictive performance of all nine ML algorithms was granularly dissected using confusion matrices across both the internal MIMIC-IV and external eICU cohorts, detailing the true positive, true negative, false positive, and false negative rates. To ensure transparency in algorithmic decision-making, the feature importance across all nine models was extracted, normalized, and visualized as a comparative bar chart, thereby confirming the consistent core factors driving sepsis risk across diverse mathematical architectures.

Model interpretation and clinical utility

To elucidate the “black-box” decision-making process of the optimal ML model, SHAP values were implemented to quantify both the global importance and local individual contributions of each predictor (16). Furthermore, RCS analysis was performed to validate the non-linear dose-response relationships between key continuous variables and sepsis risk (21). For direct clinical translation, a dynamic web-based nomogram calculator was developed based on the optimal predictive model. The detailed mathematical formulation driving the static nomogram and the web calculator is provided in Appendix 1. Finally, decision curve analysis (DCA) was performed to ascertain the net clinical benefit of utilizing this nomogram across a spectrum of high-risk threshold probabilities.

Statistical analysis

All aforementioned statistical analyses and visualizations were performed using R software (version 4.5.1). The complete R scripts used for the ML framework and statistical analyses are available in Appendix 2. A two-sided P value <0.05 was considered statistically significant.

Results

Baseline characteristics of the study population

A total of 4,649 eligible patients with pleural effusion from the MIMIC-IV database were enrolled in this study (Figure 1), including 2,482 males (53%) and 2,167 females (47%), with a median age of 69 years [interquartile range (IQR), 59–79 years]. The primary outcome, ICU-acquired sepsis, occurred in 2,960 patients, indicating a high incidence rate of 63.7% in this specific cohort. The baseline demographic, clinical, and laboratory characteristics of the patients, stratified by the occurrence of sepsis, are comprehensively detailed in Table 1.

Figure 1.

Study flowchart. Flow diagram detailing the step-by-step selection process of adult patients with pleural effusion from the internal MIMIC-IV and external eICU databases. AUC, area under the curve; DCA, decision curve analysis; GBM, gradient boosting machine; HIV, human immunodeficiency virus; ICU, intensive care unit; LR, logistic regression; KNN, K-nearest neighbors; MIMIC-IV, Medical Information Mart for Intensive Care IV; NN, neural network; RCS, restricted cubic spline; SHAP, Shapley Additive exPlanations; SVM, support vector machine; XGBoost, eXtreme gradient boosting.

Table 1. Baseline demographic and clinical characteristics of patients stratified by ICU-acquired sepsis.

Variables	Total [n=4,649]	Non-sepsis [n=1,689]	Sepsis [n=2,960]	P
Demographics
Age, years	69 [59–79]	70 [60–79]	69 [58–79]	0.17
Gender				0.008
Female	2,167 [47]	831 [49]	1,336 [45]
Male	2,482 [53]	858 [51]	1,624 [55]
Height, cm	168 [160–175]	168 [160–175]	168 [160–175]	0.31
Weight, kg	76 [63.85–91.25]	75 [63.43–90]	76.85 [64–92.15]	0.04
Comorbidities
Hyperlipidemia				<0.001
No	2,938 [63]	966 [57]	1,972 [67]
Yes	1,711 [37]	723 [43]	988 [33]
Stroke				0.18
No	4,292 [92]	1,547 [92]	2,745 [93]
Yes	357 [8]	142 [8]	215 [7]
Tuberculosis				0.001
No	4,499 [97]	1,654 [98]	2,845 [96]
Yes	150 [3]	35 [2]	115 [4]
Pneumonia				<0.001
No	2,836 [61]	1,245 [74]	1,591 [54]
Yes	1,813 [39]	444 [26]	1,369 [46]
Cirrhosis				<0.001
No	4,211 [91]	1,598 [95]	2,613 [88]
Yes	438 [9]	91 [5]	347 [12]
Acute kidney injury				<0.001
No	2,732 [59]	1,167 [69]	1,565 [53]
Yes	1,917 [41]	522 [31]	1,395 [47]
Chronic kidney disease				0.15
No	3,776 [81]	1,391 [82]	2,385 [81]
Yes	873 [19]	298 [18]	575 [19]
Malignant tumor				<0.001
No	3,518 [76]	1,215 [72]	2,303 [78]
Yes	1,131 [24]	474 [28]	657 [22]
Myocardial infarction				0.70
No	4,325 [93]	1,575 [93]	2,750 [93]
Yes	324 [7]	114 [7]	210 [7]
Heart failure				0.01
No	3,365 [72]	1,259 [75]	2,106 [71]
Yes	1284 [28]	430 [25]	854 [29]
Diabetes mellitus				0.62
No	3,364 [72]	1,230 [73]	2,134 [72]
Yes	1,285 [28]	459 [27]	826 [28]
Hypertension				<0.001
No	2,780 [60]	955 [57]	1,825 [62]
Yes	1,869 [40]	734 [43]	1,135 [38]
COPD				0.07
No	3,825 [82]	1,413 [84]	2,412 [81]
Yes	824 [18]	276 [16]	548 [19]
Vital signs
Heart rate, beats/min	91 [79–108]	90 [80–106]	92 [79–108]	0.03
Respiratory rate, breaths/min	20 [16–24]	19 [16–24]	20 [16–24]	0.38
SpO2, %	97 [95–100]	97 [95–100]	98 [95–100]	0.055
Temperature, °C	36.72 [36.44–37.06]	36.72 [36.44–37]	36.72 [36.44–37.11]	0.71
Laboratory markers
WBC, ×109/L	11.6 [8–16.4]	10.9 [7.8–14.9]	12.1 [8.2–17.2]	<0.001
Platelets, ×109/L	203 [135–297]	214 [147–306]	198 [128–294]	<0.001
Hemoglobin, g/dL	9.8 [8.4–11.4]	9.9 [8.6–11.4]	9.7 [8.4–11.3]	0.04
RDW, %	15.1 [13.8–16.9]	14.6 [13.5–16.5]	15.3 [14–17.1]	<0.001
MCH, pg	29.9 [28.3–31.5]	29.9 [28.3–31.4]	29.9 [28.3–31.5]	0.29
MCHC, g/dL	32.5 [31.4–33.5]	32.5 [31.5–33.5]	32.5 [31.3–33.6]	0.20
MCV, fL	92 [87–96]	92 [87–96]	92 [87.75–97]	0.051
Creatinine, mg/dL	0.9 [0.7–1.5]	0.9 [0.6–1.2]	1 [0.7–1.6]	<0.001
Glucose, mmol/L	6.89 [5.67–8.67]	6.67 [5.67–8.17]	7.06 [5.72–9]	<0.001
Anion gap, mmol/L	14 [11–16]	13 [11–15]	14 [12–17]	<0.001
Bicarbonate, mmol/L	23 [20–26]	24 [21–26]	23 [20–26]	<0.001
Chloride, mmol/L	103 [99–108]	103 [99–107]	104 [99–108]	<0.001
Sodium, mmol/L	138 [135–141]	138 [135–140]	138 [135–141]	<0.001
PT, s	14.9 [13.1–18]	14.5 [12.9–17]	15.3 [13.3–18.5]	<0.001
Albumin, g/dL	2.8 [2.4–3.2]	2.9 [2.5–3.3]	2.8 [2.4–3.2]	<0.001
Treatments and medications
Antihypertensives				<0.001
No	883 [19]	376 [22]	507 [17]
Yes	3,766 [81]	1,313 [78]	2,453 [83]
Vasopressors				<0.001
No	1,781 [38]	913 [54]	868 [29]
Yes	2,868 [62]	776 [46]	2,092 [71]
Glucocorticoids				<0.001
No	3,284 [71]	1,268 [75]	2,016 [68]
Yes	1,365 [29]	421 [25]	944 [32]
Sedatives and analgesics				<0.001
No	1,636 [35]	849 [50]	787 [27]
Yes	3,013 [65]	840 [50]	2,173 [73]
Immunosuppressants				0.03
No	4,441 [96]	1,629 [96]	2,812 [95]
Yes	208 [4]	60 [4]	148 [5]
CRRT				<0.001
No	4,333 [93]	1,663 [98]	2,670 [90]
Yes	316 [7]	26 [2]	290 [10]
Mechanical ventilation				<0.001
No	408 [9]	222 [13]	186 [6]
Yes	4,241 [91]	1,467 [87]	2,774 [94]
Severity scores
SOFA	5 [3–7]	3 [1–5]	6 [4–8]	<0.001

Data are presented as median [interquartile range] or n [%]. COPD, chronic obstructive pulmonary disease; CRRT, continuous renal replacement therapy; ICU, intensive care unit; MCH, mean corpuscular hemoglobin; MCHC, mean corpuscular hemoglobin concentration; MCV, mean corpuscular volume; PT, prothrombin time; RDW, red cell distribution width; SOFA, sequential organ failure assessment; SpO₂, peripheral oxygen saturation; WBC, white blood cell.

Univariate analysis revealed profound physiological and interventional disparities between the two cohorts. Patients who developed sepsis exhibited a significantly higher baseline severity of illness, as evidenced by markedly elevated SOFA scores (median 6 vs. 3, P<0.001). Regarding comorbidities and infection sources, the sepsis cohort presented a significantly higher prevalence of pneumonia (46% vs. 26%, P<0.001) and acute kidney injury (47% vs. 31%, P<0.001), while interestingly showing a lower proportion of preexisting hyperlipidemia (33% vs. 43%, P<0.001).

In terms of laboratory markers, the sepsis group demonstrated significant systemic inflammation and metabolic derangement, characterized by higher white blood cell (WBC) counts (median 12.1 vs. 10.9 ×10⁹/L, P<0.001) and anion gap levels (median 14 vs. 13 mEq/L, P<0.001), alongside significantly lower serum albumin levels (median 2.8 vs. 2.9 g/L, P<0.001) and platelet counts (median 198 vs. 214 ×10⁹/L, P<0.001). Expectedly, the demand for high-intensity ICU interventions was overwhelmingly higher in the sepsis group, including the utilization of vasopressors (71% vs. 46%, P<0.001), sedatives and analgesics (73% vs. 50%, P<0.001), mechanical ventilation (94% vs. 87%, P<0.001), and continuous renal replacement therapy (CRRT) (10% vs. 2%, P<0.001). These substantial baseline differences underscore the necessity for advanced multidimensional modeling to capture the complex trajectory of sepsis onset. Furthermore, an independent external validation cohort comprising 2,870 patients from the eICU database was analyzed, among which 1,670 patients (57.1%) developed sepsis.

Optimal feature selection and clinical landscape

To rigorously identify the most critical predictors of sepsis from the high-dimensional clinical data, a tri-algorithm ML intersection strategy was deployed. As illustrated in Figure 2, the Boruta algorithm initially filtered the variables by confirming their importance against shadow features (Figure 2A,2B). Subsequently, the LASSO regression model, optimized via 10-fold cross-validation, compressed the coefficients to counteract multicollinearity (Figure 2C,2D). Concurrently, the RFE algorithm determined the optimal subset of features that maintained high predictive accuracy (Figure 2E). By taking the precise intersection of these three distinct methodologies, a robust core set of 10 critical variables was locked in: SOFA score, vasopressors, sedatives and analgesics, mechanical ventilation, CRRT, anion gap, WBC count, pneumonia, hyperlipidemia, and serum albumin (Figure 2F).

Figure 2.

Optimal feature selection via a tri-algorithm intersection strategy. (A) Box plot of variable importance using the Boruta algorithm. (B) Boruta feature filtering importance score plot across classifier runs. (C) The LASSO coefficient profiles of the clinical features against the λ sequence. (D) Selection of the optimal penalization coefficient (λ) in the LASSO model via 10-fold cross-validation. (E) RFE indicating the optimal number of variables based on cross-validation accuracy. (F) A Venn diagram illustrating the intersection of the robust predictors uniformly identified by Boruta, LASSO, and RFE, resulting in a core subset of 10 features. LASSO, least absolute shrinkage and selection operator; RFE, recursive feature elimination.

To decipher the clinical landscape of these 10 core features, a Spearman correlation matrix and an unsupervised hierarchical clustering heatmap were constructed (Figure 3). The correlation analysis revealed moderate positive associations among critical care interventions and organ failure metrics (e.g., SOFA and vasopressors, r=0.41; SOFA and sedatives/analgesics, r=0.32), reflecting the complex interplay of physiological deterioration. More importantly, the Z-score normalized heatmap visually delineated distinct expression patterns between cohorts: patients who ultimately developed sepsis exhibited visibly denser clusters of high-expression indicators (red zones) for SOFA, anion gap, WBC, and intensive interventions compared to their non-sepsis counterparts, validating the strong discriminative potential of this 10-feature subset.

Figure 3.

Clinical landscape and feature correlations. (A) Spearman correlation matrix of the 10 core features. The matrix employs a mixed-style layout with a colorblind-friendly viridis palette. The upper right triangle visualizes the correlation magnitude using varying circle sizes and color intensities, while the lower left triangle presents the exact Spearman correlation coefficients. Positive correlations are displayed in lighter/yellow colors and negative correlations in darker/purple colors. (B) Unsupervised hierarchical clustering and risk-sorted heatmap illustrating the distribution of continuous clinical characteristics (Z-score standardized) and categorical interventions across different sepsis risk probabilities. CRRT, continuous renal replacement therapy; SOFA, sequential organ failure assessment; WBC, white blood cell.

Performance comparison of nine ML algorithms

To identify the most robust predictive engine for sepsis, the 10 core features were fed into nine distinct supervised ML algorithms. Their discriminative performance, calibration, and clinical utility were comprehensively evaluated across both internal (MIMIC-IV) and external (eICU) validation cohorts (Figure 4).

Figure 4.

Comprehensive performance evaluation of the nine machine learning algorithms. The left column illustrates the model performance in the internal validation cohort (MIMIC-IV), displaying the ROC curves (A), DCA curves (B), and calibration curves (C). The right column presents the corresponding performance in the external validation cohort (eICU), including the ROC curves (D), DCA curves (E), and calibration curves (F). The evaluated models include logistic regression, SVM, GBM, neural network, KNN, XGBoost, AdaBoost, LightGBM, and CatBoost. AUC, area under the curve; CI, confidence interval; DCA, decision curve analysis; GBM, gradient boosting machine; KNN, K-nearest neighbors; MIMIC-IV, Medical Information Mart for Intensive Care IV; ROC, receiver operating characteristic; SVM, support vector machine; XGBoost, eXtreme gradient boosting.

In the internal validation phase, the models exhibited fierce competition. The LR, GBM, and XGBoost models achieved the highest identical area under the curve (AUC) of 0.770 (Figure 4A). However, the true test of algorithmic robustness lies in its generalizability to unseen data. In the external eICU cohort, the LR model emerged as the superior algorithm, achieving the highest AUC of 0.705 (95% CI: 0.686–0.724), effectively outperforming complex tree-based and NN models that typically suffer from overfitting (Figure 4D).

Furthermore, to justify the clinical efficiency of our feature selection strategy, we compared the predictive performance of the final optimized model (10 core variables) against a “Full Model” incorporating all initially extracted 46 baseline variables. As detailed in Table S1, despite a drastic dimensionality reduction from 46 to 10 variables, the optimized model exhibited no statistically significant drop in discriminative power compared to the full model (AUC: 0.770 vs. 0.773; DeLong’s test P=0.67). This confirms that the 10-feature subset successfully captures the essential prognostic information while significantly reducing the data collection burden at the bedside, mitigating overfitting risks, and enhancing model interpretability.

Beyond discrimination, the practical value of the models was granularly dissected. Confusion matrices (Figure 5) confirmed that the Logistic model maintained a stable and highly balanced true positive and true negative rate across both internal and external cohorts (Figure 5A,5B), minimizing dangerous false-negative misclassifications in critical care settings. Furthermore, calibration curves (Figure 4C,4F) demonstrated that the predicted probabilities of the logistic model closely hugged the ideal 45-degree diagonal line, proving its superior “honesty” in risk estimation without significant overestimation or underestimation. Concurrently, DCA (Figure 4B,4E) revealed that the Logistic model consistently provided the highest standardized net benefit across the widest range of high-risk thresholds compared to other algorithms and the “treat-all/treat-none” baseline scenarios.

Figure 5.

Comparison of predictive performance via confusion matrices. Granular visualization of the classification accuracy for all nine machine learning models. The matrices detail the true positive, true negative, false positive, and false negative rates in the internal validation cohort (A) and the external validation cohort (B), highlighting the stable and balanced performance of the logistic model in minimizing false-negative misclassifications. GBM, gradient boosting machine; KNN, K-nearest neighbors; MIMIC-IV, Medical Information Mart for Intensive Care IV; SVM, support vector machine; XGBoost, eXtreme gradient boosting.

To ensure transparency and confirm biological plausibility across different mathematical architectures, the feature importance was extracted and normalized across all nine models (Figure 6). Remarkably, despite their fundamentally different underlying algorithms, SOFA score, pneumonia, and the use of sedatives/analgesics were consistently ranked as the top driving factors for sepsis risk across almost all models. This cross-algorithmic consensus strongly validates the robustness of our selected features. Consequently, given its unparalleled external generalizability, excellent calibration, and inherent interpretability, the LR model was definitively selected as the optimal predictive model for subsequent clinical translation. Furthermore, we systematically compared our optimal ML model with the traditional SOFA score. DeLong’s test revealed that the ML model achieved a statistically significantly higher AUC [0.770, 95% confidence interval (CI): 0.740–0.800] compared to the SOFA score alone (0.729, 95% CI: 0.696–0.761; P<0.001). Importantly, the DCA comparing the ML model to the SOFA score demonstrated a consistently higher net clinical benefit for the ML model across primary decision-making thresholds (Figure S1), thereby confirming its superior clinical utility at the bedside.

Figure 6.

Feature importance across nine machine learning algorithms. The bar charts display the normalized importance scores of the 10 core predictors to ensure transparency and confirm biological plausibility across different mathematical architectures. The evaluated models include logistic regression, SVM, GBM, neural network, KNN, XGBoost, AdaBoost, LightGBM, and CatBoost. CRRT, continuous renal replacement therapy; GBM, gradient boosting machine; KNN, K-nearest neighbors; SOFA, sequential organ failure assessment; SVM, support vector machine; WBC, white blood cell; XGBoost, eXtreme gradient boosting.

Model interpretability and dose-response validation

To demystify the decision-making process of the optimal LR model and enhance its clinical transparency, SHAP values were employed (Figure 7). The SHAP summary bar plot (Figure 7A) and beeswarm plot (Figure 7B) delineated the global feature importance and the directional impact of each variable on sepsis risk. Consistent with clinical pathophysiology, higher values (red dots) of the SOFA score, anion gap, and clinical interventions (pneumonia, vasopressors, sedatives/analgesics, CRRT, mechanical ventilation) strongly propelled the model towards a positive sepsis prediction. Conversely, hyperlipidemia and serum albumin acted as protective factors, effectively lowering the predicted risk.

Figure 7.

Model interpretation using SHAP. (A) The SHAP summary bar plot delineating the global feature importance based on mean absolute SHAP values. (B) The SHAP beeswarm plot illustrating the directional impact of each variable on sepsis risk, where the color gradient represents the feature value. (C) SHAP waterfall plot and (D) force plot demonstrating the local interpretability of the model, explicitly showing how an individual patient’s specific clinical parameters incrementally adjust the baseline risk to reach the final predictive probability. (E) SHAP dependence plots exploring the precise clinical thresholds and non-linear, dose-dependent impacts of individual continuous features on the model’s output. CRRT, continuous renal replacement therapy; SHAP, Shapley Additive exPlanations; SOFA, sequential organ failure assessment; WBC, white blood cell.

Crucially, SHAP dependence plots (Figure 7E) were generated to explore the precise clinical thresholds and non-linear effects of individual features on the model’s output. These plots revealed distinct risk trajectories; for instance, the SHAP value for the SOFA score sharply transitioned from negative to positive as the score exceeded 4–5, indicating a critical risk inflection point. Similarly, continuous variables like anion gap and albumin exhibited clear dose-dependent impacts, perfectly foreshadowing the subsequent traditional statistical validations. Furthermore, the SHAP waterfall plot (Figure 7C) and force plot (Figure 7D) successfully demonstrated the local interpretability of the model, illustrating how the baseline risk is incrementally adjusted by an individual patient’s specific clinical parameters to reach the final predictive probability.

To validate these ML-derived insights within a traditional epidemiological framework, a multivariable LR forest plot and RCS analyses were conducted (Figure 8). The forest plot (Figure 8A) corroborated the SHAP findings, confirming features like pneumonia [odds ratio (OR): 2.19, P<0.001] and SOFA score (OR: 1.23, P<0.001) as significant independent risk factors. Crucially, the RCS analyses unveiled the complex, non-linear dose-response relationships that linear models often obscure. Matching the SHAP dependence plots, the risk of sepsis escalated sharply as the SOFA score increased from 0 to 5, plateauing thereafter (Figure 8E). Most remarkably, the RCS curve for WBC count (Figure 8C) exhibited a distinct “U-shaped” relationship. While WBC appeared statistically non-significant in the linear model (P=0.43, Figure 8A), the RCS plot clearly revealed that both severe leukopenia and pronounced leukocytosis drastically amplified the risk of sepsis, perfectly mirroring the complex immunological responses encountered in real-world critical care settings.

Figure 8.

Traditional epidemiological validation and non-linear dose-response analyses. (A) Forest plot of the multivariable logistic regression model displaying the independent ORs, 95% CIs, and P values for the 10 core predictors. (B-E) RCS curves illustrating the adjusted non-linear dose-response associations between continuous clinical variables and the risk of sepsis: (B) Anion gap, (C) WBC count, (D) Albumin, and (E) SOFA score. The solid curves represent the estimated ORs, while the shaded areas indicate the 95% CIs. The reference OR of 1.0 is marked by the dashed horizontal line. Notably, the RCS curve for WBC count (C) reveals a distinct “U-shaped” relationship, and the SOFA score (E) demonstrates a sharp risk escalation that plateaus after a score of 5. CI, confidence interval; CRRT, continuous renal replacement therapy; OR, odds ratio; RCS, restricted cubic spline; SOFA, sequential organ failure assessment; WBC, white blood cell.

Nomogram construction and clinical utility

To facilitate the bedside application of the optimal LR model for critical care physicians, a visual static nomogram and a dynamic web-based calculator were constructed based on the 10 identified core predictors (Figure 9). The static nomogram (Figure 9A) translates the complex mathematical equations into a user-friendly scoring system, where each clinical variable is assigned a specific point value on a dedicated axis. By summing the points across all variables to derive a “Total Points” score, clinicians can easily estimate the individualized probability of sepsis onset by drawing a vertical line down to the “Risk of Sepsis” axis. To further enhance clinical accessibility, a dynamic online calculator (Figure 9B, freely accessible at https://guanghao.shinyapps.io/Sepsis_Predictor/) was developed, enabling real-time, precise risk computation by simply inputting the patient’s parameters into a web interface.

Figure 9.

Visual prediction tools for estimating individualized sepsis risk based on the optimal logistic regression model. (A) A static nomogram incorporating 10 identified core predictors. This visual tool translates complex mathematical equations into a user-friendly scoring system. Clinicians can estimate a patient’s probability of sepsis by assigning points to each clinical variable, summing them to calculate the “Total Points”, and drawing a vertical line down to the “Risk of Sepsis” axis. For categorical variables, "0" denotes “no” (absence) and “1” indicates “yes” (presence). (B) A dynamic web-based calculator interface. Developed to enhance clinical accessibility, this online tool (freely accessible at https://guanghao.shinyapps.io/Sepsis_Predictor/) enables real-time, precise risk computation by allowing users to input patient parameters directly via interactive sliders. CRRT, continuous renal replacement therapy; SOFA, sequential organ failure assessment; WBC, white blood cell.

The clinical utility and reliability of this nomogram were rigorously evaluated through calibration and DCA in both the internal (MIMIC-IV) and external (eICU) cohorts (Figure 10). The calibration curves (Figure 10C,10D) revealed a high degree of concordance between the nomogram-predicted probabilities and the actual observed sepsis frequencies, particularly in the internal validation cohort where the calibration line closely adhered to the ideal 45-degree reference line (Brier score =0.183).

Figure 10.

Evaluation of the clinical utility and reliability of the predictive nomogram. (A,B) DCA evaluating the net clinical benefit of the nomogram in the internal (MIMIC-IV) (A) and external (eICU) (B) validation cohorts. The nomogram curves (red and green lines) consistently remain above the “treat-all” and “treat-none” reference lines across a wide range of high-risk thresholds (approximately 0 to >0.8), indicating substantial net clinical benefit for guiding interventions. (C,D) Calibration curves assessing the concordance between the predicted probabilities and actual observed sepsis frequencies in the internal (C) and external (D) cohorts. The curves demonstrate a high degree of agreement, particularly in the internal validation cohort (Brier score =0.183), where the calibration line closely adheres to the ideal 45-degree reference line. Dxy, Somers’ D rank correlation; C (ROC), concordance statistic; R2, Nagelkerke R-squared index; D, discrimination index; U, unreliability index; Q, quality index; Emax, maximum absolute error; E90, 90th percentile of absolute error; Eavg, average absolute error; S:z and S:p, Spiegelhalter’s Z-statistic and corresponding P value. DCA, decision curve analysis; MIMIC-IV, Medical Information Mart for Intensive Care IV; ROC, receiver operating characteristic.

Crucially, the DCA (Figure 10A,10B) proved the immense practical value of the model. In both the internal and external validation sets, the decision curve for the nomogram (red and green lines) consistently positioned itself significantly above both the “treat-all” and “treat-none” reference lines across an exceptionally wide range of high-risk thresholds (approximately from 0 to >0.8). This indicates that utilizing this nomogram to guide early clinical interventions (e.g., proactive antibiotic escalation or intensive monitoring) for pleural effusion patients in the ICU will yield a substantial net clinical benefit compared to arbitrary decision-making strategies.

Discussion

The primary aim of this study was to develop and externally validate an interpretable ML framework for predicting ICU-acquired sepsis specifically in patients with pleural effusion. To the best of our knowledge, this is the first multicenter study to comprehensively evaluate and deploy such a model. By rigorously intersecting Boruta, LASSO, and RFE algorithms—a hybrid ensemble feature selection strategy increasingly recognized for reducing dimensional noise in complex biomedical datasets (22)—we successfully distilled 10 robust core predictors from a high-dimensional clinical dataset. Subsequent rigorous algorithm competition demonstrated that the LR model possessed the optimal balance of discriminative power and external generalizability. More importantly, by integrating SHAP and RCS analyses, we not only demystified the algorithmic “black box” but also unveiled critical non-linear dose-response relationships—such as the U-shaped trajectory of WBC counts—that are often overlooked by traditional linear models. The final deployment of a dynamic nomogram, validated by DCA for superior clinical net benefit, provides intensive care physicians with an actionable and trustworthy bedside tool.

The core strength of our study lies in the deployment of a robust and highly transparent ML pipeline to decipher the multidimensional pathophysiology of sepsis in pleural effusion patients. Traditional predictive modeling often relies on subjective clinical judgment or single-algorithm screening, which is highly susceptible to overfitting and collinearity biases in critical care settings (23). By employing a tri-algorithm intersection strategy (Boruta, LASSO, and RFE), we systematically eliminated redundant variables and locked in a parsimonious subset of 10 critical predictors. Interestingly, during the comprehensive algorithmic competition, the LR model effectively outperformed complex “black-box” models, such as XGBoost and NNs, particularly in the external validation cohort. This observation aligns with recent clinical data science consensus suggesting that in well-engineered structured tabular data, regularized linear models often provide superior generalizability by penalizing extreme decision boundaries (24). The integration of interpretable frameworks is crucial for overcoming the inherent ‘black-box’ nature of traditional ML, a necessity highlighted by recent studies for real-time ICU triage (25). Furthermore, the integration of SHAP values thoroughly demystified the predictive mechanics, addressing the critical need for “XAI” to foster clinical trust in life-and-death triage settings (26). The global SHAP analysis unequivocally identified the baseline SOFA score, the presence of pneumonia, and the preemptive use of sedatives and analgesics as the top-tier drivers of sepsis risk. While the predictive weight of SOFA and pneumonia is pathophysiologically intuitive, reflecting advanced organ dysfunction and primary infectious burdens (1), the high SHAP importance of sedatives/analgesics highlights a crucial clinical proxy: patients requiring profound sedation are typically experiencing severe respiratory or hemodynamic compromise, directly predisposing them to prolonged mechanical ventilation and secondary ICU-acquired infections (27).

In addition to clinical interventions, our interpretable model highlighted the profound prognostic value of metabolic and nutritional biomarkers, specifically the anion gap and serum albumin. The anion gap, a traditional marker of metabolic acidosis, exhibited a strong positive correlation with sepsis risk in both SHAP and RCS analyses. In the context of critical illness, where acid-base derangements often precede overt organ failure (28), an elevated anion gap is often a reliable surrogate for occult tissue hypoperfusion and lactic acid accumulation, serving as an early radar for microcirculatory dysfunction even before macro-hemodynamic collapse becomes clinically apparent (29). This is particularly relevant for patients with massive pleural effusion, where dramatic alterations in intrathoracic pressure can further compromise venous return and cardiac output, exacerbating systemic hypoxia (30). Conversely, serum albumin emerged as a crucial protective factor. Moreover, the inclusion of routine laboratory markers such as albumin in our predictive framework aligns strongly with recent MIMIC-IV-based investigations, which continue to emphasize the profound prognostic value of nutritional and inflammatory indices in septic patients (31). Hypoalbuminemia in these cohorts is not merely a reflection of baseline malnutrition; more importantly, it signifies acute systemic inflammation leading to increased capillary permeability and the degradation of the endothelial glycocalyx (32). The rapid extravasation of albumin not only exacerbates the accumulation of pleural fluid but also compromises the transport of endogenous ligands and therapeutic agents, ultimately accelerating the trajectory toward septic shock (33). Thus, monitoring the dynamic interplay between the anion gap and albumin offers intensive care physicians a critical window into the patient’s subclinical metabolic deterioration.

To bridge the gap between algorithmic intelligence and traditional clinical epidemiology, we performed RCS analysis, which yielded perhaps the most striking pathophysiological insight of our study: a distinct non-linear, “U-shaped” dose-response relationship between baseline WBC count and sepsis risk. Traditional linear multivariable models often obscure this biphasic clinical nature, misinterpreting the offsetting extremes as statistical insignificance (as observed in our linear forest plot, P=0.43). On the right extreme of the RCS curve, pronounced leukocytosis monotonically amplifies the risk of sepsis. In the context of pleural effusion, this typically mirrors a hyperactive immune cascade and severe systemic inflammatory response syndrome (SIRS) driven by massive bacterial loads, such as in complicated parapneumonic effusions or frank empyema (34). Conversely, and perhaps more dangerously, the left-sided tail of the U-curve reveals a precipitous escalation in sepsis probability when WBC counts drop to or below the lower normal limits. In critical care settings, sepsis-associated leukopenia is a well-documented harbinger of profound immunosuppression and catastrophic prognosis (35). This phenomenon often signifies critical immune exhaustion and the onset of compensatory anti-inflammatory response syndrome (CARS) (36), overwhelming bone marrow suppression, or massive leukocyte margination—where circulating neutrophils rapidly adhere to the vascular endothelium in response to severe Gram-negative bacteremia, leading to a deceptive drop in measurable WBCs despite a systemic infectious storm (37). Consequently, our non-linear findings strongly caution clinicians against the false reassurance of a “normal or low” WBC count in pleural effusion patients, highlighting a highly vulnerable sub-population that mandates immediate and aggressive surveillance.

Beyond elucidating pathophysiological mechanisms, the ultimate objective of clinical predictive modeling is bedside applicability. While ML algorithms excel at capturing high-dimensional patterns, their inherent “black-box” nature often impedes clinical trust. To overcome this, we translated the optimal LR model into an intuitive static nomogram and a dynamic web-based calculator. This dual-format deployment allows clinicians to rapidly quantify individual sepsis risk using readily available routine parameters, irrespective of their computational expertise. More importantly, the DCA convincingly demonstrated the clinical utility of our tool. Unlike traditional metrics like the AUC that merely assess statistical discrimination, DCA quantifies the actual clinical consequences of false positives and false negatives (38). In both the internal and external validation cohorts, nomogram-guided interventions consistently provided a superior net clinical benefit compared to arbitrary “treat-all” or “treat-none” strategies across a broad spectrum of risk thresholds. This suggests that deploying our model as an early warning system in the ICU could effectively optimize resource allocation, guiding preemptive antibiotic escalation or intensive monitoring for high-risk pleural effusion patients while avoiding unnecessary interventions in low-risk individuals.

Several limitations of our study must be acknowledged. First, given the retrospective observational design utilizing the MIMIC-IV and eICU databases, inherent selection biases and missing data cannot be entirely eliminated, and causality cannot be definitively established. Second, our model relies exclusively on baseline variables recorded upon ICU admission. Sepsis is a highly dynamic syndrome, and incorporating longitudinal trajectories of inflammatory markers over time might further enhance predictive accuracy. Third, specific unmeasured confounders, such as the precise microbial etiology of the pleural effusion (39) and the timing of pre-ICU antibiotic administration, were not fully captured in the current dataset. Future prospective, multicenter studies incorporating dynamic multi-omics data (40) are warranted to externally validate and refine our findings. Ultimately, the core clinical message we intend to send is that utilizing this highly transparent, data-driven early warning system can assist physicians in proactively identifying high-risk individuals, thereby optimizing bedside personalized interventions and improving patient outcomes.

Conclusions

In summary, we successfully developed and externally validated a highly interpretable ML framework for predicting ICU-acquired sepsis in patients with pleural effusion. By identifying 10 critical predictors and unveiling vital non-linear relationships (e.g., the U-shaped impact of WBCs), our study bridges the gap between algorithmic performance and clinical transparency. The deployed nomogram and web calculator offer a robust, data-driven tool to assist critical care physicians in personalized risk stratification and timely clinical decision-making.

Supplementary

The article’s supplementary files as

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study utilized de-identified retrospective data from the publicly accessible MIMIC-IV and eICU databases. Given the anonymized nature of the data, the requirement for individual patient consent and formal ethical approval was waived by the institutional review boards of the PhysioNet platform.