Development and multicenter validation of machine learning models for 28-day mortality in critically ill patients with acute exacerbation of chronic obstructive pulmonary disease

Abstract

Background

Acute exacerbation of chronic obstructive pulmonary disease (AECOPD) is a common critical illness in the intensive care unit (ICU) and is associated with poor outcomes. Marked clinical heterogeneity limits the ability of conventional scoring systems to achieve accurate early mortality risk stratification. This study aimed to develop and validate machine learning (ML) models for early prediction of 28-day mortality in critically ill patients with AECOPD.

Methods

In this multicentre retrospective study, ML models were developed using the Medical Information Mart for Intensive Care IV database and externally validated in the eICU Collaborative Research Database. Adult patients with AECOPD undergoing first ICU admission were included, and routinely available clinical variables within the first 24 h after ICU admission were extracted. Least absolute shrinkage and selection operator regression was used for feature selection, and nine ML models were developed. Model performance was assessed in terms of discrimination, precision–recall performance, calibration, and clinical utility, and compared with conventional clinical scores. Class weighting was used in the primary analysis, while SMOTE and additional sensitivity analyses were performed to assess robustness.

Results

A total of 715 patients were included, of whom 84 (11.7%) died within 28 days. LASSO selected 15 predictive variables. In the internal test set, support vector machine achieved the highest AUC of 0.722. In the external validation cohort, LightGBM achieved the highest AUC of 0.764 and F1 score, GBM showed the highest sensitivity, XGBoost achieved the highest negative predictive value and balanced accuracy, and random forest achieved the highest PR-AUC. Calibration performance varied across models and datasets. Compared with conventional clinical scores, several ML models showed more favourable numerical performance, although no single model consistently outperformed the others across all evaluation dimensions. Sensitivity analyses suggested that the main conclusions were broadly robust.

Conclusions

ML models based on routinely available ICU data showed exploratory potential for 28-day mortality risk stratification in critically ill patients with AECOPD. Given the limited number of death events, class imbalance, and risk of overfitting, these findings should be interpreted cautiously. The models are not ready for clinical implementation and require further prospective validation, local recalibration, and evaluation within real-world clinical workflows.

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s12890-026-04478-7.

Background

Acute exacerbation of chronic obstructive pulmonary disease (AECOPD) is one of the leading causes of hospitalization and mortality worldwide and represents a common indication for admission to the intensive care unit (ICU) [1, 2]. In critically ill patients with AECOPD requiring ICU care, the acute phase is frequently characterized by marked aggravation of airway inflammation, ventilation–perfusion mismatch, and respiratory muscle fatigue. A substantial proportion of patients rapidly progress to acute respiratory failure (ARF), necessitating non-invasive or invasive mechanical ventilation [3]. Despite ongoing advances in respiratory support technologies and comprehensive therapeutic strategies, the short-term prognosis of critically ill patients with AECOPD remains poor, with persistently high 28-day mortality rates [4, 5]. Owing to pronounced heterogeneity in baseline pulmonary function, comorbidity profiles, and acute stress responses, reliance on conventional severity scoring systems or single clinical indicators alone often fails to enable timely and accurate identification of patients at high risk of death, thereby limiting the implementation of early risk stratification and precision-targeted interventions [6].

With the widespread adoption of electronic health record systems and the establishment of large-scale critical care databases, machine learning (ML) approaches have been increasingly applied to prognostic research in critically ill populations [7, 8]. Compared with traditional statistical models, ML algorithms are capable of simultaneously handling high-dimensional clinical variables, complex non-linear relationships, and interactions between predictors, and have demonstrated potential advantages in predicting mortality risk among ICU patients in multiple studies [7]. Since 2024, ML-based research has continued to expand across a range of critical care subgroups, including sepsis, acute respiratory distress syndrome (ARDS), and cardiovascular critical illness, suggesting that the role of ML in prognostic assessment among critically ill patients is still evolving [9, 10].

However, most existing studies have focused either on broader and more heterogeneous general ICU populations or on specific syndromes such as sepsis and ARDS. By contrast, studies that systematically compare multiple ML models and perform rigorous external validation in patients with AECOPD remain scarce, particularly in the context of short-term mortality prediction [11]. Previous studies have explored the use of ML to predict ICU admission and in-hospital mortality among hospitalised patients with AECOPD. Nevertheless, these studies primarily involved the overall hospitalised population, and their outcomes were largely limited to in-hospital mortality [12, 13]. Another study developed a nomogram based on the Medical Information Mart for Intensive Care IV (MIMIC-IV) database to predict short-term mortality in patients with COPD and chronic heart failure, but the study population was confined to a specific comorbid subgroup and the modelling strategy relied primarily on conventional regression methods [14]. In addition, an interpretable Light Gradient Boosting Machine (LightGBM) model incorporating echocardiographic variables has been developed to predict in hospital mortality in patients with AECOPD. Nevertheless, the echocardiographic features required by this model generally depend on specialised training, additional examination time, and operator experience, which may limit their routine and standardized acquisition during the early phase of ICU admission [15]. Therefore, a systematic comparison of multiple ML models for predicting 28-day mortality in ICU patients with AECOPD, together with validation in an independent external dataset, remains of clear clinical and research significance.

Accordingly, this study used two large multicenter critical care databases, MIMIC-IV and the eICU Collaborative Research Database (eICU-CRD), to focus on patients with AECOPD admitted to the ICU. Using the first available clinical data obtained early after ICU admission, we developed and validated multiple ML models for predicting 28-day mortality after ICU admission in patients with AECOPD. By systematically comparing discrimination, calibration, and clinical utility across models, we aimed to evaluate the potential value and limitations of ML-based prediction models for early risk stratification in critically ill patients with AECOPD.

Methods

Study design and data sources

This multicenter retrospective study developed and internally validated prediction models using the MIMIC-IV (V 3.1) database, with independent external validation performed using the eICU-CRD.

MIMIC-IV is a large, publicly available critical care database developed and maintained by the Laboratory for Computational Physiology at the Massachusetts Institute of Technology. It contains de-identified clinical data from patients admitted to Beth Israel Deaconess Medical Center between 2008 and 2022 [16]. The eICU-CRD is a multicenter critical care database comprising ICU admissions from multiple hospitals across the United States and was used for external model validation [17].

Prior to data access, the research team completed the National Institutes of Health training on human subjects research and passed the Protecting Human Research Participants examination (Record ID: 62695673). As all data in both databases were fully de-identified, informed consent and additional institutional review board approval were not required.

Study population and outcome definition

Inclusion criteria: (1) age ≥ 18 years; (2) admission to the ICU with AECOPD as the primary diagnosis, identified using International Classification of Diseases, Ninth and Tenth Revision (ICD-9 and ICD-10) codes; (3) inclusion of only the first ICU admission during the index hospitalization; (4) an ICU length of stay exceeding 24 h.

Exclusion criteria: (1) the presence of malignant disease; (2) concomitant pulmonary embolism or respiratory failure attributable to clearly defined non-AECOPD causes; (3) asthma or asthma-related conditions as the primary diagnosis. These exclusion criteria were applied to minimize potential confounding from respiratory failure unrelated to AECOPD during model development.

The primary study outcome was the all-cause mortality rate within 28 days after admission to the ICU.

Data extraction

Based on the predefined inclusion and exclusion criteria, eligible patients with AECOPD were identified from the MIMIC-IV database using structured SQL queries, and their clinical data were extracted for model development and internal validation. Data required for external validation were obtained from the eICU-CRD using identical variable definitions and extraction strategies.

The extracted data comprised the following categories: (1) demographic characteristics, including age and sex; (2) comorbidities, identified from documented pre-existing diagnoses; (3) vital signs, such as heart rate, blood pressure, and respiratory rate; (4) laboratory variables, encompassing complete blood counts, biochemical parameters, electrolytes, and metabolism-related indices; and (5) disease severity scores, including the Acute Physiology Score III (APS III), Simplified Acute Physiology Score II (SAPS II), Sequential Organ Failure Assessment (SOFA) score, and Systemic Inflammatory Response Syndrome (SIRS) criteria, which were used to reflect overall illness severity at ICU admission.

To ensure a clearly defined prediction time point and minimise the risk of information leakage, all predictor variables were limited to the first recorded values obtained within 24 h of ICU admission. For vital signs and laboratory parameters, the first measurements recorded during this 24-hour window were extracted. For disease severity scores, the first scores documented within 24 h of ICU admission were used. The external validation cohort applied the same variable definitions, extraction windows, and aggregation rules as the development cohort. Detailed information on extraction time windows, aggregation rules, and variable mapping for the final predictors included in model training is presented in Supplementary Table S1.

Data splitting and information leakage prevention

After study population selection, the development cohort was randomly divided into a training set and an internal test set at a ratio of 7:3. To reduce the risk of information leakage, data splitting was performed before all subsequent data processing and model development procedures. Missing data handling, outlier control, feature selection, class imbalance treatment, hyperparameter tuning, model training, and classification threshold determination were all performed exclusively within the training set. The internal test set and external validation cohort were used only for final independent performance evaluation.

Missing data handling and outlier control

Before model development, missingness was quantified for each candidate variable in both the development and external validation cohorts, as detailed in Supplementary Tables S2 and S3. Variables with more than 30% missing data were excluded, whereas those with lower levels of missingness were retained for subsequent analyses. Missing data were handled using multiple imputation with the mice package in R. Predictive mean matching was used for continuous variables, and logistic regression was used for binary variables. Twenty imputed datasets were generated, with 20 iterations per dataset. To minimise information leakage, the imputation model was derived from the training data and then applied to the internal test set and external validation cohort. The outcome variable was not included in the imputation models. Model development was repeated within each imputed training dataset. For the internal test set and external validation cohort, predicted probabilities were averaged across the twenty imputed datasets for each individual, and model performance metrics were calculated using these pooled predictions.

Outliers in continuous variables were addressed using Winsorisation. The 1st and 99th percentile thresholds were derived from the training set and then applied to the internal test set and external validation cohort. The Winsorisation thresholds for each variable, along with the proportions of modified observations, are reported in Supplementary Table S4. To evaluate the impact of outlier handling on model performance, an additional sensitivity analysis was conducted by comparing model performance with and without Winsorisation under the same modelling framework.

Statistical analysis

Baseline characteristic analysis

Baseline characteristics were analysed to summarise the overall profile of patients in the development cohort and to compare differences between the 28-day survivor and non-survivor groups. Continuous variables were assessed for distributional characteristics and are presented as median and interquartile range because most variables showed non-normal distributions. Between-group comparisons were performed using the Mann–Whitney U test. Categorical variables were summarised as numbers and percentages and compared using the chi-squared test or Fisher’s exact test, as appropriate. All statistical tests were two-sided, and a P value of less than 0.05 was considered statistically significant.

Feature selection

Within each inner cross validation loop, least absolute shrinkage and selection operator (LASSO) regression was first applied for candidate variable selection, after which hyperparameter optimisation was performed for each candidate model using the selected features. After completion of the inner optimisation process, the model was refitted on the corresponding outer training fold with the optimal feature set and hyperparameter combination, and its performance was subsequently evaluated on the outer validation fold. All LASSO-based feature selection procedures were performed exclusively within the training data, and the internal test set and external validation cohort were not involved in feature selection. After completion of nested cross-validation, LASSO selection was repeated on the full training set to determine the final predictor set used for final model refitting and independent validation. Feature-selection stability was further assessed across the 20 imputed full-training LASSO analyses and the fold-level LASSO procedures generated from 20 imputed datasets × 10 cross-validation folds (Supplementary Table S8).

Model development and hyperparameter tuning

To minimise optimism bias during model development as far as possible, feature selection, hyperparameter tuning, and model construction were performed within the training set using a nested cross validation framework. A single 10 fold cross validation procedure was used for both the outer and inner loops. The outer loop was used to evaluate the overall model development pipeline, whereas the inner loop was used for feature selection and hyperparameter optimisation. Given the limited number of outcome events, candidate predictors were first reduced by LASSO selection to reduce the risk of overfitting. After model development had been completed within the training set, the final model was refitted using the full training set and then independently evaluated in the internal test set and external validation cohort.

Based on the selected features, nine ML models were developed: logistic regression (LR), elastic net (EN), random forest (RF), extreme gradient boosting (XGBoost), gradient boosting machine (GBM), support vector machine (SVM), k nearest neighbours (kNN), neural network (NN), and LightGBM. Multiple ML algorithms were evaluated as a benchmarking framework to compare models with different levels of complexity and flexibility under the same feature set, modelling procedure, and internal and external validation framework, thereby allowing a systematic assessment of their stability and generalisability. No global standardisation using the full dataset, internal test set, or external validation cohort was performed. For scale-sensitive models, including SVM, kNN, and NN, scaling parameters were estimated exclusively from the training data or the corresponding training folds within cross-validation, and the same transformation was then applied to the matched validation folds, internal test set, and external validation cohort. Tree-based models, including RF, XGBoost, GBM, and LightGBM, were fitted using predictors on their original scales after imputation and outlier handling. For all models, model-related procedures, including feature selection, class weighting, hyperparameter tuning, and threshold determination, were restricted to the training data. The internal test set and external validation cohort were not used during model development and were reserved exclusively for independent performance evaluation.

All preprocessing, training, and hyperparameter tuning procedures were conducted exclusively within the training set. Hyperparameter optimisation was performed using grid search within the nested cross validation framework, with the highest mean area under the receiver operating characteristic curve as the primary optimisation criterion. Specifically, the main hyperparameters tuned were the mixing parameter α and penalty parameter λ for EN; mtry, min.node.size, num.trees, and splitrule for RF; nrounds, max depth, eta, gamma, min_child_weight, colsample_bytree, and subsample for XGBoost; n.trees, interaction.depth, shrinkage, and n.minobsinnode for GBM; cost for SVM; number of neighbours k for kNN; and size and decay for NN. For LightGBM, the main structural parameters were searched, and the final number of trees was determined by early stopping. Detailed information on the candidate hyperparameter search spaces, fixed or default parameter settings, and the final selected parameters for each model is provided in Supplementary Table S5.

Class imbalance handling and sensitivity analysis

As 28-day mortality was a relatively low frequency outcome, a degree of class imbalance was present in the development cohort. Therefore, class weighting was incorporated into model training in the primary analysis to improve identification of minority class events [18]. Class weighting was applied only within the training set. During nested cross validation, this strategy was used exclusively in the corresponding training folds and was not applied to the validation folds, internal test set, or external validation cohort, in order to minimise the risk of information leakage and to provide a more realistic assessment of model generalisability.

To further examine the impact of class imbalance handling on model performance and the robustness of the study findings, a sensitivity analysis was performed using the synthetic minority over sampling technique (SMOTE) to resample the minority class only within the training folds, and all models were rebuilt under the same feature set and modelling framework [19]. The validation folds, internal test set, and external validation cohort all retained the original class distribution. Class weighting was used as the primary strategy because it addresses class imbalance during model training without changing the original sample structure, which was considered more appropriate because the internal test set and external validation cohort retained their original event distributions. SMOTE, which creates synthetic minority-class samples and may alter the original feature distribution, was therefore used as a sensitivity analysis to assess whether the findings were robust to an alternative imbalance-handling strategy.

Model performance evaluation

Model performance was comprehensively evaluated from three perspectives: discrimination, calibration, and potential clinical utility. Discrimination was assessed using the area under the receiver operating characteristic curve (AUC). In addition, accuracy, sensitivity, specificity, positive predictive value, negative predictive value, F1 score, balanced accuracy, and area under the precision–recall curve (PR-AUC) were reported. For metrics based on binary class labels, the classification threshold was first determined in the training set according to the ROC curve, incorporating the Youden index threshold, the closest.topleft, observed event rate threshold, and the central tendency of these candidate thresholds was used as the final threshold. This threshold was then fixed and applied to the internal test set and external validation cohort to calculate the corresponding classification metrics.

Calibration was assessed using calibration curves together with quantitative calibration indices, including the calibration intercept, calibration slope, and Brier score, to evaluate agreement between predicted probabilities and observed outcomes. Specifically, calibration curves were constructed using a decile‑based binning approach, in which the mean predicted probability within each decile was plotted against the observed event rate, and a LOESS smoothing curve was overlaid for visualization. Given that 28‑day mortality was a relatively low‑frequency outcome with most predicted probabilities concentrated at lower values, the calibration curves were primarily displayed over the predicted probability range of 0–0.5 to improve graphical readability and clinical interpretability.

To enhance model interpretability, feature importance analyses were performed for the better performing models to quantify the relative contribution of individual variables to prediction. In addition, decision curve analysis (DCA) and clinical impact curves (CIC) were used to evaluate the potential clinical net benefit of the models across different threshold probabilities and their implications for patient risk stratification.

External validation

All developed ML models were further applied to the eICU-CRD for external validation. During external validation, the same variable definitions, selected features, and model parameter settings as those used in the development cohort were strictly retained, with no model retraining or additional hyperparameter tuning, in order to evaluate model generalisability and stability across different population structures and healthcare settings.

In addition, to further assess the incremental predictive value of the ML models, the better performing ML models were compared with conventional clinical scoring systems in the external validation cohort, including the SOFA, APS III, SAPS II, and SIRS criteria. These comparisons included receiver operating characteristic curves, balanced accuracy, F1 score, PR-AUC, and decision curve analysis, in order to systematically evaluate the predictive performance and potential clinical utility of ML models relative to conventional scoring systems.

Results

Baseline characteristics

According to the inclusion and exclusion criteria, a total of 715 patients with AECOPD admitted to the ICU were included in this study (Fig. 1), of whom 84 (11.7%) died within 28 days (Table 1). Compared with survivors, non-survivors were significantly older, with a median age of 76.00 years versus 71.00 years, respectively (P < 0.001). In contrast, no statistically significant differences were observed between the two groups in weight, body temperature, heart rate, mean arterial pressure, respiratory rate, sex, or racial distribution (P > 0.05).

Fig. 1.

Study flowchart of patient selection and cohort construction

Table 1.

Baseline demographic and clinical characteristics of the included patients

Variable	Overall (N = 715)	Survival (N = 631)	Non-Survival (N = 84)	P value
Weight	78.50 (62.23–96.68)	78.50 (62.55–96.80)	78.85 (60.16–93.40)	0.521
Temperature	98.20 (97.80–98.70)	98.30 (97.80–98.70)	98.10 (97.80-98.62)	0.324
Age	72.00 (65.00–81.00)	71.00 (64.00–80.00)	76.00 (69.00-82.25)	< 0.001
HR	90.00 (77.00-103.50)	89.00 (77.00-103.00)	93.00 (76.75–104.00)	0.413
MAP	84.00 (73.00–97.00)	84.00 (73.00-97.50)	82.00 (72.75–94.25)	0.381
RR	21.00 (17.00–25.00)	21.00 (17.00–25.00)	21.00 (18.00–27.00)	0.480
Sex, n (p%)				1.000
Male	361 (50.49%)	319 (50.55%)	42 (50%)
Female	354 (49.51%)	312 (49.45%)	42 (50%)
Race, n (p%)				0.321
White	475 (66.43%)	422 (66.88%)	53 (63.10%)
Black	70 (9.79%)	64 (10.14%)	6 (7.14%)
Other	170 (23.78%)	145 (22.98%)	25 (29.76%)
Comorbidities
HTN, n (p%)				0.228
No	500 (69.93%)	436 (69.10%)	64 (76.19%)
Yes	215 (30.07%)	195 (30.90%)	20 (23.81%)
HEP, n (p%)				0.566
No	684 (95.66%)	602 (95.40%)	82 (97.62%)
Yes	31 (4.34%)	29 (4.60%)	2 (2.38%)
PNA, n (p%)				< 0.001
No	250 (34.97%)	235 (37.24%)	15 (17.86%)
Yes	465 (65.03%)	396 (62.76%)	69 (82.14%)
CKD, n (p%)				0.471
No	505 (70.63%)	449 (71.16%)	56 (66.67%)
Yes	210 (29.37%)	182 (28.84%)	28 (33.33%)
T2DM, n (p%)				0.564
No	452 (63.22%)	396 (62.76%)	56 (66.67%)
Yes	263 (36.78%)	235 (37.24%)	28 (33.33%)
T1DM, n (p%)				0.466
No	710 (99.30%)	627 (99.37%)	83 (98.81%)
Yes	5 (0.70%)	4 (0.63%)	1 (1.19%)
HF, n (p%)				0.358
No	310 (43.36%)	278 (44.06%)	32 (38.10%)
Yes	405 (56.64%)	353 (55.94%)	52 (61.90%)
MI, n (p%)				0.071
No	598 (83.64%)	534 (84.63%)	64 (76.19%)
Yes	117 (16.36%)	97 (15.37%)	20 (23.81%)
IHD, n (p%)				0.779
No	389 (54.41%)	345 (54.68%)	44 (52.38%)
Yes	326 (45.59%)	286 (45.32%)	40 (47.62%)
AKI, n (p%)				< 0.001
No	148 (20.70%)	143 (22.66%)	5 (5.95%)
Yes	567 (79.30%)	488 (77.34%)	79 (94.05%)
Severity score
SOFA	5.00 (3.00–7.00)	4.00 (3.00–7.00)	6.50 (4.00–10.00)	< 0.001
APS III	43.00 (33.00–55.00)	42.00 (33.00–54.00)	57.00 (45.50-72.25)	< 0.001
SIRS	3.00 (2.00–3.00)	3.00 (2.00–3.00)	3.00 (2.00–3.00)	0.009
SAPS II	37.00 (30.00–46.00)	36.00 (29.00–44.00)	48.50 (36.75–56.25)	< 0.001
GCS	15.00 (13.00–15.00)	15.00 (14.00–15.00)	14.50 (11.00–15.00)	0.017
APACHE II	21.00 (17.00–25.00)	20.00 (17.00–24.00)	25.00 (20.00-29.25)	< 0.001
Laboratory test results
TCO2	29.00 (24.00–34.00)	29.00 (25.00–34.00)	26.00 (22.00–33.00)	0.012
Lactate	1.50 (1.10–2.20)	1.50 (1.10–2.15)	1.60 (1.10–2.60)	0.289
PaCO2	50.00 (41.00-64.50)	50.00 (42.00-64.50)	52.50 (40.00-64.25)	0.839
pH	7.35 (7.28–7.41)	7.35 (7.28–7.41)	7.34 (7.25–7.38)	0.029
PaO2	59.00 (42.00-89.50)	59.00 (42.00–88.00)	69.00 (43.50-98.25)	0.274
Platelets	204.00 (146.00-261.50)	206.00 (148.00-263.00)	183.00 (128.00-245.25)	0.012
Hb	10.70 (9.00-12.30)	10.80 (9.10-12.45)	10.10 (8.10–11.50)	0.015
RDW	15.30 (14.00–17.00)	15.20 (14.00-16.95)	15.45 (14.00-17.33)	0.508
Anion gap	14.00 (11.00–17.00)	13.00 (11.00–17.00)	14.00 (12.00–17.00)	0.155
Calcium	8.70 (8.20–9.05)	8.70 (8.20–9.10)	8.60 (8.00-8.90)	0.094
Potassium	4.40 (4.00-4.90)	4.30 (4.00-4.90)	4.40 (4.10–4.82)	0.175
Glucose	144.00 (114.00-181.50)	143.00 (113.00-180.00)	150.50 (119.00-195.50)	0.158
Sodium	139.00 (135.00-142.00)	139.00 (135.00-142.00)	139.00 (134.00-142.00)	0.704
PT	13.00 (11.80–15.50)	12.80 (11.60–15.20)	14.00 (12.50-18.35)	< 0.001
TBil	0.50 (0.40–0.90)	0.50 (0.30–0.80)	0.70 (0.40–1.65)	0.002
Creatinine	1.30 (0.90–2.25)	1.30 (0.90–2.10)	2.05 (1.20–3.68)	< 0.001
BUN	39.00 (26.00–62.00)	37.00 (25.00–59.00)	55.50 (40.00–89.00)	< 0.001
AST	34.00 (21.00–80.00)	32.00 (20.00–71.00)	58.50 (26.75-158.69)	< 0.001
ALT	29.00 (15.00–62.00)	27.00 (14.50–58.50)	40.00 (18.00–97.00)	0.013
LOS ICU, days	3.05 (1.83–6.13)	2.89 (1.79–5.82)	4.01 (2.68–7.96)	< 0.001
Hospital stay, days	9.06 (5.50-16.97)	9.06 (5.68–16.97)	9.25 (4.26–16.86)	0.193

Abbreviations p% percentage, HR heart rate, MAP mean arterial pressure, RR respiratory rate, HTN hypertension, HEP hepatic disease, PNA pneumonia, CKD chronic kidney disease, T2DM type 2 diabetes mellitus, T1DM type 1 diabetes mellitus, HF heart failure, MI myocardial infarction, IHD ischemic heart disease, MV mechanical ventilation, AKI acute kidney injury, SOFA Sequential Organ Failure Assessment, APS III Acute Physiology Score III, SIRS Systemic Inflammatory Response Syndrome, SAPS II Simplified Acute Physiology Score II, OASIS Oxford Acute Severity of Illness Score, GCS Glasgow Coma Scale, CCI Charlson Comorbidity Index, APACHE II Acute Physiology and Chronic Health Evaluation II, TCO2 total carbon dioxide, PaCO2 partial pressure of carbon dioxide, PaO2 partial pressure of oxygen, HCT hematocrit, Hb hemoglobin, RDW red cell distribution width, WBC white blood cell count, RBC red blood cell count, PT prothrombin time, INR international normalized ratio, TBil total bilirubin, BUN blood urea nitrogen, AST aspartate aminotransferase, ALT alanine aminotransferase, LOS ICU length of ICU stay

With regard to comorbidities, pneumonia was significantly more common in the non-survivor group than in the survivor group, at 82.14% versus 62.76% (P < 0.001), and acute kidney injury (AKI) was also more frequent, at 94.05% versus 77.34% (P < 0.001). The distribution of other comorbidities, including hypertension, chronic kidney disease, diabetes mellitus, heart failure, ischaemic heart disease, and cerebrovascular disease, was comparable between the two groups (P > 0.05). In addition, non-survivors had substantially greater disease severity at ICU admission, as reflected by significantly higher SOFA, APS III, SAPS II, SIRS, and APACHE II scores (all P < 0.01), together with a significantly lower Glasgow Coma Scale (GCS) score (P = 0.017).

Laboratory findings indicated more pronounced organ dysfunction and metabolic disturbance in non-survivors, mainly reflected by lower total carbon dioxide (TCO₂), pH, platelet count, and haemoglobin, together with higher prothrombin time (PT), total bilirubin (TBil), creatinine, blood urea nitrogen (BUN), aspartate aminotransferase (AST), and alanine aminotransferase (ALT), all with P < 0.05. By contrast, lactate, partial pressure of carbon dioxide (PaCO₂), partial pressure of oxygen (PaO₂), red cell distribution width (RDW), anion gap, calcium, potassium, glucose, and sodium did not differ significantly between the two groups (P > 0.05). Non-survivors also had a longer ICU stay than survivors, with a median length of ICU stay of 4.01 (2.68–7.96) days versus 2.89 (1.79–5.82) days (P < 0.001), whereas hospital length of stay was not significantly different between the two groups (P = 0.193).

Feature selection

Candidate variables were screened using LASSO regression with 10-fold cross validation. The cross-validation error curve across different values of the regularisation parameter λ is presented in Fig. 2A. The optimal regularisation parameter for subsequent model development was selected according to the minimum binomial deviance criterion, namely λ_min. At this value, the regression coefficients of some variables were shrunk to zero, retaining only those features with greater predictive contribution. As the degree of regularisation increased, the number of variables with non-zero coefficients progressively decreased, as shown in Fig. 2B.

Fig. 2.

Feature selection using LASSO regression.A Ten-fold cross-validation curve of the LASSO model. The x-axis represents log λ, and the y-axis represents binomial deviance. The red dashed vertical line indicates the optimal regularization parameter corresponding to the minimum cross-validated error λ_min, whereas the blue dashed vertical line indicates λ_1se, defined as the largest λ within one standard error of the minimum. Error bars denote the standard errors of the cross-validation estimates. B LASSO coefficient profiles of candidate predictors. The x-axis represents log λ, and the y-axis represents regression coefficients. As the degree of penalization increases, the coefficients are progressively shrunk toward zero. Predictors with non-zero coefficients at λ_min were retained as candidate features for subsequent machine learning model development. A total of 15 predictors were ultimately selected

Ultimately, LASSO regression identified a set of features with non-zero coefficients, including SIRS, SAPS II, PaO₂, platelet count, AKI, hepatic disease (HEP), pneumonia, chronic kidney disease (CKD), T2DM, age, PT, TBil, BUN, creatinine, and AST. These variables were subsequently used as input features for the development and validation of multiple ML models for predicting 28-day mortality in patients with AECOPD.

Predictive performance of ML models

Using the variables selected by LASSO, we developed multiple ML models for predicting 28-day mortality in ICU patients with AECOPD. Overall model performance varied across the training set, internal test set, and external validation cohort (Table 2, Supplementary Table S6, and Figs. 3 and 4).

Table 2.

Performance of class-weighted machine learning models in the internal test and external validation cohorts

Cohort	Model	Accuracy	Sensitivity	Specificity	AUC (95% CI)	Balanced Accuracy	F1 Score	PPV	NPV	PR-AUC
Internal Test	Logistic Regression	0.565	0.720	0.545	0.692 (0.561–0.824)	0.632	0.279	0.173	0.936	0.363
	Elastic Net	0.780	0.440	0.825	0.698 (0.572–0.823)	0.633	0.319	0.250	0.918	0.364
	Random Forest	0.738	0.520	0.767	0.706 (0.587–0.824)	0.644	0.317	0.228	0.924	0.393
	XGBoost	0.720	0.520	0.746	0.694 (0.580–0.808)	0.633	0.302	0.213	0.922	0.272
	GBM	0.654	0.640	0.656	0.658 (0.539–0.776)	0.648	0.302	0.198	0.932	0.245
	SVM	0.710	0.640	0.720	0.722 (0.599–0.846)	0.680	0.340	0.232	0.938	0.378
	kNN	0.729	0.560	0.751	0.701 (0.575–0.826)	0.656	0.326	0.230	0.928	0.393
	Neural Network	0.780	0.480	0.820	0.720 (0.602–0.838)	0.650	0.338	0.261	0.923	0.395
	LightGBM	0.771	0.400	0.820	0.692 (0.577–0.807)	0.610	0.290	0.227	0.912	0.263
External Validation	Logistic Regression	0.687	0.281	0.722	0.615 (0.528–0.702)	0.502	0.126	0.081	0.920	0.140
	Elastic Net	0.739	0.312	0.777	0.636 (0.552–0.720)	0.545	0.161	0.109	0.928	0.154
	Random Forest	0.835	0.312	0.880	0.705 (0.618–0.793)	0.596	0.233	0.185	0.936	0.262
	XGBoost	0.709	0.656	0.714	0.733 (0.650–0.816)	0.685	0.266	0.167	0.960	0.224
	GBM	0.634	0.688	0.629	0.743 (0.671–0.814)	0.658	0.232	0.139	0.959	0.231
	SVM	0.729	0.438	0.755	0.676 (0.586–0.766)	0.596	0.206	0.135	0.939	0.230
	kNN	0.659	0.469	0.676	0.640 (0.554–0.726)	0.572	0.181	0.112	0.936	0.107
	Neural Network	0.704	0.438	0.728	0.691 (0.607–0.774)	0.583	0.192	0.123	0.937	0.210
	LightGBM	0.810	0.469	0.839	0.764 (0.689–0.839)	0.654	0.283	0.203	0.948	0.213

AUC and PR-AUC were calculated independently in each cohort. Threshold-dependent metrics, including accuracy, sensitivity, specificity, balanced accuracy, F1 score, PPV, and NPV, were calculated using the optimal threshold determined in the training set and fixed for the internal test and external validation cohorts. AUC area under the receiver operating characteristic curve, CI confidence interval, PR-AUC area under the precision-recall curve, PPV positive predictive value, NPV negative predictive value

Fig. 3.

ROC curves and AUC comparison of different ML models in the training, internal test, and external validation sets. A ROC curves of different ML models for predicting 28-day mortality in patients with AECOPD in the training set. B ROC curves of different ML models in the internal test set. C ROC curves of different ML models in the external validation cohort from eICU. D Comparison of AUCs of all models across the training, internal test, and external validation sets

Fig. 4.

Precision-recall curves of different machine learning models in the training, internal test, and external validation sets. A Training set. B Internal test set. C eICU-CRD external validation cohort. Because 28-day mortality was a low-frequency outcome, precision–recall analysis was used to further evaluate the ability of each model to identify positive events under class imbalance

In the internal test set, SVM achieved the highest AUC of 0.722, followed by neural network with an AUC of 0.720 and random forest with an AUC of 0.706. Logistic regression showed the highest sensitivity at 0.720, whereas SVM achieved the highest balanced accuracy and F1 score, at 0.680 and 0.340, respectively. Neural network achieved the highest PR-AUC at 0.395, followed by random forest and kNN, both with PR-AUC values of 0.393.

In the external validation cohort, LightGBM achieved the highest AUC among the evaluated models, with an AUC of 0.764, followed by GBM, XGBoost, and random forest, with AUCs of 0.743, 0.733, and 0.705, respectively. GBM showed the highest sensitivity at 0.688, suggesting relatively greater ability to identify patients who died within 28 days. XGBoost achieved the highest balanced accuracy and NPV, at 0.685 and 0.960, respectively. LightGBM showed the highest F1 score at 0.283, whereas random forest achieved the highest PR-AUC at 0.262.

Taken together, the models showed complementary strengths across different evaluation metrics. In the internal test set, SVM performed relatively well in terms of discrimination and threshold-dependent overall balance, whereas neural network showed the highest PR-AUC. In the external validation cohort, LightGBM showed the strongest discrimination and F1 score, GBM showed the highest sensitivity, XGBoost showed the highest balanced accuracy and NPV, and random forest showed the highest PR-AUC. These findings suggest that model performance should be interpreted according to the intended clinical objective, rather than relying on a single summary metric.

From a clinical screening perspective, sensitivity and NPV were considered particularly relevant because the intended use of the models was early risk stratification rather than definitive treatment allocation, and missed high-risk patients may delay intensified monitoring or treatment escalation. In the internal test set, logistic regression achieved the highest sensitivity, whereas SVM showed the highest NPV and balanced accuracy. In the external validation cohort, GBM achieved the highest sensitivity, whereas XGBoost showed the highest NPV and balanced accuracy. These findings suggest that screening-oriented interpretation should consider sensitivity and NPV alongside AUC, while PR-AUC and F1 score provide complementary information on positive-event identification under class imbalance.

Clinical utility and calibration of the ML models

Quantitative calibration analysis showed notable differences across models in the internal test set and external validation cohort (Table 3). In the internal test set, SVM showed the most favourable calibration profile, with a calibration intercept close to zero, a calibration slope close to one, and the lowest Brier score. In the external validation cohort, RF and LightGBM achieved the lowest Brier scores, whereas the neural network showed a calibration slope closest to one. However, several models still showed substantial deviations in calibration intercepts or slopes, indicating that favourable discrimination did not necessarily translate into accurate probability estimation. Overall, no single model consistently achieved optimal calibration across all indices and datasets.

Table 3.

Calibration intercept, calibration slope, and Brier score of class-weighted machine learning models in the internal test and external validation cohorts

Cohort	Model	Calibration Intercept	Calibration Slope	Brier Score
Internal Test	Logistic Regression	-1.196	0.522	0.105
	Elastic Net	-0.810	0.863	0.101
	Random Forest	4.218	3.187	0.099
	XGBoost	2.555	3.813	0.112
	GBM	3.655	4.316	0.110
	SVM	-0.033	0.958	0.091
	kNN	1.025	1.531	0.096
	Neural Network	-0.146	1.484	0.102
	LightGBM	9.096	5.889	0.102
External Validation	Logistic Regression	-2.047	0.189	0.131
	Elastic Net	-2.016	0.301	0.124
	Random Forest	3.483	2.874	0.072
	XGBoost	3.247	4.799	0.093
	GBM	5.789	6.260	0.088
	SVM	-2.013	0.256	0.088
	kNN	-0.922	0.692	0.075
	Neural Network	-1.374	0.887	0.097
	LightGBM	12.615	8.034	0.074

Abbreviations GBM gradient boosting machine, SVM support vector machine, kNN k-nearest neighbors, LightGBM light gradient boosting machine, XGBoost extreme gradient boosting

Calibration curves, clinical impact curves, and decision curve analyses were further plotted for all models across the training set, internal test set, and external validation cohort (Fig. 5). Most calibration curves deviated from the ideal reference line to varying degrees, especially in the external validation cohort. The clinical impact curves showed that the number of high-risk patients and true positives decreased as the threshold probability increased. Decision curve analysis suggested that several models provided positive net benefit within selected low-to-moderate threshold ranges, but the benefit was less stable at higher thresholds. These findings suggest that the ML models may provide supplementary information for risk stratification in AECOPD patients, although their calibration and clinical utility should be interpreted cautiously.

Fig. 5.

Calibration curves, CICs, and DCA of the all models in the training, internal test, and external validation sets.A–C Calibration curves in the training set, internal test set, and external validation cohort, respectively. The x-axis represents predicted probability, and the y-axis represents the observed event rate. The grey dashed diagonal line indicates ideal calibration. D–F Clinical impact curves in the corresponding datasets. The x-axis represents threshold probability, and the y-axis represents the number of patients. Solid lines indicate the number of patients classified as high risk, whereas dashed lines indicate the number of true positives. G–I Decision curve analyses in the corresponding datasets. The x-axis represents threshold probability, and the y-axis represents net benefit. Reference lines indicate the treat-all and treat-none strategies

SHAP feature importance analysis

To further interpret model predictions, SHAP analysis was performed in the external validation for GBM, LightGBM, XGBoost, and RF (Fig. 6). The key features identified across models showed a high degree of consistency. BUN, creatinine, SAPS II, and PT were consistently ranked among the most important features in multiple models, indicating that renal dysfunction, disease severity, and coagulation abnormalities play important roles in predicting 28-day mortality in patients with AECOPD. In addition, PNA, age, first PO₂, AKI, total bilirubin, and platelet count also contributed to model prediction across different models.

Fig. 6.

SHAP feature importance of different ML models in the external validation set. A–D SHAP feature importance plots for the LightGBM, GBM, XGBoost, and random forest models, respectively. The x-axis represents SHAP values, indicating the direction and magnitude of each feature’s contribution to model prediction, and the y-axis shows features ranked by mean absolute SHAP value. Bars indicate the mean absolute SHAP values for each feature, whereas points show individual-level SHAP values coloured by feature value

Specifically, in the LightGBM model, BUN, SAPS II, and creatinine showed the greatest contributions. A similar pattern was observed in the GBM model, in which BUN, SAPS II, and creatinine were also the most influential features. XGBoost placed greater emphasis on creatinine, SAPS II, BUN, and PT, whereas in the RF model, PNA, AKI, BUN, PT, SAPS II, and creatinine ranked among the leading predictors.

Comparison between ML models and conventional clinical scores

To further assess the incremental predictive value of ML models over conventional clinical scoring systems, we compared their discriminative performance, positive-event identification, and potential clinical utility in the external validation cohort (Table 4). Overall, several ML models showed numerically higher AUC and F1 score than conventional clinical scores. Among the conventional scores, SAPS II showed the best overall performance, with the highest balanced accuracy among all compared methods, whereas APS III showed the highest sensitivity and NPV but relatively low specificity, indicating a stronger tendency towards high-risk screening. By contrast, SOFA and SIRS showed comparatively weaker overall discriminative performance.

Table 4.

Comparative predictive performance of class-weighted machine learning models and conventional clinical scoring systems for 28-day mortality in the external validation cohort

Model	Accuracy	Sensitivity	Specificity	AUC (95% CI)	Balanced Accuracy	F1 Score	PPV	NPV
LightGBM	0.810	0.469	0.839	0.764 (0.689–0.839)	0.654	0.283	0.203	0.948
GBM	0.634	0.688	0.629	0.743 (0.671–0.814)	0.658	0.232	0.139	0.959
XGBoost	0.709	0.656	0.714	0.733 (0.650–0.816)	0.685	0.266	0.167	0.960
Random Forest	0.835	0.312	0.880	0.705 (0.618–0.793)	0.596	0.233	0.185	0.936
SAPS II (Traditional)	0.671	0.719	0.668	0.703 (0.612–0.793)	0.693	0.260	0.159	0.965
APS III (Traditional)	0.465	0.875	0.436	0.688 (0.598–0.778)	0.655	0.210	0.119	0.976
SOFA (Traditional)	0.438	0.812	0.409	0.645 (0.551–0.738)	0.611	0.189	0.107	0.962
SIRS (Traditional)	0.559	0.562	0.559	0.559 (0.469–0.649)	0.561	0.170	0.100	0.936

Abbreviations LightGBM light gradient boosting machine, GBM gradient boosting machine, XGBoost extreme gradient boosting, SAPS II Simplified Acute Physiology Score II, APS III Acute Physiology Score III, SOFA Sequential Organ Failure Assessment, SIRS Systemic Inflammatory Response Syndrome, AUC area under the receiver operating characteristic curve, PR-AUC area under the precision-recall curve

The ROC curves and performance heatmap further supported these findings (Fig. 7). LightGBM achieved the highest AUC, whereas XGBoost achieved the highest balanced accuracy among ML models. Precision–recall curve analysis showed that RF achieved the highest PR-AUC among the evaluated ML models, followed by GBM and SVM, although the absolute PR-AUC values remained modest. Decision curve analysis indicated that the net benefit of all methods was threshold dependent. RF showed a relatively higher and more stable net benefit across part of the threshold range, whereas the net benefit of conventional clinical scores appeared relatively limited within the evaluated threshold range. Taken together, these findings suggest that ML models may provide complementary predictive information beyond conventional scores, but their incremental value should be interpreted cautiously rather than as uniform superiority.

Fig. 7.

Comparison of ML models and traditional scores in the external validation set. A Receiver operating characteristic curves of machine learning models and conventional clinical scores for predicting 28-day mortality in patients with AECOPD in the eICU-CRD external validation cohort. The dashed line indicates the reference line for no discriminative ability. B Decision curve analysis of machine learning models and conventional clinical scores in the external validation cohort. C Precision–recall curves of machine learning models and conventional clinical scores in the external validation cohort. D Comparison of AUC values between machine learning models and conventional clinical scores in the external validation cohort. E Heatmap of model performance metrics in the external validation cohort, including AUC, balanced accuracy, sensitivity, specificity, and F1 score

Sensitivity analysis

As a sensitivity analysis for class imbalance, SMOTE was applied to oversample the minority class in the training set, and all ML models were rebuilt and revalidated using the same feature set (Supplementary Table S7 and Supplementary Fig. 1). After SMOTE, RF achieved the highest AUC in the training set at 0.820, whereas SVM showed the highest AUC in the internal test set at 0.728. In the external validation cohort, LightGBM showed the best discriminative performance, with an AUC of 0.754, balanced accuracy of 0.731, and F1 score of 0.249. Logistic regression, neural network, and LightGBM showed high sensitivity, all reaching 0.969, but this was accompanied by relatively low specificity. These findings suggest that SMOTE tended to increase sensitivity in several models but often at the expense of specificity.

Supplementary Fig. 1 showed that SMOTE improved apparent model performance in the training set, but its benefit was less consistent in the internal test set and external validation cohort. The external calibration curves and decision curves did not show clear improvement in probability estimation or clinical net benefit. Overall, SMOTE had limited impact on external generalisability, calibration, and clinical utility, supporting the robustness of the primary class-weighted analysis.

Additional comparison of model performance with and without Winsorisation showed only minor changes in the training set and internal test set. Although some models showed changes in the external validation cohort, the AUCs of the better-performing models remained broadly stable, suggesting that Winsorisation did not materially alter the main conclusions (Supplementary Table S9).

Discussion

In this multicentre retrospective study, we developed and validated multiple ML models for predicting 28-day mortality in ICU patients with AECOPD using two large critical care databases. Our findings suggest that models based on routinely available clinical characteristics, laboratory variables, and disease severity scores can, to a certain extent, distinguish patients at high risk of short-term death from those at lower risk. Several models showed moderate discriminative performance in the internal validation set and retained a limited degree of generalisability in the independent external cohort. However, different models showed distinct strengths across evaluation domains. LightGBM showed the strongest discriminative performance in the external validation cohort, GBM showed the highest sensitivity and therefore may be relevant to screening-oriented risk assessment, RF showed the highest PR-AUC and relatively favourable performance in positive-event identification, whereas calibration performance varied substantially across models and datasets. These findings indicate that, in the setting of a low event rate, evaluation of short-term mortality prediction models for AECOPD should not rely solely on receiver operating characteristic related metrics, but should also take into account PR performance, calibration, and the intended clinical use.

Given the intended use of the models for early ICU risk stratification rather than definitive treatment allocation, sensitivity and NPV were considered particularly relevant from a clinical screening perspective. Higher sensitivity may reduce missed recognition of patients at high short-term mortality risk, whereas higher NPV may improve the safety of classifying patients as lower risk. In this study, GBM achieved the highest sensitivity in the external validation cohort, while XGBoost achieved the highest NPV and balanced accuracy. These findings suggest that screening-oriented interpretation should consider sensitivity and NPV alongside AUC, while PR-AUC and F1 score provide complementary information on positive-event identification under class imbalance.

In summary, we did not identify a single universally optimal model; instead, different models showed complementary strengths across different evaluation dimensions. This observation highlights an important and often underrecognised issue in clinical prediction modelling: the evaluation metric chosen can fundamentally influence which model appears to perform best. Whereas discrimination metrics such as AUC reflect the ability to rank patients according to risk, calibration metrics reflect the accuracy of predicted probabilities, and sensitivity oriented metrics emphasise the identification of minority class events. In AECOPD, where 28-day mortality is a relatively infrequent outcome, these dimensions of model performance do not necessarily align. A model with a high AUC may still yield poorly calibrated probabilities, thereby limiting its usefulness for individualised decision making, whereas a well calibrated model may do so at the expense of some ranking performance.

Accordingly, rather than seeking a one size fits all solution, model selection should be guided by the intended use case, whether for population level risk stratification, where LightGBM may be advantageous because of its relatively favourable discrimination in the external validation cohort; for clinical scenarios requiring identification of positive events under class imbalance, where RF may be useful because of its higher PR-AUC; or for early screening strategies designed to minimise missed high-risk patients, where GBM may be considered because of its relatively higher sensitivity. However, no model showed uniformly superior performance across discrimination, PR performance, calibration, and decision curve analysis, underscoring the need for model selection according to clinical purpose rather than a single summary metric.

However, the practical relevance of these complementary strengths is tempered by the observed decline in performance across databases. Indeed, most models showed varying degrees of performance deterioration in the eICU-CRD cohort compared with the internal test set, suggesting that cross database transportability remained limited and that a degree of optimistic performance estimation may still be present.

In previous studies, prognostic assessment in patients with AECOPD has relied predominantly on conventional scoring systems or single statistical models. Although the application of ML methods has increased in recent years, important limitations persist [20]. Most prior studies have developed models using single centre data without independent external validation, leaving the stability of their findings across different healthcare settings uncertain [13, 21, 22]. Moreover, model evaluation has often focused primarily on discrimination, with insufficient attention to the calibration of predicted probabilities and the actual clinical benefit of model guided decision making [23, 24]. Compared with these earlier studies, the present study not only systematically compared multiple mainstream ML algorithms using the same feature set, but also adopted a more comprehensive evaluation framework incorporating PR curves, calibration analysis, decision curve analysis, and clinical impact curves. In particular, given that the 28-day mortality rate in this study was only 11.7%, reliance on AUC alone could easily overestimate a model’s ability to identify minority class events, whereas PR-AUC, F1 score, and balanced accuracy provide a more informative assessment of model performance under class imbalance.

From a clinical application perspective, several ML models showed more favourable numerical performance than conventional clinical scoring systems in the external validation cohort, suggesting potential incremental value. However, this potential advantage should be interpreted cautiously in the context of domain shift, the limited number of death events, and model complexity. Most models performed less well in the eICU-CRD cohort than in the internal test set, and some conventional scores, particularly SAPS II, showed performance close to that of certain ML models, indicating that greater model complexity does not necessarily translate into proportionate performance gains. Therefore, for predicting 28-day mortality in patients with AECOPD, model selection should take into account not only predictive performance but also interpretability, computational complexity, and clinical feasibility. The real world utility of these models still requires further confirmation through local validation, model recalibration, and prospective studies. Given the limited number of outcome events, the comparative performance of complex models should therefore be regarded as exploratory rather than definitive.

At the mechanistic level, feature importance analysis provided valuable insights into model predictions. Although the underlying algorithmic structures differed across models, the key features identified were broadly consistent. BUN, creatinine, SAPS II, and PT were all ranked among the most important predictors, suggesting that renal dysfunction, overall disease severity, and coagulation abnormalities are closely associated with 28-day mortality risk in patients with AECOPD. This finding further suggests that the short-term prognosis of critically ill patients with AECOPD may be driven more by systemic physiological disturbance and multisystem decompensation than by isolated injury to a single organ [25, 26].

During the acute phase of AECOPD, airway inflammation is often markedly intensified, while the combined effects of hypoxaemia and hypercapnia may further precipitate systemic inflammatory responses, haemodynamic instability, and impaired tissue perfusion [27]. Pre-existing chronic pulmonary dysfunction and persistent background inflammation may further compromise physiological reserve and compensatory capacity in these patients [28, 29]. Accordingly, composite severity scores reflecting the overall burden of illness consistently retained high importance across different models, whereas the predictive contribution of individual laboratory variables or isolated organ specific parameters was comparatively limited. Consistent with this interpretation, a study involving 1,647 patients showed that the presence of either ARF or AKI alone was associated with a significant increase in in-hospital mortality, whereas the coexistence of both conditions increased the risk of death nearly 40-fold [30].

In addition, renal function related variables such as creatinine and BUN, together with infection related factors, were repeatedly identified as important predictors across multiple models, suggesting that mortality risk in patients with AECOPD is closely linked to coordinated decompensation along the lung–kidney–infection axis. Persistent hypoxia and inflammation may precipitate or exacerbate AKI through redistribution of renal blood flow, tubular hypoxia, and immune mediated injury [31, 32]. In turn, renal dysfunction may further increase respiratory and circulatory burden through fluid retention, acid–base disturbance, and impaired clearance of inflammatory mediators, thereby establishing a self amplifying vicious cycle [33]. At the same time, pneumonia is not only a major trigger of acute exacerbation and respiratory failure in AECOPD, but its presence may also reflect a greater inflammatory burden and a more complex clinical course [27]. The importance of coagulation and liver function related variables in some ML models further supports a potential role of microcirculatory dysfunction and inflammation–coagulation interaction in shaping mortality risk in AECOPD complicated by ARF.

However, several limitations of this study should be acknowledged. First, this was a retrospective study. Although multicentre databases were used and systematic data preprocessing was undertaken, residual confounding could not be fully eliminated. Second, the number of 28-day death events in the development cohort was relatively limited, and the outcome was imbalanced. Although strategies including LASSO-based feature selection, nested cross-validation, class weighting, SMOTE-based sensitivity analysis, and external validation were applied to minimise optimism and overfitting as far as possible, these approaches cannot fully eliminate the uncertainty introduced by the limited number of events. In particular, given that only 84 deaths were observed and 15 predictors were retained in the final feature set, the stability of model estimation may still have been affected by the relatively low amount of outcome information. This issue is especially relevant for more flexible algorithms with larger parameter spaces, which may capture sample-specific noise rather than stable prognostic signals. Third, differences between databases in patient case mix, clinical practice patterns, and data acquisition processes may have influenced the external validation results. Fourth, missing data were handled using MICE imputation. Although this approach reduced data loss and was implemented within the training process to minimise information leakage, the imputation procedure may still have introduced uncertainty into model development and affected external calibration because of differences in missingness patterns and variable distributions across datasets. Fifth, these models have not yet undergone prospective validation in a real time clinical setting, and their actual impact on clinical decision making and patient outcomes remains uncertain.

Finally, the models in this study were constructed using static variables obtained early after ICU admission and therefore reflected only the initial risk profile at admission, without incorporating dynamic changes occurring during hospitalisation. Given the marked time dependence and dynamic evolution of critical illness, such a static modelling strategy may not adequately capture disease progression or temporal changes in risk. Several previous studies have suggested that dynamic temporal information may provide additional prognostic value in the ICU [34–36]. For example, Thorsen-Meyer et al. developed a dynamically updated ICU mortality prediction model based on high frequency electronic health record time series data and demonstrated that continuous temporal information could support real time risk assessment [37]. Zheng et al. also developed a dynamic real time ICU mortality prediction model using irregular longitudinal electronic medical record data [34]. In addition, Britsch et al. proposed the LGBM-48 h dynamic prediction framework, which continuously updates the estimated risk of death over the subsequent 48 h during ICU stay, further highlighting the potential value of dynamically updated models for short-term risk stratification and clinical decision support [35].

Therefore, future studies should further integrate dynamic physiological parameters, treatment related information, and imaging or biomarker data to develop dynamic prediction models that can be continuously updated throughout hospitalisation, thereby improving the characterisation of disease evolution and the detection of short-term changes in risk. At the same time, prospective multicentre studies remain necessary to validate the real time predictive performance, generalisability, and clinical utility of such models, and to determine their potential value in optimising clinical decision making and improving patient outcomes.

Conclusions

Using the MIMIC-IV and eICU-CRD databases, this study developed and validated multiple ML models for predicting 28-day mortality in ICU patients with AECOPD. Different models showed complementary strengths across evaluation domains, with LightGBM performing favourably in discrimination, GBM in sensitivity, XGBoost in NPV and balanced accuracy, and random forest in PR-AUC. Calibration performance varied across models and datasets. Compared with conventional clinical scoring systems, ML models showed potential incremental predictive value, but this finding should be interpreted cautiously given the limited number of death events, class imbalance, and risk of overfitting. No single model consistently outperformed the others across all evaluation dimensions. Therefore, model selection should be guided by the specific clinical priority, whether maximising overall discrimination, obtaining well calibrated probability estimates, or prioritising sensitivity. These models should be regarded as exploratory prediction models that may provide supplementary information for early risk stratification at ICU admission, rather than as tools ready for direct clinical implementation. Further prospective validation, local recalibration, and evaluation within real-world clinical workflows are required before clinical application.

Abbreviation

AECOPD

Acute exacerbation of chronic obstructive pulmonary disease

AKI

Acute kidney injury

ALT

Alanine aminotransferase

APACHE II

Acute physiology and chronic health evaluation II

APS III

Acute physiology score III

ARDS

Acute respiratory distress syndrome

ARF

Acute respiratory failure

AST

Aspartate aminotransferase

AUC

Area under the receiver operating characteristic curve

BUN

Blood urea nitrogen

CCI

Charlson comorbidity index

CIC

Clinical impact curve

CKD

Chronic kidney disease

COPD

Chronic obstructive pulmonary disease

DCA

Decision curve analysis

eICU-CRD

eICU collaborative research database

EMR

Electronic medical record

EN

Elastic net

EHR

Electronic health record

F1 score

Harmonic mean of precision and recall

GBM

Gradient boosting machine

GCS

Glasgow coma scale

HCT

Haematocrit

HEP

Hepatic disease

HF

Heart failure

HR

Heart rate

HTN

Hypertension

ICD-9

International classification of diseases, ninth revision

ICD-10

International classification of diseases, tenth revision

ICU

Intensive care unit

IHD

Ischaemic heart disease

INR

International normalised ratio

kNN

k nearest neighbours

LASSO

Least absolute shrinkage and selection operator

LightGBM

Light gradient boosting machine

LOS

Length of stay

LR

Logistic regression

MAP

Mean arterial pressure

MI

Myocardial infarction

MIMIC-IV

Medical information mart for intensive care IV

ML

Machine learning

MV

Mechanical ventilation

NIH

National institutes of health

NN

Neural network

NPV

Negative predictive value

OASIS

Oxford acute severity of illness score

PaCO2

Partial pressure of carbon dioxide

PaO2

Partial pressure of oxygen

PNA

Pneumonia

PO2

Partial pressure of oxygen

PPV

Positive predictive value

PR-AUC

Area under the precision–recall curve

PT

Prothrombin time

RBC

Red blood cell count

RDW

Red cell distribution width

RF

Random forest

RR

Respiratory rate

SAPS II

Simplified acute physiology score II

SHAP

Shapley additive explanations

SIRS

Systemic inflammatory response syndrome

SMOTE

Synthetic minority over-sampling technique

SOFA

Sequential organ failure assessment

SVM

Support vector machine

T1DM

Type 1 diabetes mellitus

T2DM

Type 2 diabetes mellitus

TBil

Total bilirubin

TCO2

Total carbon dioxide

WBC

White blood cell count

XGBoost

Extreme gradient boosting

Authors’ contributions

JY: Writing – original draft, Data curation, Formal analysis. TH: Writing – original draft, Data curation, Formal analysis. HL: Methodology, Supervision, Writing – review & editing. JL: Conceptualization, Funding acquisition, Supervision, Validation, Writing – review & editing. KZ: Validation, Resources, Writing – review & editing. TQ: Methodology, Visualization, Writing – review & editing. All authors were involved in writing the paper and had final approval of the submitted and published versions.

Funding

This study was financially supported by the National Natural Science Foundation of China (Grant No.82360372), the Guangxi Zhuang Autonomous Region Health Commission Self-funded Research Program (Grant No. Z-R20251623), the Guigang Science and Technology Project (GKJ2203022, GKG2300032) and the Youth Science Foundation of Guangxi Medical University (No.GXMUYSFB2026063).

Data availability

Publicly available data sets were analyzed in this study. These data can be found here: (https:/mimic.mit.edu/docs/iv) and https://physionet.org/content/eicu-crd/2.0/.

Declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.