Construction and Validation of a Hospital Mortality Risk Model for Advanced Elderly Patients with Heart Failure Based on Machine Learning

Abstract

Purpose

This study aimed to develop and validate a model based on machine learning algorithms to predict the risk of in-hospital death among advanced elderly patients with Heart Failure (HF).

Methods

A total of 4580 advanced elderly patients who were admitted to the hospital and diagnosed with HF from May 2012 to September 2023 were included in this study, among whom 552 cases (12.5%) died. The least absolute shrinkage and selection operator (LASSO) regression and Boruta feature selection were used to screen the baseline variables to identify the variables significantly associated with death. Subsequently, seven different machine learning models were constructed and their prediction performances were evaluated. The Shapley Additive Explanations (SHAP) values were used to analyze the impact of key variables on the model prediction results.

Results

A total of seven variables significantly associated with death were selected by LASSO regression and Boruta feature selection, including white blood cell count (WBC), neutrophil percentage (Neut %), C-reactive protein (CRP), D-dimer, glycated serum protein (GSP), N-terminal pro-B-type natriuretic peptide (NT-ProBNP), and body mass index (BMI). Among all the models, the extreme gradient boosting (XGB) model performed the best, with an area under the curve (AUC) value of 0.933, a sensitivity of 0.79, a specificity of 0.89, a recall of 0.79, and an F1 score of 0.59 on the validation set. The SHAP analysis showed that CRP, BMI, NT-ProBNP, D-dimer, and GSP were the main influencing factors for death.

Conclusion

This study successfully constructed a prediction model for the in-hospital death risk of advanced elderly patients with HF, and the XGB model exhibited excellent prediction performance. This model can be used for the early clinical identification of high-risk patients and thus provide support for individualized treatment strategies.

Introduction

Heart failure (HF) is a severe manifestation or the advanced stage of various cardiac diseases, with persistently high mortality and hospitalization rates. The prevalence of HF in adults in developed countries is approximately 1–2%. In the China-HF study, the case fatality rate of hospitalized patients with HF was 4.1%.^1,2 As the trend of global population aging intensifies, the number of elderly patients with HF is constantly increasing, which not only has a significant impact on the quality of life of patients but also brings a huge burden to the medical system. According to the World Health Organization (WHO), the elderly are usually defined as individuals aged 65 years and above, while those aged 75 years and above are considered to belong to the advanced elderly category.³ For these advanced elderly patients, the assessment of the in-hospital death risk is a crucial challenge in clinical work. Accurate risk assessment can assist doctors in devising better treatment plans and making rational use of medical resources, thereby improving the treatment outcomes.

Traditional risk assessment tools, such as the Framingham score,⁴ can provide certain guidance for clinical practice. However, their predictive ability is limited. Especially when faced with the high-risk group of advanced elderly patients with HF, they often fail to fully capture the individual differences of patients. In recent years, with the progress of big data analysis technology, machine learning methods have gradually emerged in the medical field and have shown great potential, especially in complex multivariate data processing and disease risk prediction.⁵ In view of this, based on retrospective cohort data, this study used the least absolute shrinkage and selection operator (LASSO) regression and Boruta feature selection for variable selection, and constructed multiple machine learning models. It aimed to explore the prediction models for the in-hospital death risk of advanced elderly patients with HF, evaluate the prediction performance of different algorithms, and analyze the important features of the models through SHAP values, with the expectation of providing an effective early risk assessment tool for clinical practice and facilitating the formulation of individualized treatment strategies.

Materials and Methods

Study Design and Population Selection

This study was a single-center, retrospective case-control study. Advanced elderly patients with HF who were hospitalized in the First Affiliated Hospital of Xinjiang Medical University from May 2012 to September 2023 were included. The inclusion criteria included: (1) age ≥ 75 years; (2) being diagnosed with HF during this admission or having a previous diagnosis of HF; (3) having complete clinical data during hospitalization. The exclusion criteria were: (1) pulmonary hemorrhage; (2) septic shock; (3) respiratory failure; (4) disseminated intravascular coagulation; (5) missing key clinical data. According to the above inclusion and exclusion criteria, patients were divided into the death group and the survival group based on their survival status during hospitalization. The detailed process is shown in Figure 1. This study has been approved by the Ethics Committee of the First Affiliated Hospital of Xinjiang Medical University (Ethics Approval Number: 231124-05) and was conducted in accordance with the principles of the Declaration of Helsinki. Since this study was a retrospective study and used anonymized existing data without imposing additional risks on patients, the informed consent was waived.

Figure 1.

Flowchart.

Data Collection

All the data in this study were obtained from the hospital information management system. We collected the following clinical data from the hospitalization medical records of elderly patients with heart failure, including: (1) Basic demographic characteristics, such as age, gender, body mass index (BMI), smoking history, and drinking history; (2) Past medical histories, including diabetes, stroke, and hypertension; (3) Medication histories, including beta-blockers, angiotensin-converting enzyme inhibitors (ACEI) / angiotensin receptor blockers (ARB) / angiotensin receptor neprilysin inhibitors (ARNI), spironolactone, and furosemide; (4) Echocardiographic examinations, including left atrial diameter (LAD), left ventricular end diastolic diameter (LVEDD), and left ventricular ejection fraction (LVEF); (5) Blood laboratory tests, including white blood cell count (WBC), red blood cell count (RBC), hemoglobin (Hb), neutrophil percentage (Neut %), platelet count (PLT), C-reactive protein (CRP), alanine aminotransferase (ALT), aspartate amino transferase (AST), gamma-glutamyltransferase (GGT), direct bilirubin (DBIL), indirect bilirubin (IBIL), albumin (ALB), globulin (GLO), creatinine (Crea), urea, uric acid (UA), total cholesterol (TC), triglyceride (TG), high density lipoprotein cholesterol (HDL-C), low density lipoprotein cholesterol (LDL-C), serum glucose (Glu), D-dimer, glycosylated serum protein (GSP), N-terminal pro-brain natriuretic peptide (NT-ProBNP), creatine kinase (CK), and estimated glomerular filtration rate at admission (eGFR).

Feature Selection

During the construction process of the prediction model, the LASSO regression was first used for feature selection. LASSO can effectively shrink the coefficients of insignificant variables to zero by introducing L1 regularization, thus screening out the variables that have the most influence on the prediction results. In this study, all variables were included in the LASSO model, and the lambda value (lambda.1se) of the model within the range of one standard error of the optimal value was selected based on 10-fold cross-validation to ensure the robustness and predictive ability of the model. The variables screened by LASSO regression were then included in the Boruta feature selection again. Based on the feature selection algorithm of random forest and after 100 iterations, the variables with the strongest predictive power for the target variable were identified. Finally, the variables with a medianImp ≥ 10 in the Boruta feature selection were included for the construction of the model.

Model Construction and Validation

Based on the selected feature variables, seven machine learning algorithms were respectively used to construct prediction models, including decision tree (DT), random forest (RF), extreme gradient boosting (XGB), support vector machine (SVM), K-nearest neighbors (KNN), Light Gradient Boosting Machine (LGBM), and logistic regression (LR). The dataset was randomly divided into a training set and a validation set at a ratio of 7:3. During the training process, 5-fold cross-validation was adopted to reduce the risk of model overfitting. The screened variables were incorporated into the prediction models, and the Receiver Operating Characteristic (ROC) curve, calibration curve, and Decision Curve Analysis (DCA) were constructed to evaluate the discrimination, effectiveness, and clinical practicability of the models. Furthermore, the validation set was used to construct the ROC curve, calibration curve, and DCA curve again to validate these models.

Model Evaluation

The predictive effect of the models was evaluated by the following indicators: (1) Area Under the Receiver Operating Characteristic Curve (AUC), which was used to evaluate the discrimination ability of the models; (2) Sensitivity, specificity, accuracy, balanced accuracy (BA), and recall, which were used to evaluate the comprehensive performance of the models under different classification situations. In addition, the Shapley Additive Explanations (SHAP) was used to interpret the models, determine the contribution of each feature to the risk of death during hospitalization, and further improve the transparency and clinical interpretability of the models.

Statistical Analysis

All data analyses were conducted using R software (version 4.2.1). After the normality test for continuous variables, those conforming to the normal distribution were expressed as mean ± standard deviation (x ± s), and the independent sample t-test was used for comparison between two groups. For those not conforming to the normal distribution, they were expressed as median (P₂₅, P₇₅), and the Mann–Whitney U-test or Kruskal–Wallis test was employed for comparison between groups. Categorical variables were expressed as frequency (%), and the χ²-test was used for comparison between groups. The predictive ability of the model was evaluated by the receiver operating characteristic curve (ROC curve). A P value less than 0.05 indicated that the difference was statistically significant.

Results

Baseline Features

According to the inclusion and exclusion criteria, this study finally analyzed 4580 elderly patients with HF (Figure 1). Among them, 552 patients died during hospitalization, with an incidence rate of 12.05%. Compared with the surviving patients, the patients in the death group had a significantly higher average age (P < 0.001), and the proportion of patients with comorbidities such as diabetes and stroke was also significantly higher (P < 0.05). In addition, the results of laboratory tests showed that the levels of WBC, Neut %, CRP, D-Dimer, and NT-ProBNP in the death group were significantly higher than those in the survival group (P < 0.05). For the detailed statistical results of the specific baseline characteristics, please refer to Table 1 and Supplement Table 1. The dataset was randomly divided into a training set (n = 3206) and a validation set (n = 1374) at a ratio of 7:3. There were no statistically significant differences in the above indicators, suggesting that the division between the training set and the validation set was relatively balanced (Table 2 and Supplement Table 2).

Table 1.

Death Group and Survival Group at Baseline

Variables	Total (n = 4580)	Survival Group (n = 4028)	Death Group (n = 552)	P
WBC, 109/L	7.11 (5.53, 9.33)	6.97 (5.48, 9)	8.63 (6.04, 12.1)	< 0.001
Neut %, %	70.2 (60.98, 80.4)	69.4 (60.6, 79)	78.3 (66.15, 87.4)	< 0.001
CRP, mg/L	26.54 (18.87, 33.56)	25.51 (17.99, 31.22)	45.62 (31.72, 57.67)	< 0.001
D-Dimer, ng/mL	733 (304, 1097)	693 (286.75, 1025)	1212.5 (577.5, 2815.25)	< 0.001
GSP, mmol/L	2.46 (1.95, 5.3)	2.47 (1.94, 5.55)	2.43 (2, 3.34)	0.035
NT-ProBNP, ng/L	3244.5 (1332.5, 5532.25)	3046.5 (1233.75, 5160.5)	5493 (2673, 6862.75)	< 0.001
BMI, Kg/m2	24.06 (21.29, 27.47)	23.94 (21.22, 27.06)	25.02 (22.08, 35.83)	< 0.001

Table 2.

Training Set and Validation Set Baseline

Variables	Total (n = 4580)	Validation Set (n = 1374)	Training Set (n = 3206)	P
Death, n (%)	552 (12)	153 (11)	399 (12)	0.231
WBC, 109/L	7.11 (5.53, 9.33)	7.16 (5.64, 9.24)	7.08 (5.48, 9.39)	0.363
Neut %, %	70.2 (60.98, 80.4)	70.4 (61.32, 79.6)	70.1 (60.9, 80.7)	0.929
CRP, mg/L	26.54 (18.87, 33.56)	26.26 (18.9, 32.72)	26.63 (18.83, 34)	0.174
D-Dimer, ng/mL	733 (304, 1097)	732 (302.25, 1091.75)	733.5 (305.25, 1100)	0.747
GSP, mmol/L	2.46 (1.95, 5.3)	2.51 (1.97, 5.57)	2.43 (1.94, 5.27)	0.113
NT-ProBNP, ng/L	3244.5 (1332.5, 5532.25)	3394.5 (1395, 5595.25)	3178 (1290, 5512.75)	0.187
BMI, Kg/m2	24.06 (21.29, 27.47)	24.17 (21.23, 27.47)	24.03 (21.3, 27.48)	0.985

Feature Selection Results

Variable selection was carried out through the LASSO regression and Boruta feature selection. Eventually, seven variables that were closely associated with the death of elderly patients with HF during hospitalization were screened out, namely WBC, Neut %, CRP, D-dimer, GSP, NT-ProBNP, and BMI (Figure 2 and Supplement Table 3).

Figure 2.

Variable screening diagram. (A) LASSO regression solution path roadmap; (B) The LASSO regression error plot; (C) Boruta Feature screening variable selection map; (D) Boruta Details diagram of feature screening.

Construction and Validation of Machine Learning Models

Seven machine learning models, including DT, KNN, LGBM, LR, RF, SVM, and XGB, were constructed based on the important variables screened out by the LASSO regression and Boruta feature selection. The predictive performances of each model showed significant differences both in the training set and the validation set. The XGB model exhibited relatively high AUC values in both the training set and the validation set, which were 0.951 and 0.933 respectively (Figure 3), indicating that this model had good discrimination ability. Meanwhile, the calibration curve of the XGB model suggested that there was no significant difference between the predicted values and the observed values, and the model had a good fit. The results of the DCA curve showed that when the risk threshold was within a relatively wide range, the net benefit rate of this model was clinically significant (Figure 3). For the detailed predictive capabilities of the models, please refer to Figure 4 and Supplement Table 4.

Figure 3.

Model construction and validation. Figures (A and D) display Receiver Operating Characteristic (ROC) curves on the training and validation sets respectively, with sensitivity on the vertical axis and 1 - specificity on the horizontal axis, and the area under the curve (AUC) quantifies model performance. Figures (B and E) illustrate the relationship between predicted and observed probabilities for calibration assessment, where an ideal curve aligns with the diagonal. Figures (C and F) are Decision Curve Analysis (DCA) plots showing net benefit against threshold, comparing model - guided decision - making with “treat all” and “treat none” strategies.

Figure 4.

Model evaluation. Figure 4 evaluates machine - learning models’ performance. (A and C) Are radar charts for training and validation sets respectively, showing metrics like Sensitivity, Specificity, and F1 for models such as DT, KNN, etc. (B and D) Are corresponding heatmaps, with color - coding to indicate metric values, helping to visually compare model performance across different sets.

Variable Importance and SHAP Interpretability Analysis

In the XGB model, the variable importance ranked in descending order is as follows: CRP, BMI, NT-ProBNP, D-Dimer, GSP, Neut %, WBC, suggesting that they are the key factors for predicting the risk of death occurrence. To better understand the prediction results of the model, we conducted SHAP interpretability analysis on the XGB model (Figure 5). In addition, the impact of individual variables on the prediction of the XGB model was also discussed. As shown in Figure 6, the y-axis represents the SHAP value of the feature, and the x-axis represents the input value of the feature. A SHAP value exceeding 0 indicates an increased risk of death. We also used single-sample force plots to explain the contribution of features to individuals. As shown in Figure 7A, for the correct prediction of in-hospital death, based on higher BMI, CRP, Neut, NT-ProBNP, and lower D-Dimer, WBC, the total SHAP value is higher than the base value. Conversely, when correctly predicting no death, lower NT-ProBNP, CRP, D-Dimer, GSP, Neut, and WBC all contribute to the negative prediction (Figure 7B). Overall, the contribution of features to individual patients is consistent with the overall results of the model, indicating that these seven variables are the most significant variables affecting the prediction results.

Figure 5.

XGBoost Model SHAP values. Figure 5 shows feature analysis via SHAP values. Figure (A) is a feature importance plot for features like CRP, BMI, etc. Bar length indicates importance, with CRP being most important. Figure (B) is a feature dependence plot. Points, colored by feature value, show the relationship between feature values and SHAP values, helping understand feature impact on model prediction.

Figure 6.

The XGBoost Model SHAP dependency.

Figure 7.

XGBoost Model single-sample force map. Figure 7 shows SHAP - based explanations for model predictions. Figure (A) displays positive SHAP values for features like BMI, CRP, etc., indicating their positive contributions to the prediction result f(x)=3.93. Figure (B) presents negative SHAP values for features such as NT - ProBNP, CRP, etc., showing their negative impacts on the prediction result f(x)=−3.81.

Discussion

In this study, we constructed and validated for the first time a prediction model for the in-hospital death risk of advanced elderly patients with HF based on machine learning algorithms. The results showed that a total of seven variables significantly associated with death were selected through the LASSO regression and Boruta feature selection. Among the seven machine learning models, the XGB model performed the best. The SHAP analysis indicated that CRP, BMI, NT - ProBNP, D - Dimer, and GSP were the main influencing factors for death.

The variables screened out in this study are of crucial significance in the condition assessment and death risk prediction for elderly patients with HF. As a sensitive marker of inflammatory response, the elevated level of CRP is significantly associated with an increased risk of death among elderly patients with heart failure.⁶ Inflammatory response plays an important role in the pathophysiological process of HF. It can promote myocardial cell damage and intensify fibrosis, thereby accelerating the deterioration of cardiac function. Burger et al research suggests that CRP is an independent risk marker for new-onset HF in patients with cardiovascular disease.⁷ High levels of CRP reflect the persistent inflammatory state within the patient’s body. This not only aggravates the burden on the heart but also may trigger a series of adverse cardiovascular events, ultimately leading to an increased risk of death.⁸

BMI is closely related to the occurrence and prognosis of HF.⁹ The results of the Framingham Heart Study suggest that for every 1 kg/m² continuous increase in BMI, the risk of developing HF increases by 5% in men and 7% in women, respectively.¹⁰ Obesity may fundamentally lead to HF through hemodynamic changes related to the activation of the renin-angiotensin-aldosterone system, increased activity of the sympathetic nervous system and mineralocorticoid receptor expression, as well as the production of inflammatory cytokines and acute-phase proteins.¹¹ However, in patients already suffering from HF, some studies have shown that overweight and obesity are associated with a better prognosis, thus giving rise to the “obesity paradox”¹² The results of this study also suggest that an extremely high BMI increases the risk of in-hospital death from HF. This may be because an excessively high BMI will subject the heart to a greater pressure load, increase myocardial oxygen consumption, and lead to the progression and deterioration of HF.

It is evident that in this study, the level of NT - ProBNP in patients was closely related to the risk of in-hospital death, and similar results have also been presented in other studies.^13,14 Recent studies have shown that the level of NT - ProBNP at the time of discharge is associated with the prognosis of patients with HF and is an independent risk factor for out-of-hospital death.¹⁵ The elevation of NT - ProBNP concentration directly reflects the degree of cardiac function impairment, indicating that the patient’s condition is more severe and the risk of adverse cardiovascular events is higher, and thus is closely linked to the risk of death. Therefore, in clinical practice, once encountering patients with extremely high NT - ProBNP levels, it is necessary to attach great importance and correct cardiac function in a timely manner.

In this study, the close association between D-Dimer and GSP levels with in-hospital death of HF patients is a focus worthy of attention. D-Dimer not only serves as a coagulation marker, but also plays an important role in the inflammatory response and myocardial remodeling of HF. Studies have shown that D-Dimer may exacerbate myocardial injury by promoting the activation and adhesion of inflammatory cells, which is closely related to its value in predicting the risk of death.^16–18 While a long - term hyperglycemic state can lead to a variety of metabolic disorders, resulting in vascular endothelial damage, increased oxidative stress, and aggravated inflammatory responses, which in turn affect the cardiac microvascular circulation and myocardial metabolism.^19–21 These pathophysiological changes will further deteriorate cardiac function and increase the risk of death in patients with HF.

Elevated WBC and Neut% indicate that HF patients may have an infection or an inflammatory state, which is closely associated with an increased risk of death. In a recent study by Wang et al, the results suggested that an elevated neutrophil - to - albumin ratio (NPAR) is independently associated with short - term and long - term all - cause mortality in HF patients. NPAR can serve as an inflammatory marker to predict the clinical prognosis of heart failure patients.²² Infection is one of the important inducers for the deterioration of the condition and death of patients with HF. It can trigger the systemic inflammatory response syndrome, aggravate the burden on the heart, and lead to a sharp decline in cardiac function.^23,24 Meanwhile, the inflammatory response may also induce complications such as arrhythmia, further increasing the risk of death for patients.

This study is innovative in multiple aspects. Firstly, by combining the LASSO regression with Boruta feature selection, the accuracy and comprehensiveness of variable selection have been significantly improved. Key variables highly relevant to the in-hospital death risk of advanced elderly patients with HF have been effectively screened out, laying a solid foundation for constructing a precise prediction model. Secondly, seven machine learning algorithms were comprehensively compared, and finally, it was determined that the XGB model performed optimally in this study, enriching the research methods in this field. Finally, the SHAP analysis was conducted to deeply interpret the model results, quantify the contributions of various factors, and enhance the interpretability of the model, providing an intuitive basis for clinical decision-making, which is relatively rare among similar studies.

In summary, the in-hospital death risk prediction model for advanced elderly patients with HF constructed in this study, especially the XGB model, exhibited outstanding predictive performance and possessed significant clinical application value. The research findings not only assist clinicians in early identification of high-risk patients but also provide strong support for the formulation of individualized treatment strategies, holding the potential to improve the clinical management and prognosis of advanced elderly patients with HF. Future research should focus on overcoming existing limitations, continuously optimizing the model to further enhance its accuracy, interpretability, and broad applicability, and promoting the improvement of the quality of medical services for advanced elderly patients with HF.

Limitation

This study was a single-center retrospective study, with a risk of selection bias and limited sample representativeness, which restricted the extrapolation of the results. Future multicenter, prospective studies are needed to validate the effectiveness and universality of the model. In addition, although a rich set of clinical variables were included, potential factors affecting prognosis such as patients’ psychological states and socioeconomic factors might still have been overlooked. Future studies should consider incorporating these variables to improve the comprehensiveness of the model. Finally, although the SHAP analysis enhanced the interpretability of the model, the interpretation of the results still requires further simplification to enhance its convenience for application in clinical practice.

Conclusion

This study successfully developed and validated a machine learning-based prediction model to assess in-hospital mortality risk among advanced elderly patients with heart failure (HF). Utilizing LASSO regression and Boruta feature selection, seven key predictors—CRP, BMI, NT-ProBNP, D-dimer, GSP, Neut%, and WBC—were identified as significantly associated with mortality. Among seven machine learning algorithms evaluated, the XGBoost (XGB) model demonstrated superior predictive performance, achieving an AUC of 0.933, sensitivity of 0.79, and specificity of 0.89 on the validation set. SHAP analysis further elucidated the clinical interpretability of the model, highlighting CRP, BMI, NT-ProBNP, D-dimer, and GSP as the most influential variables. These findings underscore the potential of integrating machine learning with clinical biomarkers to enhance early risk stratification and guide personalized therapeutic strategies for high-risk elderly HF patients.

Funding Statement

This work was supported by the Key R&D Program of Xinjiang Uygur Autonomous Region (2022B03023).

Data Sharing Statement

The data that support the findings of this study were 4580 patients from clinical patients. Specific clinical data information can be further obtained by contacting the corresponding author. Further clinical studies are currently being conducted on the basis of these data, and therefore it is not feasible to make the data public at this time.

Ethics Approval and Informed Consent

We certify that the research study titled [Construction and Validation of a Hospital Mortality Risk Model for Advanced Elderly Patients with Heart Failure Based on Machine Learning] has been approved by the Medical Ethics Committee of the First Affiliated Hospital of Xinjiang Medical University. The approval number and date of approval are as follows: [231124-05] and [17th November, 2023].

Author Contributions

All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this work.