A method for predicting mortality in acute mesenteric ischemia: Machine learning

ABSTRACT

BACKGROUND:

This study aimed to develop and validate an artificial intelligence model using machine learning (ML) to predict hospital mortality in patients with acute mesenteric ischemia (AMI).

METHODS:

A total of 122 patients diagnosed with AMI at Sakarya University Training and Research Hospital between January 2011 and June 2023 were included in the study. These patients were divided into a training cohort (n=97) and a validation cohort (n=25), and further categorized as survivors and non-survivors during hospitalization. Serum-based laboratory results served as features. Hyperfeatures were eliminated using Recursive Feature Elimination (RFE) in Python to optimize outcomes. ML algorithms and data analyses were performed using Python (version 3.7).

RESULTS:

Of the patients, 56.5% were male (n=69) and 43.5% were female (n=53). The mean age was 71.9 years (range 39-94 years). The mortality rate during hospitalization was 50% (n=61). To achieve optimal results, the model incorporated features such as age, red cell distribution width (RDW), C-reactive protein (CRP), D-dimer, lactate, globulin, and creatinine. Success rates in test data were as follows: logistic regression (LG), 80%; random forest (RF), 60%; k-nearest neighbor (KN), 52%; multilayer perceptron (MLP), 72%; and support vector classifier (SVC), 84%. A voting classifier (VC), aggregating votes from all models, achieved an 84% success rate. Among the models, SVC (sensitivity 1.0, specificity 0.77, area under the curve (AUC) 0.90, Confidence Interval (95%): (0.83-0.84)) and VC (sensitivity 1.0, specificity 0.77, AUC 0.88, Confidence Interval (95%): (0.83-0.84)) were noted for their effectiveness.

CONCLUSION:

Independent risk factors for mortality were identified in patients with AMI. An efficient and rapid method using various ML models to predict mortality has been developed.

INTRODUCTION

Acute mesenteric ischemia (AMI) is a condition characterized by interrupted blood flow to the intestine, leading to ischemia and subsequent inflammation. Although it is rare, its incidence increases with age, accounting for approximately 0.09-0.2% of all acute surgical cases.[1] The non-specific nature of its symptoms makes diagnosis difficult, significantly impacting mortality rates. Currently, contrast-enhanced computed tomography (CT) is utilized for its diagnosis.[2] Despite advancements in surgical techniques and endovascular treatments, mortality rates can climb as high as 50% if treatment is delayed.[3] Consequently, some studies have focused on identifying high-risk patients early by analyzing risk factors associated with hospital mortality from AMI.[4,5] However, the determinants of early mortality and prognosis in AMI are not completely understood, and the risk of hospital death remains difficult to predict accurately. Therefore, improving mortality prediction in AMI patients could provide more accurate clinical information for patients and their families, optimize treatment management, and inform future research. Accurate predictions may also help manage medical costs more effectively and maximize the allocation of medical resources.

Machine learning (ML) algorithms analyze historical data, extract useful information, and learn from patterns to make diagnoses and predictions.[6] Increasingly, ML is applied in medicine to derive insights from data and facilitate predictions.[7,8] These algorithms can analyze growing amounts of biological data, identify treatment targets, and design new therapeutic compounds.[9] Additionally, ML can diagnose diseases and predict risk factors and mortality rates.[10-12]

Our goal is to develop and compare models utilizing various machine learning classification algorithms to predict hospital mortality among AMI patients and to validate the performance of these models.

MATERIALS AND METHODS

This study was approved by the Local Ethics Committee (No. 71522473/050.01.04/318692-404; date: 28. 12. 2023). It included all 122 patients diagnosed with AMI at our center between January 2011 and June 2023. AMI diagnosis was confirmed using angiography-computed tomography. At diagnosis, features such as serum-based laboratory results (hemogram, C-reactive protein (CRP), albumin, D-dimer, fibrinogen, lactate, globulin, creatinine, International Normalized Ratio (INR)), age, and the Charlson Comorbidity Index (CCI) were collected. Patients were categorized into two groups: survivors and non-survivors during hospitalization. They were further divided into a training cohort (n=97) and a validation cohort (n=25). In Python, hyperfeatures were eliminated using Recursive Feature Elimination (RFE) to achieve optimal results. Models created included logistic regression (LG), random forest (RF), k-nearest neighbors (KN), multilayer perceptron (MLP), support vector classifier (SVC), and voting classifier (VC).

Statistical Analysis

Traditional statistical methods were employed to identify differences between survivors and non-survivors of AMI. The conformity of variables to normal distribution was assessed using the Kolmogorov-Smirnov and Shapiro-Wilk tests. Descriptive statistics were applied, utilizing mean and standard deviation for normally distributed variables, and median and interquartile range for non-normally distributed variables. To detect significant differences between survivors and non-survivors of AMI, the Independent Samples t-test or Mann-Whitney U-test was used for numerical variables, and the Chi-square test for categorical variables. Statistical analysis was conducted using IBM SPSS (Statistical Package for the Social Sciences) Statistics (Version 25.0, Armonk, NY: IBM Corp). A p-value less than 0.05 was considered statistically significant.

The performance of each ML model was evaluated using a confusion matrix. ML algorithms and data analysis were conducted using Python (Version 3.7, Wilmington, Delaware, USA: PSF).

RESULTS

Among the patients, 56.5% were male (n=69) and 43.5% were female (n=53). The mean age was 71.9 years (range 39-94 years). The mortality rate during hospitalization was 50% (n=61). Significant differences were observed between survivors and non-survivors in terms of age, CCI, red cell distribution width (RDW), D-dimer, lactate, creatinine, and INR (Table 1).

Table 1.

Comparison of characteristics of survivors and non-survivors

Patient characteristics	Survivors (n=61)	Non-survivors (n=61)	p-value
Age	69.68±13.39	74.21±12.02	<0.05**
Gender
Male	37 (53.6%)	32 (46.4%)	0.46***
Female	24 (45.3%)	209 (54.7%)
Charlson comorbidity index	4 (0-8)	5.00 (0.0-10.0)	<0.05*
Neutrophil (K/uL)	15.73±8.60	15.59±8.57	0.92**
Platelet (K/uL)	247.21±81.12	251.77±100.89	0.78**
Lymphocyte (K/uL)	1.35±1.05	1.25±1.35	0.65**
Mean platelet volume (fl)	9.34±1.80	9.64±2.37	0.43**
Red cell distribution width (%)	15.4 (11.4-33.8)	16.9 (11.75-30.70)	<0.05*
White blood cell (K/uL)	18.34±9.109	17.88±8.96	0.77**
Monocyte (K/uL)	0.99±0.56	0.96±0.78	0.81**
C-Reactive protein (mg/L)	110 (0.53-458)	196.00 (2.40-519)	0.12*
Albumin (g/dL)	3.15±0.63	3.08±0.86	0.66**
Globulin (g/dL)	2.73±0.40	2.81±0.59	0.46**
D-Dimer (ugFEU/L)	1385 (257-7550)	3670 (869-18200)	<0.05*
Fibrinogen (g/L)	6.31 (3.05-497)	7.46 (0.32-415)	0.88*
Lactate (mmol/L)	2.40 (0.90-11.3)	6.7 (1.30-19.0)	<0.001*
Creatinine (mg/dL)	1.07 (0.46-3.32)	1.51 (0.44-6.29)	<0.001*
INR	1.24 (0.85-12)	1.41 (0.87-12)	<0.001*

^*Mann-Whitney U Test.

^**Independent Samples T-Test.

^***Chi-square Test.

Recursive Feature Elimination identified age, RDW, CRP, D-dimer, lactate, globulin, and creatinine as the most relevant features for optimal results. The coefficients were as follows: -0.41 for age, -0.44 for RDW, -0.24 for CRP, -0.33 for D-dimer, -0.53 for lactate, 0.05 for globulin, and -0.75 for creatinine. The success rates in the test data were: 80% for logistic regression (LG), 60% for random forest (RF), 52% for k-nearest neighbor (KN), 72% for multilayer perceptron (MLP), 84% for support vector classifier (SVC). The voting classifier (VC), which integrated all models, also achieved an 84% success rate. Among the models, SVC demonstrated high effectiveness with a sensitivity of 1.0, a specificity of 0.77, and an area under the curve (AUC) of 0.90, with a confidence interval of 95% (0.83-0.84), and VC showed a sensitivity of 1.0, a specificity of 0.77, and an AUC of 0.88 with a confidence interval of 95% (0.83-0.84) (Table 2, Fig. 1).

Table 2.

Comparison of models

Models	Sensitivity	Specificity	AUC	Confidence Interval (95%)
Logistic regression	1.0	0.72	0.88	0.79-0.80
Random forest	0.71	0.55	0.74	0.58-0.60
K-Nearest neighbor	1.0	0.33	0.85	0.51-0.52
MLP	0.71	0.72	0.76	0.71-0.72
SVC	1.0	0.77	0.90	0.83- 0.84
Voting classifier	1.0	0.77	0.88	0.83- 0.84

Figure 1.

Receiver operating characteristic (ROC) curve of models.

DISCUSSION

This study utilized various ML models to predict hospital mortality in patients with acute mesenteric ischemia. Our findings indicate that ML models can accurately predict hospital mortality in AMI patients.

Increases in age, RDW, CRP, D-dimer, lactate, and creatinine, along with a decrease in globulin, were identified as independent risk factors for mortality in AMI. Literature recognizes age as a risk factor for mortality in AMI cases,[13,14] which supports the findings of our study. Additionally, lactate is an important parameter often associated with hypoxia, necrosis, and inflammation. Consistent with our findings, other studies have identified lactate as a risk factor for mortality in AMI.[4,14] Similarly, elevated serum creatinine levels have been linked to increased mortality in AMI cases.[15] In our research, we also found that elevated RDW was a risk factor. Although previous studies have associated elevated RDW with AMI prognosis, it has also been linked to sepsis in other research.[16] This may be due to damage to the intestinal mucosa, which compromises its resistance to bacteria and leads to sepsis. This could explain the poor prognosis in AMI patients with elevated RDW. Destek S. et al. identified CRP as a potential prognostic biomarker in AMI,[17] and Gorla et al. reported that D-dimer served as a prognostic marker in patients with aortic syndrome.[18] Contrarily, the literature reports that globulin levels increase in chronic inflammation, with elevated globulin levels associated with poor prognosis, especially in cancer patients.[19,20] Among the features selected by RFE, only globulin differs from those commonly reported in the literature. In studies involving acute ischemic stroke patients, elevated globulin levels were also linked to a poor prognosis.[21] However, there are no comprehensive studies using globulin in the context of AMI or sepsis for comparison. In our study, the most potent feature was creatinine, while globulin was the least effective.

RFE operates by eliminating redundant and weak features to minimize training errors. It is an independent and powerful technique that enhances the model’s generalization performance.[22] It starts by building a model with all features, ranking each according to its importance. The process then involves removing the least essential feature, rebuilding the model, and recalculating its importance. This method quickly calculates the combination of all features until the best performance is achieved.[23] RFE has been used in numerous medical studies[24-26] and selects the most potent features. It improves model performance by ranking many features based on a specific ML method.[27]

Our study utilized well-established basic classifier algorithms commonly referenced in medical research.[28-31] However, each algorithm has its advantages and disadvantages. RF and KN require large datasets. As the number of independent variables increases, KN’s performance significantly slows, adversely affecting its accuracy. In RF, each tree makes a prediction, and these predictions are collectively voted on to derive the final result. This method tends to perform slowly and may not yield optimal outcomes with complex datasets. It is more effective with larger datasets and in image analysis.[32,33] On the other hand, MLP are versatile and applicable to different datasets, but they may underperform due to overfitting in the training data. Additionally, there is a risk of getting stuck in local minima during optimization.[34] SVC performs well with clear separation margins and in high-dimensional spaces, offering faster predictions and greater accuracy. However, they are unsuitable for large datasets due to slow learning times.[34] LG is easy to implement and interpret, and performs well if the dataset can be linearly separated. It is less prone to overfitting but can still overfit in large datasets.[35]

To evaluate the predictive performance of our models, we used accuracy, sensitivity, and specificity metrics, calculated through a confusion matrix that includes True Negative, False Negative, True Positive, False Positive values. The receiver operating characteristic (ROC) curve was used to assess the models. The discriminative power of the prediction models was evaluated by the area under the curve (AUC), accompanied by a 95% confidence interval (CI). The methods we employed to evaluate the model are frequently used and have been validated in the literature.[23,36,37]

To our knowledge, no specific scoring system exists for AMI. The Acute Physiology and Chronic Health Evaluation (APACHE) score, Simplified Acute Physiology Score (SAPS), and Sequential Organ Failure Assessment (SOFA) score are employed to predict outcomes in intensive care patients. However, they are insufficient for predicting the prognosis in AMI cases. In a study involving 82 AMI patients, Yilmaz A. et al. found that APACHE, SAPS, and SOFA scores were inadequate for predicting prognosis.[38] In this context, six different ML predictive models were developed, among which the SVC demonstrated the highest performance (sensitivity: 1.0, specificity: 0.77, AUC: 0.90, Confidence Interval (95%): 0.83-0.84).

The limitations of our study include its retrospective nature and the restriction to a single center. Additionally, data regarding the duration of symptoms and vital signs at the time of diagnosis were not available in our digital data system.

CONCLUSION

In conclusion, by utilizing conventional statistical methods and routine blood tests, we identified increased age, CCI, RDW, D-Dimer, lactate, creatinine, and INR as risk factors for mortality in AMI. Additionally, we developed an inexpensive, accurate, and fast artificial intelligence model to predict hospital mortality in AMI cases. We believe this model can offer significant clinical benefits. Moreover, ML methods can be utilized in the diagnosis and prognosis prediction of various other diseases. Therefore, there is a need for prospective multicenter studies.