Abstract
Objective
To evaluate the performance fairness of a long short-term memory (LSTM) model in predicting in-hospital mortality for inpatients with varying severity, as reflected by length of stay (LOS) and initial clinical scores.
Materials and methods
This retrospective study used the Medical Information Mart for Intensive Care (MIMIC)-IV database, which includes records from over 50,000 ICU patients. Patients were divided into subgroups based on LOS and Simplified Acute Physiology Score (SAPS) II. The LSTM model was trained on the training set and then tested on these subgroups in the test set. Metrics such as area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), accuracy, sensitivity, and specificity were used to evaluate model performance. Statistical analyses, including logistic regression and Bonferroni correction, were conducted to compare performance across subgroups.
Results
The LSTM model’s performance varied significantly among different LOS and SAPS II score groups. The overall AUROC was 0.834, but the model performed better for patients with shorter LOS and lower SAPS II scores. The highest AUROC of 0.931 was observed in the [12, 94) hours LOS group, and 0.811 in the [0, 25) SAPS II score group. The model’s accuracy decreased with increasing LOS and SAPS II scores. Logistic regression confirmed that LOS and SAPS II scores significantly affected model accuracy, with longer LOS and higher SAPS II scores associated with poorer model performance.
Discussion and conclusion
When using long-term outcomes like in-hospital death to build early assessment models, there are significant fairness issues in model performance across LOS and SAPS II groups. Developing dynamic prediction models using short-term outcomes may help reduce these fairness issues.
Supplementary Information
The online version contains supplementary material available at 10.1186/s12911-026-03448-7.
Keywords: Deep learning, Fairness, In-hospital mortality, Severity, Clinical prediction models
Background
Accurately identifying the severity of patients early in hospitalization can bring many benefits, such as providing more effective treatment plans and better allocation of medical resources [1]. For a long time, models based on large-scale retrospective data were thought to perform similarly across subgroups [2, 3]. However, recent evidence suggests that existing scoring systems’ performance may vary significantly among different subgroups, sparking discussions about fairness [4–6, 7]. In developing artificial intelligence (AI) models, differences in samples and pattern complexity often lead to fairness issues, such as varying performance among patients of different ages or genders [8, 9]. It is an objective fact that patients with more severe conditions have a more urgent need for high-performance models [10–12]. The length of hospital stay and the initial clinical score at admission may effectively reflect the severity and complexity of a patient’s condition [13]. Although model performance may vary among patients with different severity levels, there has been no empirical analysis of this issue to date [14]. there has been no empirical analysis of this issue to date. Considering the potential integration of early prognosis prediction models into expert diagnostic systems, it is crucial to determine whether AI model performance is biased by the severity of the patient’s condition [15–17].
Many advanced AI models have shown excellent performance in early prognosis prediction [3, 18, 19], with the long short-term memory neural network (LSTM), known for learning temporal relationships, standing out particularly [18, 20]. we focus on a common task in intensive care: predicting in-hospital mortality within 12 h of admission based on clinical information to assess the severity of the patient’s condition, as the 12-hour mark is a critical time point influencing clinical decisions [21, 22].
Using the large public database Medical Information Mart for Intensive Care (MIMIC)-IV [23], we retrospectively divided patients into subgroups based on their length of stay and clinical scores, evaluating the performance of LSTM in these different subgroups. It was hypothesized that these two factors could independently represent the severity of the patient’s condition. Our overall research strategy is shown in Fig. 1.
Fig. 1.
Study overview: from raw data to group performance evaluation
Materials and methods
Dataset
In this study, the MIMIC-IV database was utilized. This openly accessible database contains de-identified records of over 50,000 individual patients admitted to the ICU or emergency department (ED) of Beth Israel Deaconess Medical Center (BIDMC) in Boston, MA, USA, between 2008 and 2019 [23]. MIMIC-IV was chosen due to its large sample size, which is suitable for fairness evaluation. Additionally, many studies over the past decade have developed early clinical prognosis prediction models based on this database [24–26].
For patients with multiple admission records, only the first admission was retained to reduce potential confounding factors affecting model performance. Since our task was to predict in-hospital mortality based on clinical information within 12 h of admission, we excluded patients with a total length of stay of less than 12 h. Patients under the age of 18 were also excluded because their medical records require parental consent according to US law, and their physiological characteristics and medical conditions differ significantly from adult patients, which may negatively impact the stability and generalizability of our predictive models. Additionally, admissions with missing admission or discharge times were excluded. The complete sample selection process is shown in Fig. 2.
Fig. 2.
Sample selection and dataset division process for the MIMIC-IV database
Data preprocessing
We selected 19 variables related to the Simplified Acute Physiology Score (SAPS) II [27], as shown in Supplementary Table S1. SAPS II is a scoring system used to evaluate the prognosis of critically ill patients. Compared to other scoring systems, it includes more physiological indicators, can more comprehensively reflect the patient’s condition, has a stronger ability to predict mortality, and is applicable to different types of critically ill patients. These variables include static information (e.g., demographics and diagnoses) and dynamic information. For outliers, we consulted with doctors to define acceptable ranges for variable values based on clinically acceptable extreme cases. Details are in Table S1. Values beyond this range are considered missing. For static variables, we use chained equations for imputation [28].
Since the sampling of dynamic variables is irregular, we resampled the dynamic variables every hour starting from the patient’s admission. One hour is a common monitoring interval that ensures data stability while retaining as much information as possible. If a variable has no measurement value within half an hour before and after a certain time point, it is recorded as missing. If there are multiple measurements, they are aggregated by the median. For missing observations, linear interpolation is used if there are observations before and after the missing value, and the last observation carried forward (LOCF) method is used if only the previous observation is available. If the first observation is missing, the median of non-missing observations for the entire time range of the patient is used for imputation. If a variable has no observations throughout the entire time range of a patient, the median of the training set is used for imputation. Continuous variables were standardized using Z-transformation, and non-binary categorical data were converted to dummy variables.
Development of model
We used a recurrent neural network consisting of LSTM units to predict in-hospital mortality because it can directly learn the temporal dependencies of longitudinal data without requiring additional manual feature engineering. The LSTM network takes dynamic variables aggregated every hour and the static variables of the patients as inputs. The static variables have a separate input layer for feature extraction, which is then concatenated with the dynamic features in the hidden layer to output the estimated in-hospital mortality probability [29] (Figure S1). Besides the LSTM model, we also tested Gated Recurrent Unit (GRU), Bidirectional LSTM (BiLSTM), and Bidirectional GRU (BiGRU) models.
We chose to use larger hyperparameters to give the model sufficient performance rather than using grid search to optimize the model hyperparameters. The data was randomly split into training, testing, and validation sets in a 7:2:1 ratio. The validation set was used for an early-stopping strategy, stopping model training when the AUROC on the validation set did not improve for 10 consecutive epochs to prevent overfitting. Additionally, L2 regularization was used to prevent overfitting.
As the distribution of in-hospital mortality is imbalanced, with more survivors than non-survivors, we retained all samples from the minority class (non-survivors) and randomly sampled an equal subset from the majority class (survivors) in each gradient descent iteration. In each epoch (one cycle through the training dataset), some data was left out, but over many epochs, the LSTM will see the full dataset.
Evaluation
We divided the patients into different groups based on their length of stay and initial SAPS II score at admission. Then, we compared the model’s predictive performance on patients in these different groups. Specifically, we divided all patients into four equal groups from two perspectives: LOS and initial SAPS II scores. This division aimed to eliminate bias due to sample size variations and to ensure representative samples for accurate analysis.
To assess the model’s ability to differentiate survivors from non-survivors, we used three metrics: accuracy, balanced accuracy, and the area under the receiver operating characteristic curve (AUROC). Accuracy represents the proportion of correctly classified cases. Balanced accuracy measures the average accuracy of each class, accounting for imbalances. AUROC indicates the model’s ability to distinguish between survivors and non-survivors. Additionally, we used area under the precision-recall curve (AUPRC), positive predictive value (PPV), negative predictive value (NPV), sensitivity, and specificity to evaluate the model’s predictive performance for different categories.
Statistical analysis
To test the balance of patient characteristics among the training, test, and validation sets, we performed analysis of variance (ANOVA) or Kruskal-Wallis tests for numerical features, depending on whether they met normal distribution assumptions, and Chi-square tests for categorical features. All comparisons were considered statistically significant if P < 0.05 (two-sided).
To compare the accuracy of the LSTM model among patients in different length of stay groups, we estimated the marginal means of accuracy for each group using the emmeans package in R software and performed pairwise comparisons using the Bonferroni method. To control for the effects of covariates, we used logistic regression to control for factors such as age, gender, in-hospital mortality, and admission type, with correct prediction as the dependent variable. We used a stepwise regression method based on the Akaike information criterion to select variables.
The LSTM network was built on PyTorch-1.8.0, and all statistical analyses were conducted using R-4.2.1.
Results
Patient characteristics
A total of 50,313 patients/admissions were included in the final dataset. The characteristics of the patients are listed in Table 1. The mean age of the patients was 63.50 ± 17.17 years. The in-hospital mortality rate was 9.5%. The majority of patients were not admitted for surgery (60.3%). Through random grouping, 35,408, 9,984, and 4,921 were assigned to the train, test, and validation datasets, respectively. There were no statistical differences in demographic characteristics between the datasets. Table S6 shows the distribution of SAPS II scores for different LOS groups. Table S7 shows the distribution of LOS for different SAPS II groups. Patients with longer LOS tend to have higher initial SAPS II scores at admission, and patients with higher SAPS II scores tend to have longer LOS.
Table 1.
Characteristics of patients
| Overall | Train | Test | Valid | P value | |
|---|---|---|---|---|---|
| N | 50,313 | 35,408 | 9984 | 4921 | |
| Age, years (mean (SD)) | 63.50 (17.17) | 63.59 (17.11) | 63.29 (17.33) | 63.35 (17.25) | 0.243 |
| Gender (%) | 0.329 | ||||
| Female | 22,238 (44.2) | 15,671 (44.3) | 4358 (43.6) | 2209 (44.9) | |
| Male | 28,075 (55.8) | 19,737 (55.7) | 5626 (56.4) | 2712 (55.1) | |
| Race (%) | |||||
| Asian | 1482 (2.9) | 1034 (2.9) | 308 (3.1) | 140 (2.8) | 0.981 |
| Black | 4592 (9.1) | 3237 (9.1) | 920 (9.2) | 435 (8.8) | |
| Hispanic/Latino | 1228 (2.4) | 858 (2.4) | 250 (2.5) | 120 (2.4) | |
| Other | 9136 (18.2) | 6427 (18.2) | 1802 (18.0) | 907 (18.4) | |
| White | 33,875 (67.3) | 23,852 (67.4) | 6704 (67.1) | 3319 (67.4) | |
| LOS (median [IQR]) | 158.02 [93.72, 266.53] | 158.56 [94.05, 266.57] | 156.12 [92.98, 265.90] | 158.00 [93.32, 266.92] | 0.62 |
| Admission type (%) | 0.628 | ||||
| Medical | 30,360 (60.3) | 21,353 (60.3) | 6026 (60.4) | 2981 (60.6) | |
| Scheduled surgical | 1579 (3.1) | 1090 (3.1) | 337 (3.4) | 152 (3.1) | |
| Unscheduled surgical | 18,374 (36.5) | 12,965 (36.6) | 3621 (36.3) | 1788 (36.3) | |
| Aids (%) | 265 (0.5) | 173 (0.5) | 68 (0.7) | 24 (0.5) | 0.059 |
| Hem (%) | 7348 (14.6) | 5153 (14.6) | 1459 (14.6) | 736 (15.0) | 0.754 |
| Mets (%) | 2414 (4.8) | 1713 (4.8) | 455 (4.6) | 246 (5.0) | 0.405 |
| SAPS II (median [IQR]) | 33.00 [25.00, 42.00] | 33.00 [25.00, 42.00] | 33.00 [25.00, 42.00] | 33.00 [25.00, 42.00] | 0.343 |
| In-hospital mortality (%) | 4777 (9.5) | 3389 (9.6) | 918 (9.2) | 470 (9.6) | 0.521 |
| LOS groups (%) | 0.892 | ||||
| [12, 94) | 12,579 (25.0) | 8809 (24.9) | 2534 (25.4) | 1236 (25.1) | |
| [94, 158) | 12,579 (25.0) | 8842 (25.0) | 2512 (25.2) | 1225 (24.9) | |
| [158, 266) | 12,577 (25.0) | 8904 (25.1) | 2446 (24.5) | 1227 (24.9) | |
| [266, Discharge) | 12,578 (25.0) | 8853 (25.0) | 2492 (25.0) | 1233 (25.1) | |
| SAPS II groups (%) | 0.838 | ||||
| [0, 25) | 13,453 (26.7) | 9468 (26.7) | 2703 (27.1) | 1282 (26.1) | |
| [25, 33) | 12,586 (25.0) | 8847 (25.0) | 2511 (25.2) | 1228 (25.0) | |
| [33, 42) | 12,450 (24.7) | 8788 (24.8) | 2432 (24.4) | 1230 (25.0) | |
| [42, Max) | 11,824 (23.5) | 8305 (23.5) | 2338 (23.4) | 1181 (24.0) |
Performance of LSTM
Figure 3 illustrates the performance of the LSTM model using data from the first 12 h after admission and the SAPS II score within 12 h to predict in-hospital mortality for all patients in the testing sets. The AUROC is 0.834 (0.829, 0.838) and 0.797 (0.794, 0.800) on the testing set, while the AUPRC is 0.379 (0.373, 0.386) and 0.343 (0.335, 0.351). The LSTM model performs significantly better than the traditional SAPS II scores. For detailed information on the performance of the GRU, BiLSTM, and BiGRU models, please see Table S8. Table S9 shows the performance of SAPS II scores in predicting in-hospital mortality across different LOS groups.
Fig. 3.
Performance of LSTM for predicting in-hospital mortality in the training and testing sets for all patients. Panel A shows the performance in terms of AUROC, and Panel B shows the performance in terms of AUPRC
Figure 4 shows the AUROC of the LSTM model in different LOS and SAPS II groups on the testing set. The AUROC decreases monotonically with longer hospital stays, with the [12, 94) hours group having the highest AUROC of 0.931 (0.917, 0.946), significantly higher than the overall performance. The AUROC decreases initially and then stabilizes with higher SAPS II scores, with the [0, 25) score group having the highest AUROC of 0.811 (0.734, 0.889), also significantly higher than the overall performance. Detailed data are provided in Table S2 in the supplementary materials.
Fig. 4.
Bar graph showing the AUROC of the LSTM model in patients with different length of stay categories and SAPS II score groups (testing set). (A) shows different LOS groups, and (B) shows different SAPS II score groups
Figure 5 shows the accuracy of the LSTM model in different LOS and SAPS II groups on the testing set. The accuracy decreases monotonically with longer hospital stays, with the [12, 94) hours group having an accuracy of 0.873 (0.860, 0.887), while the [266, Discharge) hours group has an accuracy of only 0.563 (0.536, 0.590), the former being significantly higher than the latter. Additionally, accuracy decreases monotonically with higher SAPS II scores. For the SAPS II groups, the [0, 25) score group has the highest accuracy at 0.940 (0.929, 0.950), while the [42, Max) score group has the lowest accuracy at 0.481 (0.455, 0.506), with the former being significantly higher than the latter. Detailed data are provided in Table S2 in the supplementary materials. Balanced accuracy also shows the same trend. Detailed results can be found in Figure S2 and Table S2. Additional metric information, such as the sensitivity, specificity, PPV, and NPV of the LSTM model in different LOS and SAPS II groups, is available in Table S5.
Fig. 5.
Bar graph showing the accuracy of the LSTM model in patients with different length of stay categories and SAPS II score groups (testing set). (A) Different LOS groups, and (B) Different SAPS II score groups
Logistic regression results show that both LOS and SAPS II groups significantly affect the model’s accuracy. Table S3 and Table S4 present the regression models for the LOS groups and SAPS II groups, respectively, displaying the beta values for each group relative to the reference group, as well as for other confounders. Through stepwise regression, the gender factor did not enter the model. Besides LOS and SAPS II groups, the beta values for admission type and age were also statistically significant, indicating that these factors also impact the fairness of the model’s predictions. Additionally, Table 2 shows the pairwise comparison of the linear responses of the models in different length of stay groups and SAPS II groups after Bonferroni correction. The results show that in the LOS groups, the LSTM model’s accuracy in the [12, 94) hours group is significantly higher than in the other three LOS groups. The accuracy in the [94, 158) hours group is also significantly higher than in the [158, 266) hours group and the [266, Discharge) group. In the SAPS II groups, a similar trend is observed. The accuracy in the [0, 25) score group is significantly higher than in the other three SAPS II groups.
Table 2.
Linear response pairwise comparisons of Bonferroni-adjusted groups on testing set
| Group variable | Contrast | Estimate | Std error | Z | P value |
|---|---|---|---|---|---|
| LOS groups | [12, 94) - [94, 158) | 0.797 | 0.089 | 8.995 | < 0.001 |
| [12, 94) - [158, 266) | 1.262 | 0.091 | 13.927 | < 0.001 | |
| [12, 94) - [266, Discharge) | 1.988 | 0.092 | 21.521 | < 0.001 | |
| [94, 158) - [158, 266) | 0.465 | 0.084 | 5.549 | < 0.001 | |
| [94, 158) - [266, Discharge) | 1.191 | 0.085 | 13.943 | < 0.001 | |
| [158, 266) - [266, Discharge) | 0.726 | 0.087 | 8.308 | < 0.001 | |
| SAPS II groups | [0, 25) - [25, 33) | 1.102 | 0.119 | 9.240 | < 0.001 |
| [0, 25) - [33, 42) | 1.962 | 0.119 | 16.470 | < 0.001 | |
| [0, 25) - [42, Max) | 2.981 | 0.124 | 23.975 | < 0.001 | |
| [25, 33) - [33, 42) | 0.860 | 0.085 | 10.103 | < 0.001 | |
| [25, 33) - [42, Max) | 1.879 | 0.090 | 20.801 | < 0.001 | |
| [33, 42) - [42, Max) | 1.019 | 0.081 | 12.516 | < 0.001 |
Discussion
In this study, we used the LSTM model, currently the most suitable for processing time series data, to build an early prediction model for in-hospital mortality using the MIMIC-IV database. We compared the model’s performance across different length of stay groups and SAPS II score groups. The results showed significant differences in performance among the groups.
Currently, much research is based on MIMIC-IV data to build early prognostic prediction models for in-hospital patients, achieving high performance in internal or external testing [30–33]. However, our results suggest that this high performance may only apply to some patients. Specifically, although the LSTM’s overall AUROC on the test set reached 0.834 (0.829, 0.838), the AUROCs for the [158, 266) and [266, Discharge) hours groups were significantly lower. The AUROC for the [12, 94) hours group was as high as 0.931 (0.917, 0.946). Similarly, the AUROCs for patients with SAPS II scores of [25, 33), [33, 42), and [42, Max) were significantly lower than the overall level. Besides AUROC, accuracy and balanced accuracy also showed the same trend, with the LSTM model performing worse in patients with higher LOS or SAPS II scores. Logistic regression results, adjusted for baseline demographics, confirmed that the performance of the LSTM model decreased monotonically with increasing LOS or SAPS II scores. These results are consistent with Harutyunyan et al.‘s findings, which showed that the prediction performance for in-hospital mortality decreases with longer LOS in the MIMIC-IV database [13]. Additionally, Sarkar, et al. [4] compared the fairness of common clinical scores, such as APACHE and SOFA, in predicting in-hospital death across different racial groups. Our study complements their findings by showing that SAPS II scores also have fairness issues in predicting in-hospital death across different LOS groups.
We found that patients in different LOS groups had varying initial SAPS II scores at admission. Patients with longer LOS typically had higher SAPS II scores, indicating more severe conditions. As the initial SAPS II scores increased, both sensitivity and NPV significantly decreased, while PPV and sensitivity increased. This means that while the model’s ability to predict positive cases improved, it lost more accuracy in identifying negative cases. In the different LOS groups, the situation differed from the SAPS II groups. In groups with higher LOS, the LSTM model’s ability to identify both positive and negative cases significantly decreased, as shown by the concurrent drop in sensitivity, specificity, PPV, and NPV.
We attempted to analyze the reasons for this unfairness. From the LOS groups, the model’s performance is significantly lower than average for patients with longer LOS, particularly for positive cases. A possible reason for this is that longer hospital stays involve more complex factors affecting patient outcomes. From the SAPS II groups perspective, the model’s unfairness mainly stems from the imbalance in outcome categories for patients with different severity levels. For less severe patients, such as those with SAPS II scores in the [0, 25) range, the LSTM model tends to underestimate positive cases but is good at identifying negative cases. For more severe patients, such as those with SAPS II scores in the [42, Max) range, the LSTM model improves in identifying positive cases but has a greater decline in identifying negative cases, leading to overall poorer performance. This may be because more severe patients have longer treatment times and more complex medical interventions, making predictions more difficult. Overall, the LSTM model shows better performance fairness across SAPS II groups compared to LOS groups. Combining the results of LOS and SAPS II groups, it appears that the primary reason for the LSTM model’s performance decline is the influence of uncontrollable potential confounding factors with longer LOS.
Therefore, when using long-term outcomes like in-hospital death to develop early risk assessment models, we must consider the potential for overestimating or underestimating risk. Overestimating a patient’s condition can lead to unnecessary medical resource use [34] while underestimating it can worsen the patient’s condition. Although we cannot predict LOS for new patients, a feasible solution is to construct real-time predictive models. These models would use dynamically adjustable short-term tasks and continuously update patients’ health assessments to minimize errors [15, 18, 35–37]. This approach can minimize the predicted time range of outcomes when building predictive models. It ensures clinical feasibility while reducing instability and potential unfairness in model performance.
Limitations
This study has some limitations. Firstly, it is a retrospective study, so the data may have retrospective bias. Although we used imputation methods to handle missing static and dynamic values, this might still introduce bias into the relationship between early patient features and in-hospital death, potentially affecting the fairness analysis of the LSTM model’s performance. Secondly, we selected the 19 variables involved in the SAPS II score for fairness analysis. These variables were chosen for their predictive ability for in-hospital death, but they do not represent the optimal feature set for predicting in-hospital death. Thirdly, we excluded admissions with LOS less than 12 h. These admissions did not have enough information to support model prediction, but their removal might still impact the results. Additionally, the present study did not include calibration analysis for evaluating the clinical applicability of the model. These aspects will be incorporated in future studies to provide a more comprehensive assessment of model performance.
Conclusion
This study identified significant fairness issues in the LSTM model’s performance for predicting in-hospital mortality using early admission data. The model showed lower accuracy and AUROC for patients with longer lengths of stay and higher SAPS II scores, indicating it is less reliable for more severe patients. To address these issues, developing dynamic prediction models using short-term outcomes may help improve fairness and accuracy.
Supplementary Information
Below is the link to the electronic supplementary material.
Author contributions
L.L. and S.H. drafted the work. J.L. and Y.C. designed the work. L.L. and J.L. analyzed the data. S.L. interpreted the data. All authors read and approved the final manuscript.
Funding
This project was supported by The National Natural Science Foundation of China (No.72204169, No.82102148). The funding bodies played no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.
Data availability
The MIMIC-IV dataset analyzed in this study is available from the MIT Laboratory for Computational Physiology and their collaborating research groups. This dataset is available upon request at https://physionet.org/content/mimiciv/2.0/. All the code files involved in this article can be found at https://github.com/ljwa2323/LSTM_Fairness_ICU_Prediction.
Declarations
Ethical approval and consent to participate
This study utilized the MIMIC-IV database, a publicly available database encompassing a substantial amount of information pertaining to patients in intensive care units. The database has received approval from the Institutional Review Board (IRB) and all data contained therein have undergone de-identification processes to safeguard the privacy and confidentiality of the patients involved. We have adhered strictly to all applicable ethical guidelines and legal regulations to ensure the protection of the rights and well-being of the subjects during the research process. We have obtained the necessary permissions to use the MIMIC-IV database and guarantee that all data are utilized in a secure and lawful manner. We pledge that the objective of our research is to foster advancements in medicine and science, not for any commercial or unlawful purposes. We also commit to using and storing this data responsibly, ensuring it will not be employed in any manner that could potentially harm the subjects.
Ethical guidelines
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare no competing interests.
Footnotes
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Lan Lan, Shixin Huang, Yang Chen and Jiawei Luo contributed equally to this work.
References
- 1.Tomašev N, Harris N, Baur S, et al. Use of deep learning to develop continuous-risk models for adverse event prediction from electronic health records. Nat Protoc. 2021;16(6):2765–87. [DOI] [PubMed] [Google Scholar]
- 2.Data MC, Pirracchio R. Mortality prediction in the icu based on mimic-ii results from the super icu learner algorithm (sicula) project. Secondary Anal Electron Health Records. 2016:295–313. [PubMed]
- 3.Purushotham S, Meng C, Che Z, et al. Benchmarking deep learning models on large healthcare datasets. J Biomed Inform. 2018;83:112–34. [DOI] [PubMed] [Google Scholar]
- 4.Sarkar R, Martin C, Mattie H, et al. Performance of intensive care unit severity scoring systems across different ethnicities in the USA: a retrospective observational study. Lancet Digit Health. 2021;3(4):e241–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Thompson HM, Brihat S, Sameer B, et al. Bias and fairness assessment of a natural language processing opioid misuse classifier: detection and mitigation of electronic health record data disadvantages across racial subgroups. J Am Med Inform Assoc. 2021;(11). [DOI] [PMC free article] [PubMed]
- 6.Decamp M, Lindvall C. Latent bias and the implementation of artificial intelligence in medicine. J Am Med Inform Assoc. 2020;(1). [DOI] [PMC free article] [PubMed]
- 7.De Hond AA, Van Buchem MM, Hernandez-Boussard T. Picture a data scientist: a call to action for increasing diversity, equity, and inclusion in the age of AI. J Am Med Inform Assoc. 2022;29(12):2178–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Feng Q, Du M, Zou N, et al. Fair machine learning in healthcare: a review. arXiv preprint arXiv:220614397. 2022.
- 9.Chen RJ, Chen TY, Lipkova J, et al. Algorithm fairness in ai for medicine and healthcare. arXiv preprint arXiv:211000603. 2021.
- 10.Dai Z, Zeng D, Cui D, et al. Prediction of COVID-19 patients at high risk of progression to severe disease. Front Public Health. 2020;8:574915. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Park DJ, Park MW, Lee H, et al. Development of machine learning model for diagnostic disease prediction based on laboratory tests. Sci Rep. 2021;11(1):7567. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Mvs A, Jbr A, Rdr B, et al. Clinical prediction models: diagnosis versus prognosis. J Clin Epidemiol. 2021;132:142–5. [DOI] [PubMed] [Google Scholar]
- 13.Harutyunyan H, Khachatrian H, Kale DC, et al. Multitask learning and benchmarking with clinical time series data. Sci data. 2019;6(1):1–18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Lequertier V, Wang T, Fondrevelle J, et al. Hospital length of stay prediction methods: a systematic review. Med Care. 2021;59(10):929–38. [DOI] [PubMed] [Google Scholar]
- 15.Hyland SL, Faltys M, Hüser M, et al. Early prediction of circulatory failure in the intensive care unit using machine learning. Nat Med. 2020;26(3):364–73. [DOI] [PubMed] [Google Scholar]
- 16.Rajkomar A, Oren E, Chen K, et al. Scalable and accurate deep learning with electronic health records. NPJ Digit Med. 2018;1(1):18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Xie W, Li Y, Meng X, et al. Machine learning prediction models and nomogram to predict the risk of in-hospital death for severe DKA: a clinical study based on MIMIC-IV, eICU databases, and a college hospital ICU. Int J Med Informatics. 2023:105049. [DOI] [PubMed]
- 18.Thorsen-Meyer H-C, Nielsen AB, Nielsen AP, et al. Dynamic and explainable machine learning prediction of mortality in patients in the intensive care unit: a retrospective study of high-frequency data in electronic patient records. Lancet Digit Health. 2020;2(4):e179–91. [DOI] [PubMed] [Google Scholar]
- 19.Xie F, Zhou J, Lee JW, et al. Benchmarking emergency department prediction models with machine learning and public electronic health records. Sci Data. 2022;9(1):658. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Lipton ZC, Kale D, Wetzel R. Directly modeling missing data in sequences with rnns: Improved classification of clinical time series; proceedings of the Machine learning for healthcare conference. PMLR; 2016.
- 21.Thorsen-Meyer H-C, Placido D, Kaas-Hansen BS, et al. Discrete-time survival analysis in the critically ill: a deep learning approach using heterogeneous data. NPJ Digit Med. 2022;5(1):142. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Elhazmi A, Al-Omari A, Sallam H, et al. Machine learning decision tree algorithm role for predicting mortality in critically ill adult COVID-19 patients admitted to the ICU. J Infect Public Health. 2022;15(7):826–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Johnson AE, Pollard TJ, Shen L, et al. MIMIC-III, a freely accessible critical care database. Sci data. 2016;3(1):1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Weissman GE, Hubbard RA, Ungar LH, et al. Inclusion of unstructured clinical text improves early prediction of death or prolonged ICU stay. Crit Care Med. 2018;46(7):1125. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Huang K, Gray TF, Romero-Brufau S, et al. Using nursing notes to improve clinical outcome prediction in intensive care patients: a retrospective cohort study. J Am Med Inform Assoc. 2021;28(8):1660–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Che Z, Purushotham S, Cho K, et al. Recurrent neural networks for multivariate time series with missing values. Sci Rep. 2018;8(1):1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Gall L, Jean-Roger. A New Simplified Acute Physiology Score (SAPS II) Based on a European/North American Multicenter Study. JAMA. 1993;270(24):2957. [DOI] [PubMed] [Google Scholar]
- 28.Van Buuren S, Groothuis-Oudshoorn K. mice: Multivariate imputation by chained equations in R. J Stat Softw. 2011;45:1–67. [Google Scholar]
- 29.Leontjeva A, Kuzovkin I. Combining static and dynamic features for multivariate sequence classification; proceedings of the 2016 IEEE International Conference on Data Science and, Analytics (DSAA). IEEE; 2016.
- 30.Liu M, Guo C, Guo S. An explainable knowledge distillation method with XGBoost for Icu mortality prediction. Comput Biol Med. 2023;152:106466. [DOI] [PubMed] [Google Scholar]
- 31.Liu C, Liu X, Mao Z, et al. Interpretable Machine Learning Model for Early Prediction of Mortality in Icu Patients with Rhabdomyolysis. Med Sci Sports Exerc. 2021;53(9):1826–34. [DOI] [PubMed] [Google Scholar]
- 32.Roy S, Mincu D, Loreaux E, et al. Multitask prediction of organ dysfunction in the intensive care unit using sequential subnetwork routing. J Am Med Inform Assoc. 2021;28(9):1936–46. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Zhang D, Yin C, Zeng J, et al. Combining structured and unstructured data for predictive models: a deep learning approach. BMC Med Inf Decis Mak. 2020;20(1):1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Emanuel EJ, Persad G, Upshur R, et al. Fair allocation of scarce medical resources in the time of Covid-19. Mass Medical Soc. 2020:2049–55. [DOI] [PubMed]
- 35.Catling FJ, Wolff AH. Temporal convolutional networks allow early prediction of events in critical care. J Am Med Inform Assoc. 2020;27(3):355–65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Luo J, Lan L, Huang S, et al. Real-time prediction of organ failures in patients with acute pancreatitis using longitudinal irregular data. J Biomed Inform. 2023;139:104310. [DOI] [PubMed] [Google Scholar]
- 37.Xiongcai C. Oscar, et al. Real-time prediction of mortality, readmission, and length of stay using electronic health record data. JAMIA. 2016. [DOI] [PMC free article] [PubMed]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The MIMIC-IV dataset analyzed in this study is available from the MIT Laboratory for Computational Physiology and their collaborating research groups. This dataset is available upon request at https://physionet.org/content/mimiciv/2.0/. All the code files involved in this article can be found at https://github.com/ljwa2323/LSTM_Fairness_ICU_Prediction.