Risk factor analysis and multiple predictive machine learning models for mortality in COVID-19: a multicenter and multi-ethnic cohort study

Abstract

Background

The COVID-19 pandemic presents a significant challenge to the global healthcare system. Implementing timely, accurate, and cost-effective screening approaches is crucial in preventing infections and saving lives by guiding disease management.

Objectives

The study aimed to use machine learning algorithms to analyze clinical features from routine clinical data to identify risk factors and predict the mortality of COVID-19.

Methods

The data used in this research was originally collected for the study titled "Neurologic Syndromes Predict Higher In-Hospital Mortality in COVID-19". A total of 4711 patients with confirmed COVID-19 were enrolled consecutively from four hospitals. Three machine learning models, including RF, PLS-DA, and SVM, were used to find risk factors and predict COVID-19 mortality.

Results

The predictive models were developed based on three machine learning algorithms. The RF model was trained with 20 variables and had a ROC value of 0.859 (95%CI: 0.804-0.920). The PLS-DA model was trained with 20 variables and had a ROC value of 0.775 (95%CI: 0.694-0.833). The SVM model was trained with 10 variables and had a ROC value of 0.828 (95%CI: 0.785-0.865). The 9 variables that were present in all three models were age, PCT, ferritin, CRP, troponin, BUN, MAP, AST, and ALT.

Conclusion

This study developed and validated three machine learning prediction models for COVID-19 mortality based on accessible clinical features. The RF model showed the best performance among the three models. The 9 variables identified in the models may warrant further investigation as potential prognostic indicators of severe COVID-19.

1. INTRODUCTION

The COVID-19 pandemic, caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), presents a significant challenge to global healthcare systems and has led the World Health Organization (WHO) to declare it a global emergency[1]. The pandemic has caused widespread disruption, including economic turmoil, public health crises, and forced confinement that has impacted daily life, education, and work[2]. Despite advancements in medical science, the pandemic has not yet been effectively controlled[3]. In this context, the implementation of timely, accurate, and cost-effective screening diagnostic approaches remains critical to prevent infections and save lives by guiding disease management.

Artificial intelligence, including deep learning and machine learning algorithms, has emerged as a valuable tool for supporting decision-making in healthcare, drug discovery, disease diagnosis, and monitoring[4]. Some recent studies have applied these methods to guide COVID-19 diagnosis, identify biomarkers of disease progression, and predict disease severity and future outbreaks[5]. However, despite the availability of several machine learning models for COVID-19 prediction, there are limitations to their practical use, including high heterogeneity among patients' clinical profiles and small sample sizes, which can reduce external validity and data generalization.

In this study, we presented a risk factor analysis and three machine learning models for COVID-19 screening based on a third-party database of numerous patients from four different hospitals. This approach may reduce bias and increase universality to some extent. The models were only based on relatively accessible clinical data, including demographics, laboratory results, and clinical characteristics, to make them applicable to low- to mid-level hospitals. Welcome to tailor the model for the current pandemic situation. Although the multiple mutations of SARS-CoV-2, the rise in a number of infected individuals, and the development of immunity in the global population are key aspects of the current scenario, we also wanted to see the forecast variance between the pandemic situation and provide relevant and effective guidance to address the ongoing pandemic.

2. METHODS

2.1. Data source

The data used in this research was originally collected for the study titled "Neurologic Syndromes Predict Higher In-Hospital Mortality in COVID-19" [6]. The database has since been made publicly available through Neurology (doi.org/10.5061/dryad.7d7wm37sz.), allowing for secondary analysis to be conducted according to different scientific hypotheses. The original research was exempt from the requirement for obtaining informed consent and was approved by the Ethics Committee for Clinical Research of the Albert Einstein College of Medicine, Montefiore Medical Center[6].

2.2. Study Cohort

In this study, a total of 4711 patients with confirmed COVID-19 were enrolled consecutively from four hospitals between March 1^st and April 16^th, 2020. The patients were divided into two groups based on their clinical outcome: the surviving group (n=3563) and the deceased group (n=1148). The diagnosis of COVID-19 was made by the WHO interim guidance and was confirmed through positive results from a real-time reverse transcriptase PCR assay test for SARS-CoV-2 RNA[7].

2.3. Data Collection

The data collected from each patient was comprehensive and included both demographic information and laboratory test results. The use of a health care surveillance software package (Clinical Looking Glass; Streamline Health, Atlanta, GA) and review of medical records ensured the accuracy and completeness of the collected data[8]. Blood samples were collected in a fasting state and tested for a range of markers including platelet (Plts), glucose (Glu), sodium, alanine transaminase (ALT), aspartate transaminase (AST), blood urea nitrogen (BUN), ferritin, white blood cell (WBC), lympho, procalcitonin (PCT), Ddimer, C-reactive protein (CRP), troponin, international normalized ratio (INR), and creatinine (Cr). These markers were measured using standardized laboratory methods, providing reliable and consistent results for each patient.

2.4. Machine Learning Algorithms

The data pre-processing steps included normalization of variables, log10 transformation, and auto-scaling to prepare the data for the application of machine learning algorithms. The three machine learning models used in the study were random forest (RF), partial least squares discriminant analysis (PLS-DA), and support vector machine (SVM) [9]. These algorithms were used to identify the risk factors and predict the mortality of COVID-19 in the patient population.

In the study, the receiver operating characteristic (ROC) curves were generated to evaluate the performance of the machine learning models. This was done using Monte-Carlo cross-validation (MCCV) by balanced sub-sampling. In each MCCV iteration, 2/3 of the patients were used to train the machine learning models. The top 3, 5, 10, 20, and all important features were used to build the models, which were then validated on the remaining 1/3 of the patients. This process was repeated multiple times to calculate the performance and confidence interval of each model. The importance of variables was also assessed in each model using the Gini criterion. All analyses were performed using the R software version 4.2.0 (R Foundation for Statistical Computing, Vienna, Austria).

2.5. Statistical Analysis

Data analysis for continuous variables was performed using mean±standard deviation, and differences between groups were determined using a Student's t-test. The data for categorical variables was expressed as frequency and percentage, and the difference between groups was analyzed using either the chi-square test or Fisher's exact test. The Venn diagram and UpSet plot were created using the Venn package and UpSetR package in R. The statistical analysis was performed using SPSS version 23.0 (SPSS Inc., Chicago, IL, USA).

3. RESULTS

3.1. The ethnic composition of the overall population

The study enrolled a total of 4711 patients, including 3563 (75.63%) surviving patients and 1148 (24.37%) deceased patients. The ethnic composition of the patients in the study was presented in Figure 1 . The results indicated that there may be a higher mortality rate among White patients (11.7%) compared to the general population (9.3%) with a P-value of 0.02. However, no significant differences were found in mortality rates between Black, Asian, and Latino patients.

Figure 1.

The ethnic composition in the (A) overall population, (B) death population, and (C) surviving population, respectively.

3.2. Patient characteristics

This study collected variables including clinical demographic information, laboratory test results, and comorbidities for each patient (Figure 2, Figure 3 ). When comparing the surviving group to the deceased group, the latter had a higher average age and a longer length of stay (LOS) in the hospital. Additionally, the deceased group had lower oxygen saturation (O₂Sats) levels and median arterial blood pressure (MAP), but there was no significant difference in the temperature between the two groups (P=0.194). The laboratory test results showed that deceased patients had higher levels of Glu, sodium, ALT, AST, BUN, ferritin, WBC, PCT, Ddimer, CRP, troponin, INR, and Cr. Meanwhile, the platelet count was lower in the deceased group.

Figure 2.

The baseline clinical characteristics between deceased and surviving groups in the overall population. MAP, median arterial blood pressure; O₂Sats, oxygen saturation; Plts, platelets; Glu, glucose; ALT, alanine aminotransferase; AST, aspartate aminotransferase; BUN, blood urea nitrogen; LOS, length of stay; WBC, white blood cells; PCT, procalcitonin; CRP, C-reactive protein; INR, international normalized ratio; Cr, creatinine.

Figure 3.

The comorbidities between deceased and surviving groups in the overall population. MI, myocardial infarction; CHF, congestive heart failure; CVD, cerebrovascular disease; COPD, chronic obstructive pulmonary disease; DM. C, diabetes mellitus complicated; DM. S, diabetes mellitus simple; CNS, central nervous system.

3.3. Variables selection

In order to identify the prominent features, three different machine learning models were conducted to select variables. Figure 4 showed the relationship between the ROC and the number of variables. The number of variables used significantly impacted the model's performance, with the best results achieved when using 20 variables for the RF model (Figure 4A), 20 variables for the PLS-DA model (Figure 4B), and 10 variables for the SVM model (Figure 4C). The ROC curve remained stable at a similar level as the number of variables continued to increase to the maximum.

Figure 4.

The relationship between the receiver operating characteristics curve and the number of variables in (A) random forest (RF), (B) partial least squares discriminant analysis (PLS), and (C) support vector machines (SVM) models for mortality of COVID-19, respectively.

3.4. Variables of importance

Based on the results of the Gini impurity analysis, the three prediction models identified the prominent features that could best predict the outcome of patients. The RF model identified five important predictors: MAP, age, PCT, BUN, and troponin, with relative importance scores of -33.5, 23.4, 15.7, 14.8, and 14.1, respectively (Figure 5 A). On the other hand, the PLS-DA model identified PCT with a relative importance score of 3.25, followed by ferritin (2.08), CRP (1.91), Ddimer (1.90), and troponin (1.89) (Figure 5B). Finally, the SVM model identified MAP (-0.44), age (0.35), AST (0.24), ALT (0.12), and CRP (0.11) as the top five predictors (Figure 5C).

Figure 5.

The importance of the variables for the mortality of COVID-19 prediction model in (A) random forest (RF), (B) partial least squares discriminant analysis (PLS), and (C) support vector machines (SVM) models, respectively.

3.5. Risk factor analysis

As the Venn diagram (Figure 6 A) showed, there were 36 variables in the whole database. Among them, 3 variables were in the PLS-DA or RF model alone. 1 variable was in the SVM model only. 8 variables were both in the PLS-DA and RF models. After comparing these 3 models, 9 variables existed in the RF, PLS-DA, and SVM models at the same time, which were age, PCT, ferritin, CRP, troponin, BUN, MAP, AST, and ALT (Figure 6B).

Figure 6.

The potential risk factors for mortality of COVID-19 in three machine learning models. The Venn diagram (A) and UpSet plot (B) showed that there might be 9 common potential risk factors for mortality of COVID-19.

3.6. Model performance and validation

For better performance of the models, we choose the number of variables in the highest ROC for each model. As showed in Figure 7 A, for the RF model, 20 variables were selected. And the trained ROC value was 0.859 (95%CI: 0.804-0.920). The ROC value of the trained PLS-DA model was 0.775 (95%CI: 0.694-0.833), of which 20 variables were chosen (Figure 7B). For the SVM model, the trained ROC value was 0.828 (95%CI: 0.785-0.865) in 10 variables chosen (Figure 7C).

Figure 7.

Receiver operating characteristics curve of (A) random forest (RF), (B) partial least squares discriminant analysis (PLS), and (C) support vector machines (SVM) models for mortality of COVID-19 in training sets, respectively.

To further evaluate the performance of each model, the ROC curves of the 3 machine learning models were assessed in the internal validation cohort (Figure 8 ). The highest ROC was RF (0.847, 95% CI: 0.769-0.901) (Figure 8A), intermediary for SVM (0.818, 95% CI: 0.728-0.878) (Figure 8C), and next for PLS-DA (0.756, 95% CI: 0.665-0.827) (Figure 8B).

Figure 8.

Receiver operating characteristics curve of (A) random forest (RF), (B) partial least squares discriminant analysis (PLS), and (C) support vector machines (SVM) models for mortality of COVID-19 in internal validation sets, respectively.

4. DISCUSSION

The COVID-19 pandemic has placed immense strain on healthcare systems worldwide, highlighting the urgent need for reliable and accurate methods of predicting patient survival[10]. In response to this need, our study aimed to develop and evaluate machine learning models, including RF, PLS-DA, and SVM, to predict the fatality of COVID-19 patients. The models were trained using clinical demographics, characteristics, and plasma samples of patients from four reference hospitals. Due to the highly variable course of COVID-19, particularly in light of the emergence of new variants and biological variability, more robust machine learning models are required, which is critical in providing accurate predictions of patient outcomes. These models may help healthcare providers make informed decisions and allocate resources effectively, ultimately improving patient outcomes and reducing the overall burden on healthcare systems.

After identifying significant differences in the clinical characteristics and plasma proteins between COVID-19 survivors and non-survivors, we trained three machine learning models with the goal to identify a predictive signature of COVID-19 mortality that could be used to classify patients based on their chance of survival from the day of hospitalization. To improve the performance of the models, we compared the number of variables used in each model with the highest ROC score. The results showed that using 20 variables in RF, 20 variables in PLS-DA, and 10 variables in SVM resulted in better performance compared to using all variables. Among the three models, RF showed the best prediction ability for COVID-19 mortality in the internal validation set. This highlighted the potential utility of RF in clinical practice to improve patient outcomes and reduce the overall burden on healthcare systems[11].

Another objective of our study was to identify the key risk factors associated with COVID-19 mortality. Using the Gini criterion, Figure 5 showed the positive or negative contributions of each feature to the model output. Based on different machine learning algorithms, we found that 9 variables were the most critical risk factors for COVID-19 mortality. These variables were: age, PCT, ferritin, CRP, troponin, BUN, MAP, AST, and ALT. The results showed that these variables made contributions to the target variable of COVID-19 mortality and were ranked in descending order based on their importance. This information might help healthcare providers make more informed decisions and provide appropriate care to patients, ultimately improving patient outcomes and reducing the burden on healthcare systems[12].

As knowledge of COVID-19 expands, inflammatory factors have been suggested for assessing the severity of the disease[13]. Our study found that PCT, ferritin, and CRP were significantly higher in non-survivors compared to survivors, indicating that excessive inflammation and immune suppression due to sepsis triggered by SARS-CoV-2 infection might be contributing to the severity of the disease and increased mortality[14]. These results were in line with recent studies that have shown the importance of inflammation in the course of COVID-19, which suggested that measuring these markers in combination with other clinical and demographic characteristics could help healthcare providers better predict the severity of the disease and make informed decisions about patient care[15, 16].

Currently, COVID-19 is a complex illness that affects many different systems and organs in the body[17]. Our study findings supported the notion that COVID-19 is a multi-systemic illness that affected not just the respiratory and vascular systems but also the hepatobiliary, cardiovascular, and renal[18]. Our results showed that increased levels of troponin, BUN, AST, and ALT were associated with higher mortality in COVID-19 patients, highlighting the importance of considering the multi-systemic nature of COVID-19 when treating patients and developing treatments[19]. It is clear that the entire body is impacted by the SARS-CoV-2 virus, and understanding the full extent of these impacts will be crucial for effectively managing the disease and reducing the number of fatalities[20]. Further research is needed to fully understand the impact of the disease on each system, and how this can inform the development of more effective treatments and management strategies[21].

The results of our study showed that there were several critical risk factors for the mortality of COVID-19, which were consistent with other studies[22-24]. However, we found a new predictor, MAP, which was the most important factor in predicting COVID-19 mortality. This is significant as MAP is a crucial parameter for intensive care and is essential for ensuring adequate organ perfusion, which is a prerequisite for survival[25]. The inclusion of MAP in our model, which was often neglected in other studies, highlighted the importance of considering not just laboratory indexes, even the chest CT manifestations, but also clinical characteristics in predicting the outcome of COVID-19[26]. The significance of MAP as a predictor of COVID-19 mortality highlighted the need for a comprehensive approach to evaluating patients, taking into account not just laboratory results, but also their overall clinical status[27]. This is especially important in the context of the ongoing COVID-19 pandemic, where the health systems are under increasing pressure due to the highly contagious variants of SARS-CoV-2[28]. Reliable early prediction of survival is crucial for the effective management of patients and can help to ensure that the limited resources of health systems are used most effectively[29].

Taken together, machine learning-based models, offer a promising approach in determining the mortality of COVID-19 patients[30]. With the recent surge of COVID-19 cases and the emergence of highly infectious SARS-CoV-2 variants, there is a pressing need for reliable and accurate predictive models[31]. Our models, which were based on clinical features from four different hospitals, may provide valuable insights into the management of COVID-19 patients and aid healthcare systems that are struggling to keep up with the increasing demand. Moreover, as the number of potential treatments for COVID-19 continues to grow, the risk factors analyzed by us might have the potential to serve as indicators of treatment efficacy, helping healthcare providers make informed decisions and choose the best course of action for their patients[32].

5. LIMITATIONS

Our study had several limitations that need to be taken into account. First, the results of our study were based on clinical data collected from four hospitals. While other datasets or algorithms may yield different outcomes. Secondly, the level of care received, such as nursing in an intensive care unit or primary care, can greatly impact the mortality of patients. However, this variable could not be considered in our study due to the lack of information and the varying admission criteria for intensive care units in different facilities. Finally, the COVID-19 pandemic has placed a significant strain on hospitals, leading to an altered objective for COVID-19-related tasks and potentially affecting the mortality of patients, which is beyond our control as it is a global crisis.

6. CONCLUSIONS

In conclusion, our study employed three machine learning models, including RF, PLS-DA, and SVM, to predict the fatality of COVID-19 using a third-party database from four different hospitals. The RF model demonstrated the best performance, providing valuable information for the early diagnosis of severe COVID-19 and guiding disease management with appropriate interventions. Additionally, the study identified several biomarkers that were associated with the prognosis of COVID-19, including age, PCT, ferritin, CRP, troponin, BUN, MAP, AST, and ALT, which merit further investigation as prognostic indicators of severe COVID-19.

8. Article Summary

• 1Why is this topic important?

The COVID-19 pandemic has caused significant morbidity and mortality worldwide. Early identification of patients at high risk for severe disease can improve patient outcomes and optimize healthcare resource allocation.

• 2What does this study attempt to show?

This study aimed to develop machine learning models to predict the fatality of COVID-19 and to identify prognostic biomarkers associated with severe COVID-19.

• 3What are the key findings?

The Random Forest (RF) model demonstrated the best performance in predicting COVID-19 fatality. Several biomarkers, including age, PCT, ferritin, CRP, troponin, BUN, MAP, AST, and ALT, were identified as significant predictors of severe COVID-19.

• 4How is patient care impacted?

This study's findings suggest that machine learning models can effectively predict the fatality of COVID-19, providing valuable information for the early diagnosis of severe COVID-19 and guiding disease management with appropriate interventions. Additionally, the identified prognostic biomarkers can be useful in risk stratification and clinical decision-making for patients with COVID-19.

Authors' contributions

Yuchen Shi, Yanwen Qin, and Jinghua Liu conceived the study and designed the protocol. Yuchen Shi and Ping Wang integrated the data. Yuchen Shi and Ze Zheng were responsible for the selection of the study, the extraction of data, and the evaluation of the quality of the study. Jinghua Liu critically revised the manuscript. All authors wrote and approved the final manuscript.

Ethics approval and consent to participate

The study protocol was approved by the Human Research Ethics Committee of Beijing Anzhen Hospital, Capital Medical University, Beijing, China, and the study adheres to all principles of the Declaration of Helsinki (as revised in 2013). Given the retrospective study design, informed consent is waived by the Human Research Ethics Committee of the Beijing Anzhen Hospital, Capital Medical University, Beijing, China.

Consent for publication

Not applicable.

Availability of data and materials

The data sets used during the study are available from the corresponding author on reasonable request.

Reporting Checklist

The authors have completed the TRIPOD reporting checklist.

REFERENCES

• [1]Muralidar S, Ambi SV, Sekaran S, Krishnan UM. The emergence of COVID-19 as a global pandemic: Understanding the epidemiology, immune response and potential therapeutic targets of SARS-CoV-2. Biochimie. 2020;179:85-100. https://doi.org/10.1016/j.biochi.2020.09.018.
• [2]Hamrouni AM, Sharif RS, Sharif SI, Hassanein MM, Abduelkarem AR. Impacts of COVID-19 Pandemic on Geopolitics, Health, Economics, Education and Sociocultural Events. Risk management and healthcare policy. 2022;15:935-43. https://doi.org/10.2147/rmhp.S362337.
• [3]Wong MK, Brooks DJ, Ikejezie J, Gacic-Dobo M, Dumolard L, Nedelec Y, et al. COVID-19 Mortality and Progress Toward Vaccinating Older Adults - World Health Organization, Worldwide, 2020-2022. MMWR Morbidity and mortality weekly report. 2023;72(5):113-8. https://doi.org/10.15585/mmwr.mm7205a1.
• [4]Ho D, Quake SR, McCabe ERB, Chng WJ, Chow EK, Ding X, et al. Enabling Technologies for Personalized and Precision Medicine. Trends in biotechnology. 2020;38(5):497-518. https://doi.org/10.1016/j.tibtech.2019.12.021.
• [5]Liang H, Guo Y, Chen X, Ang KL, He Y, Jiang N, et al. Artificial intelligence for stepwise diagnosis and monitoring of COVID-19. European radiology. 2022;32(4):2235-45. https://doi.org/10.1007/s00330-021-08334-6.
• [6]Eskandar EN, Altschul DJ, de la Garza Ramos R, Cezayirli P, Unda SR, Benton J, et al. Neurologic Syndromes Predict Higher In-Hospital Mortality in COVID-19. Neurology. 2021;96(11):e1527-e38. https://doi.org/10.1212/wnl.0000000000011356.
• [7]Pascarella G, Strumia A, Piliego C, Bruno F, Del Buono R, Costa F, et al. COVID-19 diagnosis and management: a comprehensive review. Journal of internal medicine. 2020;288(2):192-206. https://doi.org/10.1111/joim.13091.
• [8]Altschul DJ, Unda SR, Benton J, de la Garza Ramos R, Cezayirli P, Mehler M, et al. A novel severity score to predict inpatient mortality in COVID-19 patients. Scientific reports. 2020;10(1):16726. https://doi.org/10.1038/s41598-020-73962-9.
• [9]Ghosh T, Zhang W, Ghosh D, Kechris K. Predictive Modeling for Metabolomics Data. Methods in molecular biology (Clifton, NJ). 2020;2104:313-36. https://doi.org/10.1007/978-1-0716-0239-3_16.
• [10]Zhang M, Kim R, Amster Y, Rantanen J, Gassert TH. Editorial: COVID-19: Challenges, opportunities and lessons for occupational health. Frontiers in public health. 2022;10:1123920. https://doi.org/10.3389/fpubh.2022.1123920.
• [11]Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, et al. Artificial intelligence in healthcare: past, present and future. Stroke and vascular neurology. 2017;2(4):230-43. https://doi.org/10.1136/svn-2017-000101.
• [12]Lundon DJ, Kelly BD, Nair S, Bolton DM, Patel G, Reich D, et al. A COVID-19 Test Triage Tool, Predicting Negative Results and Reducing the Testing Burden on Healthcare Systems During a Pandemic. Frontiers in medicine. 2021;8:563465. https://doi.org/10.3389/fmed.2021.563465.
• [13]Hamer M, Kivimäki M, Gale CR, Batty GD. Lifestyle risk factors, inflammatory mechanisms, and COVID-19 hospitalization: A community-based cohort study of 387,109 adults in UK. Brain, behavior, and immunity. 2020;87:184-7. https://doi.org/10.1016/j.bbi.2020.05.059.
• [14]Ni M, Tian FB, Xiang DD, Yu B. Characteristics of inflammatory factors and lymphocyte subsets in patients with severe COVID-19. Journal of medical virology. 2020;92(11):2600-6. https://doi.org/10.1002/jmv.26070.
• [15]Stasiak A, Perdas E, Smolewska E. Risk factors of a severe course of pediatric multi-system inflammatory syndrome temporally associated with COVID-19. European journal of pediatrics. 2022;181(10):3733-8. https://doi.org/10.1007/s00431-022-04584-8.
• [16]Deng F, Zhang L, Lyu L, Lu Z, Gao D, Ma X, et al. Increased levels of ferritin on admission predicts intensive care unit mortality in patients with COVID-19. Medicina clinica (English ed). 2021;156(7):324-31. https://doi.org/10.1016/j.medcle.2020.11.015.
• [17]Bourgonje AR, Abdulle AE, Timens W, Hillebrands JL, Navis GJ, Gordijn SJ, et al. Angiotensin-converting enzyme 2 (ACE2), SARS-CoV-2 and the pathophysiology of coronavirus disease 2019 (COVID-19). The Journal of pathology. 2020;251(3):228-48. https://doi.org/10.1002/path.5471.
• [18]Kouhpayeh H. Clinical features predicting COVID-19 mortality risk. European journal of translational myology. 2022;32(2)https://doi.org/10.4081/ejtm.2022.10268.
• [19]Ghahramani S, Tabrizi R, Lankarani KB, Kashani SMA, Rezaei S, Zeidi N, et al. Laboratory features of severe vs. non-severe COVID-19 patients in Asian populations: a systematic review and meta-analysis. European journal of medical research. 2020;25(1):30. https://doi.org/10.1186/s40001-020-00432-3.
• [20]Hartog N, Faber W, Frisch A, Bauss J, Bupp CP, Rajasekaran S, et al. SARS-CoV-2 infection: molecular mechanisms of severe outcomes to suggest therapeutics. Expert review of proteomics. 2021;18(2):105-18. https://doi.org/10.1080/14789450.2021.1908894.
• [21]Deinhardt-Emmer S, Wittschieber D, Sanft J, Kleemann S, Elschner S, Haupt KF, et al. Early postmortem mapping of SARS-CoV-2 RNA in patients with COVID-19 and the correlation with tissue damage. eLife. 2021;10https://doi.org/10.7554/eLife.60361.
• [22]Chen Y, Klein SL, Garibaldi BT, Li H, Wu C, Osevala NM, et al. Aging in COVID-19: Vulnerability, immunity and intervention. Ageing research reviews. 2021;65:101205. https://doi.org/10.1016/j.arr.2020.101205.
• [23]Hu R, Han C, Pei S, Yin M, Chen X. Procalcitonin levels in COVID-19 patients. International journal of antimicrobial agents. 2020;56(2):106051. https://doi.org/10.1016/j.ijantimicag.2020.106051.
• [24]Ponti G, Maccaferri M, Ruini C, Tomasi A, Ozben T. Biomarkers associated with COVID-19 disease progression. Critical reviews in clinical laboratory sciences. 2020;57(6):389-99. https://doi.org/10.1080/10408363.2020.1770685.
• [25]Ait-Oufella H, Lemoinne S, Boelle PY, Galbois A, Baudel JL, Lemant J, et al. Mottling score predicts survival in septic shock. Intensive care medicine. 2011;37(5):801-7. https://doi.org/10.1007/s00134-011-2163-y.
• [26]Stucchi R, Poli G, Fumagalli R. Hemodynamic monitoring in ICU. Minerva anestesiologica. 2006;72(6):483-7.
• [27]Cai W, Liu T, Xue X, Luo G, Wang X, Shen Y, et al. CT Quantification and Machine-learning Models for Assessment of Disease Severity and Prognosis of COVID-19 Patients. Academic radiology. 2020;27(12):1665-78. https://doi.org/10.1016/j.acra.2020.09.004.
• [28]Khankeh H, Kolivand P, Fathi M, Lornejad H, Abbasabadi-Arab M. Risk Assessment of COVID-19 in the Iranian Health System. Disaster medicine and public health preparedness. 2022;16(5):1806-10. https://doi.org/10.1017/dmp.2021.168.
• [29]Almeida F. Exploring the Impact of COVID-19 on the Sustainability of Health Critical Care Systems in South America. International journal of health policy and management. 2021;10(8):462-4. https://doi.org/10.34172/ijhpm.2020.116.
• [30]Fleuren LM, Dam TA, Tonutti M, de Bruin DP, Lalisang RCA, Gommers D, et al. Predictors for extubation failure in COVID-19 patients using a machine learning approach. Critical care (London, England). 2021;25(1):448. https://doi.org/10.1186/s13054-021-03864-3.
• [31]Abd-Alrazaq A, Alajlani M, Alhuwail D, Schneider J, Al-Kuwari S, Shah Z, et al. Artificial Intelligence in the Fight Against COVID-19: Scoping Review. Journal of medical Internet research. 2020;22(12):e20756. https://doi.org/10.2196/20756.
• [32]Greenwood D, Taverner T, Adderley NJ, Price MJ, Gokhale K, Sainsbury C, et al. Machine learning of COVID-19 clinical data identifies population structures with therapeutic potential. iScience. 2022;25(7):104480. https://doi.org/10.1016/j.isci.2022.104480.