Abstract
To date, the COVID-19 pandemic has caused millions of deaths across the world. Prognostic scores can improve the clinical management of COVID-19 diagnosis and treatment. The objective of this study was to assess the predictive role of 4C Mortality, CURB-65, and NEWS in COVID-19 mortality among the Romanian population. A single-center, retrospective, observational study was conducted on patients with reverse transcriptase-polymerase chain reaction (RT-PCR)-proven COVID-19 admitted to the Municipal Emergency Clinical Hospital of Timisoara, Romania, between 1 October 2020 and 15 March 2021. Receiver operating characteristic (ROC) and area under the curve (AUC) analyses were performed to determine the discrimination accuracy of the three scores. The mean values of the risk scores were higher in the non-survivors group (survivors group vs. non-survivors group: 8 vs. 15 (4C Mortality Score); 3 vs. 8.5 (NEWS); 1 vs. 3 (CURB-65)). In terms of mortality risk prediction, the NEWS performed best, with an AUC of 0.86, and the CURB-65 score performed poorly, with an AUC of 0.80. CURB-65, NEWS, and 4C Mortality scores were significant mortality predictors in the analysis, with acceptable calibration. Among the scores assessed in our study, NEWS had the highest performance in predicting in-hospital mortality in COVID-19 patients. Thus, the findings from this study suggest that the use of NEWS may be beneficial to the early identification of high-risk COVID-19 patients and the provision of more aggressive care to reduce mortality associated with COVID-19.
Keywords: COVID-19, prediction, mortality, 4C Mortality, NEWS, CURB-65
1. Introduction
In March 2020, WHO declared the new coronavirus outbreak (COVID-19) a global pandemic due to the rapidly increasing number of SARS-CoV-2 infections reported since December 2019 [1]. By 13 February 2022, 410,848,671 cases of COVID-19 and 5,829,566 deaths had been reported worldwide. During this time, 2,531,597 cases of COVID-19 and 61,376 deaths had been registered in Romania [2]. Patients affected by COVID-19 may be asymptomatic; may experience mild symptoms; or may develop atypical pneumonia, respiratory failure and acute respiratory distress syndrome (ARDS) (Appendix A–Table A1), which may lead to death [3,4,5]. Prognostic scores can improve the clinical management of COVID-19 diagnosis and treatment. Early identification of severe cases can lead to more aggressive care and lower mortality. Studies investigating the accuracy of these scores can provide important data for large-scale clinical implementation. Several prognostic scores have been developed to identify the high risk of death in patients with poor outcomes [6,7]. The 4C Mortality Score (where 4C refers to Coronavirus Clinical Characterization Consortium) is a validated predictor of mortality in hospitalized patients with COVID-19 [8,9]. CURB-65 is a commonly applied severity score in community-acquired pneumonia (CAP) management [10,11]. The National Early Warning Score (NEWS) was developed in 2012 in the United Kingdom by the NEWS Development and Implementation Group on behalf of the Royal College of Physicians [12,13]. The objective of the present study was to assess the predictive ability of 4C Mortality, CURB-65, and NEWS in COVID-19 mortality among the Romanian population.
2. Materials and Methods
2.1. Study Design and Setting
A single-center, retrospective, observational study was conducted on patients confirmed to be COVID-19 positive based on the reverse transcription-polymerase chain reaction (RT-PCR) test, and admitted to the Municipal Emergency Clinical Hospital of Timisoara, Romania, between 1 October 2020 and 15 March 2021. The study was approved by the Ethics Committee of the University of Medicine and Pharmacy “Victor Babes” Timisoara (No. 22726/17 November 2021).
2.2. Participants
Participants enrolled in the study met the following criteria: (1) admitted to the Municipal Emergency Clinical Hospital following a positive test for SARS-CoV-2 by real-time reverse transcriptase chain reaction (RT-PCR) on a nasopharyngeal swab between 1 October 2020 and 15 March 2021; (2) had complete clinical and laboratory data documented in electronic medical records; (3) aged over 18 years. Patients under 18 years of age or with missing data were excluded.
2.3. Data Sources Measurement and Outcome
Demographic, clinical, and outcome data were collected from electronic medical records using a prespecified case report form. Demographic data included age and gender. Clinical data included comorbidities and presenting symptoms or signs. Clinical outcomes were also examined, including admission to the intensive care unit, mechanical ventilation, and mortality. The 4C Mortality Score incorporates eight independent parameters: age, gender, number of comorbidities, respiratory rate, peripheral oxygen saturation level, Glasgow Coma Scale, blood urea nitrogen level, and C-reactive protein level. CURB-65 involves five easily obtainable parameters: new-onset confusion, urea >7 mmol/L, respiratory rate ≥30/minute, systolic blood pressure <90 mmHg and/or diastolic blood pressure ≤60 mmHg, and age ≥65 years. Each parameter present is scored with 1 point. The National Early Warning Score is based on six parameters that can be obtained from hospitalized patients or those in the emergency department: respiratory rate, oxygen saturation, temperature, systolic blood pressure, pulse rate, and level of consciousness. The clinical endpoint was in-hospital mortality.
2.4. Statistical Analysis
Statistical analysis was performed using RStudio. Categorical variables were reported as absolute number (n) and relative frequency (%), and were compared using Fisher’s exact test. Depending on the normality of the distribution, continuous variables were represented as a median (interquartile range) or as a mean (±SD). The Mann–Whitney test was used to compare non-parametric data. The Shapiro–Wilk test was used to assess the normality of the distribution. The receiver operating characteristic area under the curve was examined to identify the accuracy of discrimination of the evaluated scores. Binary logistic regression was used to determine the independent predictive value of all the risk scores.
3. Results
3.1. Baseline Characteristics
Our study included 133 COVID-19 patients confirmed by RT-PCR, of which 18 (13.5%) died (Table 1). The median age was 65 years (IQI = 21), with more males (51.1%) than females. The most common comorbidities were hypertension (65.4%) and chronic kidney disease (51.9%) (Table 1).
Table 1.
Baseline characteristics of 133 COVID-19 patients.
| Variables | Overall (n = 133) | Survivors (n = 115) | Died (n = 18) | p-Value |
|---|---|---|---|---|
| Demographics | ||||
| Gender (n/%) |
||||
| Female | 65/48.9% | 58/50.4% | 7/38.9% | 0.45 (OR = 1.59) |
| Male | 68/51.1% | 57/ 49.6% | 11/61.1% | |
| Age (years; median (IQR)) |
65 (21) | 62 (20.5) | 70 (15.5) | 0.02 |
|
Comorbidities (n/%) |
||||
| Hypertension | 87/65.4% | 72/62.6% | 15/83.3% | 0.11 (OR = 2.96) |
| Diabetes | 59/44.4% | 49/42.6% | 10/55.6% | 0.32 (OR = 1.67) |
| Cardiovascular disease | 44/33.1% | 32/27.8% | 12/66.7% | 0.002 (OR = 5.11) |
| CKD | 69/51.9% | 55/47.8% | 14/77.8% | 0.02 (OR = 3.78) |
| COPD/asthma | 26/19.5% | 20/17.4% | 6/33.3% | 0.12 (OR = 2.35) |
| Cancer | 15/11.3% | 11/9.6% | 4/22.2% | 0.12 (OR = 2.67) |
|
Presenting symptoms (n/%) |
||||
| Cough | 77/57.9% | 65/56.5% | 12/66.7% | 0.45 (OR = 1.53) |
| Dyspnea | 69/51.9% | 59/51.3% | 10/55.6% | 0.80 (OR = 1.18) |
| Chest pain | 18/13.5% | 14/12.2% | 4/22.2% | 0.26 (OR = 2.04) |
| Fatigue | 82/61.7% | 68/59.1% | 14/77.8% | 0.19 (OR = 2.40) |
| Myalgia | 33/24.8% | 28/24.3% | 5/27.8% | 0.77 (OR = 1.19) |
| No smell/taste | 18/13.5% | 17/14.8% | 1/5.56% | 0.46 (OR = 0.34) |
| Gastrointestinal symptoms | 52/39.1% | 44/38.3% | 8/44.4% | 0.61 (OR = 1.28) |
|
Clinical course (n/%) |
||||
| Mechanic ventilation | 9/6.77% | 2/1.74% | 7/38.9% | <0.001 (OR = 33.8) |
| ICU admission | 10/7.52% | 4/3.48% | 6/33.3% | <0.001 (OR = 13.3) |
COPD, chronic obstructive pulmonary disease; CKD, chronic kidney disease; ICU, intensive care unit.
3.2. Comparison of 4C Mortality Score, NEWS and CURB-65 Scores
The mean values of the risk scores were higher in the non-survivors group: 4C Mortality Score, 8 (IQI = 8) vs. 15 (IQI = 5.35); NEWS, 3 (IQI = 4) vs. 8.5 (IQI =4.5); and CURB-65, 1 (IQI = 2) vs. 3 (IQI = 1.75). Mortality rates within the 4C Mortality Score risk groups were 0% (Low), 5.56% (Intermediate), 38.9% (High), and 55.6% (Very High) (Table 2).
Table 2.
Comparison of 4C Mortality, NEWS, and CURB-65 scores.
| Score | Overall | Survived | Died | p-Value |
|---|---|---|---|---|
| 4C Mortality Score median (IQI) |
9 (9) | 8 (8) | 15 (5.25) | <0.001 |
| Low (0–3) n (%) |
28 (21.1%) | 28 (24.3%) | - | |
| Intermediate (4–8) n (%) |
34 (25.6%) | 33 (28.7%) | 1 (5.56%) | |
| High (9–14) n (%) |
48 (36.1%) | 41 (35.7%) | 7 (38.9%) | |
| Very high (15–21) n (%) |
23 (17.3%) | 13 (11.3%) | 10 (55.6%) | |
| NEWS median (IQI) |
4 (5) | 3 (4) | 8.5 (4.5) | <0.001 |
| CURB-65 median (IQI) |
1 (2) | 1 (2) | 3 (1.75) | <0.001 |
To determine the discrimination accuracy of the 4C Mortality Score, NEWS, and CURB-65 score, a ROC-AUC analysis was performed (Figure 1). The most accurate scale was the NEWS, with an AUC value of 0.861 (95% CI 0.784–0.939) (Table 3).
Figure 1.
Receiver operating characteristic curve of clinical risk scores in predicting mortality.
Table 3.
Accuracy of 4C Mortality Score, NEWS, and CURB-65.
| Score | AUC | p-Value | 95% CI | |
|---|---|---|---|---|
| Lower | Upper | |||
| 4C Mortality score | 0.818 | <0.001 | 0.718 | 0.919 |
| NEWS | 0.861 | <0.001 | 0.784 | 0.939 |
| CURB-65 | 0.801 | <0.001 | 0.681 | 0.922 |
In terms of mortality risk prediction, the NEWS performed best, with an AUC of 0.86. The CURB-65 score performed poorly, with an AUC of 0.80 (Table 3).
Table 4 shows the mortality accuracy measures for cut-offs 3, 8, 11, and 14 at the 4C Mortality Score. The sensitivity of each cut-off ranged from 100% for 4C > 3 and NEWS > 3 to 55% for 4C > 14. Specificity ranged from 24% for NEWS > 3 to 92% for CURB-65 > 2.
Table 4.
Diagnostic accuracy measures for mortality prediction at cut-offs of the 4C Mortality Score, NEWS, and CURB-65.
| Cut-Off | Sensitivity (%) | Specificity (%) |
|---|---|---|
| 4C Mortality Score | ||
| >3 | 100% | 24% |
| >8 | 94% | 53% |
| >11 | 72% | 67% |
| >14 | 55% | 88% |
| NEWS | ||
| >3 | 100% | 51% |
| CURB-65 | ||
| >2 | 55% | 92% |
3.3. Association of 4C Mortality Score, NEWS, and CURB-65 Scores with COVID-19 Mortality
Kaplan–Meier curves and a univariate binary regression model were created using various cutoff points for 4C Mortality Score, NEWS, and CURB-65. Figure 2 and Figure 3 show the Kaplan–Meier survival curve as a function of the 4C Mortality Score cutoff values. Differences in survival for patients above the baseline 4C Mortality Score of 8, 11, and 14, compared to patients with scores below these values, were statistically significant (Figure 2 and Figure 3). However, differences in survival for patients with 4C Mortality Score values above the threshold of 2, compared to those below the threshold, were not statistically significant (p = 0.12).
Figure 2.
Kaplan–Meier survival curves of hospitalized COVID-19 patients: (a) according to 4C Mortality Score cutoff value of 2; (b) according to 4C Mortality Score cutoff value of 8.
Figure 3.
Kaplan–Meier survival curves of hospitalized COVID-19 patients: (a) according to 4C Mortality Score cutoff value of 11; (b) according to 4C Mortality Score cutoff value of 14.
In addition, differences in patient survival by 4C Mortality Score classification showed highly statistically significant differences (p < 0.001) (Figure 4).
Figure 4.
Kaplan–Meier survival curves of hospitalized COVID-19 patients according to 4C Mortality Score risk.
Differences in survival for patients with NEWS and CURB-65 scores above the baseline, compared to those below the baseline, were highly statistically significant (p < 0.001 for each) (Figure 5).
Figure 5.
Kaplan–Meier survival curves of hospitalized COVID-19 patients: (a) according to NEWS cutoff value of 3; (b) according to CURB-65 cutoff value of 2.
Univariate regression analysis was performed to determine the link between the risk scores and in-hospital mortality. All risk scores were significant mortality predictors in the analysis, with acceptable calibration. The results are presented in Table 5.
Table 5.
Univariate logistic regression analysis of mortality risk scores.
| Score | Odds Ratio | p-Value | 95% CI | |
|---|---|---|---|---|
| Upper | Lower | |||
| 4C Mortality Score | 1.33 | <0.001 | 1.15 | 1.55 |
| NEWS | 1.56 | <0.001 | 1.28 | 1.91 |
| CURB-65 | 3.52 | <0.001 | 1.95 | 6.38 |
4. Discussion
The predictive role of 4C Mortality Score, CURB-65, and NEWS in COVID-19 mortality has not been previously evaluated in the Romanian population. Thus, we evaluated these scores among 133 patients confirmed to be COVID-19 positive by RT-PCR who were admitted to the Municipal Emergency Clinical Hospital of Timisoara, Romania. The mortality rate among these patients was 13.5%, which is lower than the mortality rate reported in a previous article conducted at the Municipal Emergency Clinical Hospital of Timisoara in another study period. The mortality rate is also lower than the 17% mortality rate found in a systematic review among inpatients [14,15]. Age, cardiovascular disease, and chronic kidney disease were associated with increased mortality among patients in this study.
In this study, the 4C Mortality Score provided acceptable specificity and sensitivity in predicting mortality in all patients. We observed an AUC of 0.81, which is higher than the deviation in the original research of 0.79 [8]. In addition, the mortality rates of the Low, Intermediate, High, and Very High risk groups on the 4C Mortality Score were 0.0%, 5.56%, 38.9%, and 55.6%, respectively. These are similar to those reported in the original study, which reported rates of 1.2%, 9.9%, 31.4%, and 61.5%, respectively.
The CURB-65 score, which consists of five parameters (confusion, blood urea nitrogen, respiratory rate, blood pressure, and age), was implemented in 2003 by Lim et al. to easily predict the increased risk of 30-day mortality in patients with CAP [13]. A study conducted by Guo et al. in a Wuhan hospital showed that CURB-65 had good performance in determining mortality, with an AUC of 0.81 [10]. In addition, a study of 481 patients with COVID-19 conducted by Doganay and Rohat demonstrated a good performance of CURB-65 (AUC 0.84) in predicting in-hospital mortality [16]. Consistent with these studies, in our study CURB-65 had a good performance, with an AUC of 0.801, in predicting mortality in COVID-19 patients.
The NEWS is a score implemented by the NEWS Development and Implementation Group on behalf of the Royal College of Physicians in 2012, and uses six parameters for the assessment and response to acute illness: respiratory rate, oxygen saturation, temperature, systolic blood pressure, pulse rate, and level of consciousness [12]. Kostakis et al. demonstrated in a study that NEWS had good performance in predicting mortality among COVID-19 patients [17]. Pokeerbux et al., in a study conducted on a French cohort that included hospitalized COVID-19 patients, showed that NEWS was an independent predictor of ICU transfer and in-hospital mortality, with an AUC of 0.82 [18]. Our study also found that NEWS had higher performance (AUC of 0.861) in predicting mortality among COVID-19 patients, and was the most accurate scale. However, Lim et al. suggest that NEWS be adjusted with a more sensitive score for oxygen demand in order to more accurately consider the development of hypoxic respiratory failure in COVID-19 patients [19].
In addition to these scores, another method of increasing the speed of combating COVID-19 is artificial intelligence. Artificial intelligence applications in clinical settings play an important role in improving the productivity and efficiency of studies involving large samples, with the intention of higher degrees of accuracy in prediction. Another benefit of artificial intelligence is that it can speed up the drug development process in emergency situations, as seen in the COVID-19 pandemic [20,21].
Therefore, prediction scores coupled with artificial intelligence can be particularly important for decision-making needs in crisis situations such as the COVID-19 pandemic.
This study has some limitations. First, the study follows a retrospective design and is based on data from a single health clinic. Second, the sample may not have been large enough to assess the predictive performance of 4C Mortality Score, NEWS, and CURB-65 for death, as there were only 18 deaths in this cohort. In addition, the baseline demographics of survivors and non-survivors were very different; there was a higher percentage of comorbidities among non-survivors.
5. Conclusions
All of the risk scores evaluated were significant mortality predictors in the analysis, with acceptable calibration. Among the scores assessed in our study, NEWS had a higher performance in predicting in-hospital mortality in COVID-19 patients. The findings from this study suggest that the NEWS may be beneficial in the early identification of high-risk COVID-19 patients and in providing more aggressive care to reduce mortality associated with COVID-19.
Appendix A
Table A1.
List of abbreviations. The abbreviations including in the text are reported alphabetically.
| Abbreviation | Full Form | Definition |
|---|---|---|
| ARDS | Acute respiratory distress syndrome | A life-threatening lung injury that allows fluid to leak into the lungs. |
| AUC | Area under the ROC curve | A performance measurement for the classification problems at various threshold settings. |
| CAP | Community-acquired pneumonia | An acute infection of the pulmonary parenchyma in someone who has not recently had close contact with the health care system. |
| CKD | Chronic kidney disease | Kidney damage or glomerular filtration rate (GFR) < 60 mL/min/1.73 m2 for 3 months or more, irrespective of cause. |
| COPD | Chronic obstructive pulmonary disease | A type of lung disease marked by permanent damage to tissues in the lungs, making it hard to breathe. |
| COVID-19 | Coronavirus disease 2019 | A respiratory disease caused by SARS-CoV-2, a coronavirus discovered in 2019. |
| ICU | Intensive care unit | The part of a hospital that provides intensive care. |
| IQI | Interquartile range | A measure of statistical dispersion. |
| .NEWS | National Early Warning Score | A tool developed by the Royal College of Physicians which improves the detection and response to clinical deterioration in adult patients. |
| ROC curve | Receiver operating characteristic curve | A graphical plot that illustrates the diagnostic ability of a binary classifier system as its discrimination threshold is varied. |
| WHO | World Health Organization | A specialized agency of the United Nations responsible for international public health. |
Author Contributions
Conceptualization, C.C., F.G. and D.M.; methodology, F.G. and A.M.; software, B.B. and F.B.; validation, I.M.C., R.P. and O.N.; formal analysis, B.B. and F.B.; investigation, M.L.G. and O.R.; resources, A.R., D.M. and O.N.; data curation, A.R., O.M.G. and R.P.; writing—original draft preparation, O.M.G., F.G. and I.M.C.; writing—review and editing, C.C. and M.L.G.; visualization, O.R.; supervision, C.C., D.M. and A.M. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Institutional Review Board Statement
The study was conducted in accordance with the Declaration of Helsinki, and approved by the Ethics Committee of the University of Medicine and Pharmacy “Victor Babes” Timisoara (No. 22726/17 November 2021).
Informed Consent Statement
Informed consent was obtained from all subjects involved in the study.
Data Availability Statement
The data sets used and/or analyzed during the present study are available from the corresponding author on reasonable request.
Conflicts of Interest
The authors declare no conflict of interest.
Footnotes
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Cucinotta D., Vanelli M. WHO Declares COVID-19 a Pandemic. Acta Biomed. 2020;91:157–160. doi: 10.23750/abm.v91i1.9397. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.COVID Live—Coronavirus Statistics—Worldometer. [(accessed on 13 February 2022)]. Available online: https://www.worldometers.info/coronavirus/
- 3.Yang X., Yu Y., Xu J., Shu H., Xia J., Liu H., Wu Y., Zhang L., Yu Z., Fang M., et al. Clinical Course and Outcomes of Critically Ill Patients with SARS-CoV-2 Pneumonia in Wuhan, China: A Single-Centered, Retrospective, Observational Study. Lancet Respir. Med. 2020;8:475–481. doi: 10.1016/S2213-2600(20)30079-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.George P.M., Barratt S.L., Condliffe R., Desai S.R., Devaraj A., Forrest I., Gibbons M.A., Hart N., Jenkins R.G., McAuley D.F., et al. Respiratory Follow-up of Patients with COVID-19 Pneumonia. Thorax. 2020;75:1009–1016. doi: 10.1136/thoraxjnl-2020-215314. [DOI] [PubMed] [Google Scholar]
- 5.Van Dam P.M.E.L., Zelis N., van Kuijk S.M.J., Linkens A.E.M.J.H., Brüggemann R.A.G., Spaetgens B., van der Horst I.C.C., Stassen P.M. Performance of Prediction Models for Short-Term Outcome in COVID-19 Patients in the Emergency Department: A Retrospective Study. Ann. Med. 2021;53:402–409. doi: 10.1080/07853890.2021.1891453. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Lombardi Y., Azoyan L., Szychowiak P., Bellamine A., Lemaitre G., Bernaux M., Daniel C., Leblanc J., Riller Q., Steichen O. External Validation of Prognostic Scores for COVID-19: A Multicenter Cohort Study of Patients Hospitalized in Greater Paris University Hospitals. Intensive Care Med. 2021;47:1426–1439. doi: 10.1007/s00134-021-06524-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Shi Y., Pandita A., Hardesty A., McCarthy M., Aridi J., Weiss Z.F., Beckwith C.G., Farmakiotis D. Validation of Pneumonia Prognostic Scores in a Statewide Cohort of Hospitalised Patients with COVID-19. Int. J. Clin. Pract. 2020;75:e13926. doi: 10.1111/ijcp.13926. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Knight S.R., Ho A., Pius R., Buchan I., Carson G., Drake T.M., Dunning J., Fairfield C.J., Gamble C., Green C.A., et al. Risk Stratification of Patients Admitted to Hospital with Covid-19 Using the ISARIC WHO Clinical Characterisation Protocol: Development and Validation of the 4C Mortality Score. BMJ. 2020;370:m3339. doi: 10.1136/bmj.m3339. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Kuroda S., Matsumoto S., Sano T., Kitai T., Yonetsu T., Kohsaka S., Torii S., Kishi T., Komuro I., Hirata K., et al. External Validation of the 4C Mortality Score for Patients with COVID-19 and Pre-Existing Cardiovascular Diseases/Risk Factors. BMJ Open. 2021;11:e052708. doi: 10.1136/bmjopen-2021-052708. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Guo J., Zhou B., Zhu M., Yuan Y., Wang Q., Zhou H., Wang X., Lv T., Li S., Liu P., et al. CURB-65 May Serve as a Useful Prognostic Marker in COVID-19 Patients within Wuhan, China: A Retrospective Cohort Study. Epidemiol. Infect. 2020;148:e241. doi: 10.1017/S0950268820002368. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Nguyen Y., Corre F., Honsel V., Curac S., Zarrouk V., Fantin B., Galy A. Applicability of the CURB-65 Pneumonia Severity Score for Outpatient Treatment of COVID-19. J. Infect. 2020;81:e96–e98. doi: 10.1016/j.jinf.2020.05.049. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Royal College of Physicians of London . National Early Warning Score (NEWS): Standardising the Assessment of Acute-Illness Severity in the NHS. Royal College of Physicians; London, UK: 2012. [Google Scholar]
- 13.Lim W., van der Eerden M.M., Laing R., Boersma W., Karalus N., Town G., Lewis S., Macfarlane J. Defining Community Acquired Pneumonia Severity on Presentation to Hospital: An International Derivation and Validation Study. Thorax. 2003;58:377–382. doi: 10.1136/thorax.58.5.377. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Citu C., Gorun F., Motoc A., Sas I., Gorun O.M., Burlea B., Tuta-Sas I., Tomescu L., Neamtu R., Malita D., et al. The Predictive Role of NLR, d-NLR, MLR, and SIRI in COVID-19 Mortality. Diagnostics. 2022;12:122. doi: 10.3390/diagnostics12010122. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Macedo A., Gonçalves N., Febra C. COVID-19 Fatality Rates in Hospitalized Patients: Systematic Review and Meta-Analysis. Ann. Epidemiol. 2021;57:14–21. doi: 10.1016/j.annepidem.2021.02.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Doğanay F., Ak R. Performance of the CURB-65, ISARIC-4C and COVID-GRAM Scores in Terms of Severity for COVID-19 Patients. Int. J. Clin. Pract. 2021;75:e14759. doi: 10.1111/ijcp.14759. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Kostakis I., Smith G.B., Prytherch D., Meredith P., Price C., Chauhan A. The Performance of the National Early Warning Score and National Early Warning Score 2 in Hospitalised Patients Infected by the Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) Resuscitation. 2021;159:150–157. doi: 10.1016/j.resuscitation.2020.10.039. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Pokeerbux M.R., Yelnik C.M., Faure E., Drumez E., Bruandet A., Labreuche J., Assaf A., Goffard A., Garabedian C., Poissy J., et al. National Early Warning Score to Predict Intensive Care Unit Transfer and Mortality in COVID-19 in a French Cohort. Int. J. Clin. Pract. 2021;75:e14121. doi: 10.1111/ijcp.14121. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Lim N.T., Pan D., Barker J. NEWS2 System Requires Modification to Identify Deteriorating Patients with COVID-19. Clin. Med. 2020;20:e133–e134. doi: 10.7861/clinmed.Let.20.4.6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Jamshidi M., Lalbakhsh A., Talla J., Peroutka Z., Hadjilooei F., Lalbakhsh P., Jamshidi M., Spada L.L., Mirmozafari M., Dehghani M., et al. Artificial Intelligence and COVID-19: Deep Learning Approaches for Diagnosis and Treatment. IEEE Access. 2020;8:109581–109595. doi: 10.1109/ACCESS.2020.3001973. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Jamshidi M.B., Talla J., Lalbakhsh A., Sharifi-Atashgah M.S., Sabet A., Peroutka Z. A Conceptual Deep Learning Framework for COVID-19 Drug Discovery; Proceedings of the 2021 IEEE 12th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON); New York, NY, USA. 1–4 December 2021; New York, NY, USA: IEEE; 2021. pp. 00030–00034. [DOI] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data sets used and/or analyzed during the present study are available from the corresponding author on reasonable request.