Abstract
Objective:
To assess ability of National Early Warning Score 2 (NEWS2), systemic inflammatory response syndrome (SIRS), quick Sequential Organ Failure Assessment (qSOFA), and CRB-65 calculated at the time of intensive care unit (ICU) admission for predicting ICU mortality in patients of laboratory confirmed coronavirus disease 2019 (COVID-19) infection.
Methods:
This prospective data analysis was based on chart reviews for laboratory confirmed COVID-19 patients admitted to ICUs over a 1-mo period. The NEWS2, CRB-65, qSOFA, and SIRS were calculated from the first recorded vital signs upon admission to ICU and assessed for predicting mortality.
Results:
Total of 140 patients aged between 18 and 95 y were included in the analysis of whom majority were >60 y (47.8%), with evidence of pre-existing comorbidities (67.1%). The most common symptom at presentation was dyspnea (86.4%). Based upon the receiver operating characteristics area under the curve (AUC), the best discriminatory power to predict ICU mortality was for the CRB-65 (AUC: 0.720 [95% confidence interval [CI]: 0.630-0.811]) followed closely by NEWS2 (AUC: 0.712 [95% CI: 0.622-0.803]). Additionally, a multivariate Cox regression model showed Glasgow Coma Scale score at time of admission (P < 0.001; adjusted hazard ratio = 0.808 [95% CI: 0.715-0.911]) to be the only significant predictor of ICU mortality.
Conclusions:
CRB-65 and NEWS2 scores assessed at the time of ICU admission offer only a fair discriminatory value for predicting mortality. Further evaluation after adding laboratory markers such as C-reactive protein and D-dimer may yield a more useful prediction model. Much of the earlier data is from developed countries and uses scoring at time of hospital admission. This study was from a developing country, with the scores assessed at time of ICU admission, rather than the emergency department as with existing data from developed countries, for patients with moderate/severe COVID-19 disease. Because the scores showed some utility for predicting ICU mortality even when measured at time of ICU admission, their use in allocation of limited ICU resources in a developing country merits further research.
Keywords: COVID-19, emergency warning scores, NEWS2, CRB-65, ICU mortality
The coronavirus disease 2019 (COVID-19) pandemic has imposed an unprecedented burden on health-care resources especially intensive care, and developing countries face a further challenge of optimizing allocation of constrained resources.1
When faced with resource challenge, patients with better chances of recovery may be preferred. Early warning scores successfully quantify the severity of illness, help decide regarding need of intensive care unit (ICU) admission, and also correlate with patient mortality. Originally, early warning scores were validated in non-COVID-19 patients, when calculated at the time of hospital admission itself, typically in the emergency areas. Their success to predict mortality and need of ICU admission in COVID-19 patients is documented in some recent data from developed countries.2–6 There are, however, no such data from developing nations.
Also, there is no evidence to identify whether the early warning scores can be used within an ICU to stratify patients at increased risk of mortality. In developing countries with resource limitations, the early warning scores are not used regularly in emergency departments to help decide about ICU admission.
The National Early Warning Score 2 (NEWS2), quick Sequential Organ Failure Assessment (qSOFA), systemic inflammatory response syndrome (SIRS) criteria, and CRB-65 are among the most commonly used early warning scores, with some evidence from developed countries supporting their use in COVID-19 patients.2–6
Against the above background, this single-center study from a COVID-dedicated large public hospital in northern India aimed to assess ability of NEWS2, SIRS, qSOFA, and CRB-65 calculated at the time of ICU admission for predicting ICU mortality in patients of laboratory confirmed COVID-19 infection.
Methods
This prospective data analysis from chart reviews was undertaken after approval of the Institutional Ethics Committee in its meeting held on September 10, 2020. A waiver for written informed consent was obtained.
Study Population
Data of patients with laboratory confirmed COVID-19 (either real-time polymerase chain reaction [RT-PCR] or rapid antigen testing) admitted to ICUs over a 1-mo period were included. Those admitted with suspected COVID-19 but subsequently testing negative were not included for analysis.
Institutional ICU admission criteria included any patient with moderate or severe COVID-19 (SpO2 on room air of 90-94% or <90%, respectively), or with comorbidities and age >65 y (at risk of developing severe disease).
Calculation of the Early Warning Scores
The following 4 clinical scores were calculated from the first recorded vital signs upon admission to ICU.
The NEWS2 is based on aggregate scoring of respiratory rate, oxygen saturation, systolic blood pressure, pulse rate, level of consciousness or new confusion, and body temperature; and additional 2 points added for supplementary oxygen.2 For calculating qSOFA score presence or absence of altered mental status, systolic blood pressure ≤100 mmHg, or respiratory rate of ≥22/min, was noted and each scored with 1 point.7 Presence of the 4 individual criteria of SIRS was sought as temperature >38°C or <36°C, heart rate >90 per min, respiratory rate >20 or PaCO2 < 32 mmHg, and white blood cells >12,000/mm3 or <4000/mm.3,8 For CRB-65 1 point each was given for new onset confusion, respiratory rate ≥30, and systolic blood pressure <90 mmHg or diastolic blood pressure ≤60 mmHg, as well as age ≥65 y.9
Variables Recorded
All the above variables needed to calculate the early warning scores at time of ICU admission were noted (blood pressure, respiratory rate, SpO2 on room air, temperature, level of consciousness, need of supplemental oxygen, and the total leucocyte count were noted from case records). Additionally, the outcome of patient in ICU (death/discharge) was recorded.
As ancillary data, the demographic details (age, gender, presence of comorbidities), duration of complaint suggestive of COVID-19 infection before hospitalization, duration of ICU stay, geographical area of inhabitation (Delhi or other state), source of ICU admission (emergency or ward), and history of prior hospitalization in another health facility with subsequent referral were noted.
Statistical Analysis and Sample Size Estimation
Statistical analysis was done using the SPSS software (version 27). Data are presented as mean ± (standard deviation), median [interquartile range {IQR}], or percentages as appropriate. To evaluate the discriminatory value of various warning scores for ICU mortality, receiver operating characteristics (ROC) analysis with area under the ROC curve (AUC) and its 95% confidence interval (95% CI) were calculated. An AUC >0.7 was regarded as indicating a test of acceptable clinical value and an AUC >0.9 as a test of high clinical value.10 To compare characteristics between survivors and nonsurvivors, chi-squared or Wilcoxon or t-test were applied as appropriate. The Cox proportional regression model was used to study association of various predictors with ICU mortality, and a Kaplan-Meier analysis done for cumulative incidence and median time to event of mortality.
Results
During the study period 512 patients with COVID-19 were admitted to our hospital, of which 146 were admitted to the enrolled ICUs having a strength of 64 beds. After excluding 6/146 of the patients due to nonconfirmatory RT-PCR for COVID-19 infection, the final analysis included 140 patients.
Epidemiology
The baseline patient characteristics, along with their comparison between survivors and nonsurvivors is shown in Table 1. The nonsurvivors had significantly greater age; longer durations of ICU as well as hospital stay; higher respiratory rate, heart rate, and FiO2 requirement; as well as lower Glasgow Coma Scale score (P < 0.05) (Table 1).
Table 1.
Patient characteristics and details
| Characteristic | Total (n = 140) | Survivors (n = 88) | Nonsurvivors (n = 52) | P-value* |
|---|---|---|---|---|
| Age (years) | 56.8 ± 15.4 | 54.1 ± 15.2 | 61.3 ± 15.0 | 0.007 |
| Gender (males) | 97 (69.2%) | 61 (69.3%) | 36 (69.2%) | 0.991 |
| Pre-existing comorbidities | 94 (67.1%) | 58 (65.9%) | 36 (69.2%) | 0.686 |
| Place of inhabitation (other state) | 54 (38.5%) | 31 (35.2%) | 23 (44.2%) | 0.290 |
| Symptom onset to hospitalization (days) | 5 [3-7] | 5 [3-9] | 5 [3-7] | 0.572 |
| Dyspnea | 121 (86.4%) | 76 (86.4%) | 45 (86.5%) | 0.977 |
| Fever | 111 (79.3%) | 71 (80.7%) | 40 (76.9%) | 0.596 |
| Cough | 87 (62.1%) | 58 (65.9%) | 29 (55.8%) | 0.232 |
| Weakness | 13 (9.3%) | 8 (9.1%) | 5 (9.6%) | 1.000 |
| Myalgia | 4 (2.9%) | 4 (4.5%) | 0 (0%) | 0.297 |
| Other symptoms | 34 (24.2%) | 20 (22.7%) | 14 (26.9%) | 0.576 |
| Duration of ICU stay | 14 [9-21] | 17 [11-23] | 8 [6-18] | 0.000 |
| Duration of hospital stay | 17 [10-24] | 19 [13-28] | 10 [6-18] | 0.000 |
| Referred from other facility | 69 (49.3%) | 42 (47.7%) | 27 (51.9%) | 0.631 |
| ICU mortality | 52 (37.1%) | – | – | – |
| FiO2 | 0.8 ± 0.1 | 0.7 ± 0.2 | 0.8 ± 0.1 | 0.006 |
| Systolic blood pressure (mmHg) | 125 ± 16 128 [117-132] |
125 ± 14 | 126 ± 19 | 0.664 |
| Diastolic blood pressure (mmHg) | 78 ± 11 79 [70-86] |
78 ± 10 | 78 ± 12 | 0.736 |
| Respiratory rate (per min) | 23 ± 6 (12-40) |
23 ± 6 | 25 ± 6 | 0.040 |
| Heart rate (per min) | 91 [85-100] (56-140) |
91 ± 13 | 98 ± 15 | 0.009 |
| Glasgow Coma Scale score | 15 [15-15] (5-15) |
15 ± 0 | 14 ± 2 | 0.001 |
| Total leucocyte count (/mm3) | 10282 [7000-12500] | 10428 [6700-13000] | 10034 [7325-12050] | 0.620 |
Note: Values are mean ± SD or median [IQR] or number of patients (%).
Comparison between survivors and nonsurvivors.
The various comorbidities that were observed included in decreasing order of incidence, hypertension (39.3%), diabetes mellitus (33.6%), cardiac ailments such as coronary artery disease, rhematic heart disease, cardiomyopathy (9.3%), respiratory afflictions such as asthma or obstructive lung disease (8.6%), hypothyroidism (7.1%), tuberculosis (5.7%), cerebrovascular accident (4.3%), chronic renal disease (2.1%), and malignancy (1.4%).
While the commonest presentation was dyspnea (86.4%) (Table 1), symptoms besides those typical of viral respiratory disease were seen in almost one-fourth of the cohort, including those related to the gastrointestinal tract (7.8%); chest pain (4.3%); neurological system, ie, slurred speech or altered sensorium (4.3%); sore throat (3.6%); headache (2.8%); palpitations (1.4%); or anosmia (0.7%).
The maximum duration of stay noted in ICU was 46 d, and in the hospital 79 d (Table 1).
Role of Early Warning Scores for Predicting ICU Mortality
ROC analysis for discriminatory power of various scores for ICU mortality were as follows: NEWS2 (P = 0.000; AUC [95% CI] = 0.712 [0.622-0.803]), qSOFA (P = 0.007; AUC [95% CI] = 0.630 [0.535-0.724]), CRB-65 (P = 0.000; AUC [95% CI] = 0.720 [0.630-0.811]), and SIRS criteria (P = 0.374; AUC [95% CI] = 0.544 [0.447-0.642]). At a cutoff value of ≥5 NEWS2 had a sensitivity and specificity of 0.7 each; at ≥2 the qSOFA was highly specific but lacked sensitivity (0.96 and 0.2, respectively), and at ≥1, the CRB-65 had sensitivity and specificity of 0.7 each.
An ROC analysis also showed significant discriminatory value of age for ICU mortality (P = 0.001; AUC [95% CI] =0.658 [0.562-0.754]). At a cutoff value of ≥55 y, age of patient had a sensitivity and specificity of 0.7 and 0.6, respectively.
Model for Predictors of ICU Mortality
The various predictors assessed for association with ICU mortality using the univariate cox proportional hazard model included age (P = 0.010), gender (P = 0.491), comorbidities (P = 0.187), time since onset of first symptom to hospitalization (P = 0.529), systolic blood pressure (P = 0.554), heart rate (P = 0.022), respiratory rate (P = 0.129), fraction of inspired oxygen needed (P = 0.024), and the Glasgow Coma Scale score (P < 0.001). A multivariate analysis testing for all significant predictors, ie, age, heart rate, fraction of inspired oxygen needed, and the Glasgow Coma Scale score, showed a significant prediction value only for the Glasgow Coma Scale score with an adjusted hazard ratio of 0.808 (95% CI: 0.715-0.911) (P < 0.001). Thus, for increase in Glasgow Coma Scale score by every unit score, the hazard of ICU mortality decreased by 20%. The Kaplan-Meier survival curve for ICU mortality is depicted in Figure 1. The median time for the survival function was 29 days (95% CI: 22-35).
Figure 1.
Kaplan-Meier survival curve for ICU mortality.
Discussion
The present study aimed to assess ability of certain early warning scores calculated at the time of ICU admission for predicting mortality in patients admitted with COVID-19 infection. The salient findings included a statistically significant discriminatory value of NEWS2, qSOFA, and CRB-65 for ICU mortality (P < 0.05), but not for SIRS. However, the low sensitivity and specificity values, as well as only fairly predictive AUCs fail to generate adequate clinical significance. The highest AUC was seen with the CRB-65 (0.720 [95% CI: 0.630-0.811]) followed very closely by the NEWS2 (0.712 [95% CI: 0.622-0.803]); with both scores offering a sensitivity as well as specificity of approximately 0.7 at cutoff values of ≥1 and ≥5, respectively.
Previously, NEWS2 was noted to have similar discriminatory power for mortality in COVID-19 patients as noted by us.2–4 The AUCs approximated 0.727, with similar sensitivity and specificity as reported by us.
Although the CRB-65 score is well-established for triaging and predicting outcome of patients with community acquired pneumonia, whether admitted in wards or ICU,9 its discriminatory power for mortality in COVID-19 patients appears to be inconclusive, with previously noted presence as well as absence of statistical significance.2,11
The one distinction of CRB-65 over NEWS2 is inclusion of increased age as a criterion to predict poor outcome. COVID-19 has shown an age-related severity and worse outcome globally.1 Increasing age also showed an independent strong association with mortality in our cohort. This is perhaps the reason for marginally greater predictive value of CRB-65 compared with even the NEWS2 observed by us.
Sepsis is a noted complication of critically ill COVID-19 patients. Thus, there have been attempts to use SIRS criteria as well as qSOFA for predicting prognosis of COVD-19 patients.6 However, previous investigations had also produced similar results of lack of correlation with outcome.6,11 Severity and outcome with COVID-19 disease is related to the primary impairment of oxygenation. The hypoxia of COVID-19 has been noted to be peculiarly asymptomatic, thus also termed “happy hypoxia.” This may account for criteria of SIRS including respiratory rate and heart rate being normal despite increasing severity of COVID-19 and, thus, a lack of significant correlation. Previous AUCs of SIRS criteria (approximately 0.6 vs 0.544) and the qSOFA for predicting mortality are similar to our observations.2,6
The present cohort also helps in shedding light on epidemiology of the disease in our country, specifically on those with severe disease. Per the classification of disease severity given by World Health Organization and the Ministry of Health and Family Welfare of India, severe disease includes those with SpO2 <90%, or those with evidence of respiratory distress.1,12
There was a distinct difference between survivors and nonsurvivors. We noted the nonsurvivors to be older, needing higher oxygen concentrations, and being more tachycardic, tachypneic, with a greater incidence of confusion compared with the survivors. These findings are similar to previous data.3–5 However, the only significant predictors for ICU mortality after adjusting for all predictors was the Glasgow Coma Scale score at presentation.
The earlier correlations of greater mortality with chronic comorbidities and male sex were not substantiated in our cohort. This could be secondary to none of our patients having mild disease. Had we compared outcomes with a cohort of mild disease, the presence of comorbidities and male sex could have been greater among the nonsurvivors.
Limitations
Limitations of the present study include lack of follow-up for long-term survival after ICU discharge. Also, this was a single-center study spread over a short period. Thus, regional and temporal trends cannot be inferred.
Conclusions
Based on our observations, it can be concluded that CRB-65 and NEWS2 scores assessed at the time of ICU admission offer only a fair discriminatory value for predicting mortality. Further evaluation after adding laboratory markers such as C-reactive protein and D-dimer may yield a more appropriate and useful prediction model. Much of the earlier data in this area of research is from developed countries and uses scoring at time of hospital admission. Our study is a novel attempt from a developing country, with the scores assessed at time of ICU admission among patients with moderate/severe COVID-19 disease. Because the scores showed some utility for predicting ICU mortality even when measured at time of ICU admission, and not in the emergency department as with earlier data from developed centers, their use in subsequent allocation of limited ICU resources in a developing country merits further research.
Author Contributions
Asha Tyagi: Helped in designing the study; analyzing the data; writing/reviewing the manuscript, approving final submission. Surbhi Tyagi: Helped in designing the study; analyzing the data; writing/reviewing the manuscript, approving final submission. Ananya Agrawal: Helped in designing the study; analyzing the data; writing/reviewing the manuscript, approving final submission. Aparna Mohan: Helped in designing the study; analyzing the data; writing/reviewing the manuscript, approving final submission. Devansh Garg: Helped in designing the study; analyzing the data; writing/reviewing the manuscript, approving final submission. Rashmi Salhotra: Helped in designing the study; analyzing the data; writing/reviewing the manuscript, approving final submission. Ashok Kumar Saxena: Helped in designing the study; analyzing the data; writing/reviewing the manuscript, approving final submission. Ashish Goel: Helped in designing the study; analyzing the data; writing/reviewing the manuscript, approving final submission.
Conflict(s) of Interest
None.
References
- 1.WHO. COVID-19 Clinical management. Living guidance. 25 January 2021. https://apps.who.int/iris/bitstream/handle/10665/338882/WHO-2019-nCoV-clinical-2021.1-eng.pdf. Accessed February 19, 2021.
- 2.Myrstad M, Ihle-Hansen H, Tveita AA, et al. National Early Warning Score 2 (NEWS2) on admission predicts severe disease and in-hospital mortality from Covid-19 – a prospective cohort study. Scand J Trauma Resusc Emerg Med. 2020;28(1):66. doi: 10.1186/s13049-020-00764-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Hu H, Yao N, Qiu Y. Predictive value of 5 early warning scores for critical COVID-19 patients. Disaster Med Public Health Prep. 2020:1-8. doi: 10.1017/dmp.2020.324 [DOI] [PMC free article] [PubMed]
- 4.Covino M, Matteis GD, Burzo ML, et al. Predicting in-hospital mortality in COVID-19 older patients with specifically developed scores. J Am Geriatr Soc. 2021;69(1):37-43. doi: 10.1111/jgs.16956 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Hu H, Yao N, Qiu Y.Comparing rapid scoring systems in mortality prediction of critically ill patients with novel coronavirus disease. Acad Emerg Med. 2020;27(6):461-468. doi: 10.1111/acem.13992 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Jang JG, Hur J, Hong KS, et al. Prognostic accuracy of the SIRS, qSOFA, and NEWS for early detection of clinical deterioration in SARS-CoV-2 infected patients. J Korean Med Sci. 2020;35(25):e234. doi: 10.3346/jkms.2020.35.e234 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):801-810. doi: 10.1001/jama.2016.0287 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.No Authors Listed. American College of Chest Physicians/Society of Critical Care Medicine Consensus Conference: definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. Crit Care Med. 1992;20(6):864-874. [PubMed] [Google Scholar]
- 9.Lim WS, van der Eerden MM, Laing R, et al. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax. 2003;58(5):377-382. doi: 10.1136/thorax.58.5.377 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Fan J, Upadhye S, Worster A.Understanding receiver operating characteristic (ROC) curves. CJEM. 2006;8(1):19-20. doi: 10.1017/s1481803500013336 [DOI] [PubMed] [Google Scholar]
- 11.Liu F-Y, Sun X-L, Zhang Y, et al. Evaluation of the risk prediction tools for patients with coronavirus disease 2019 in Wuhan, China: a single-centered, retrospective, observational study. Crit Care Med. 2020;48(11):e1004-e1011. doi: 10.1097/CCM.0000000000004549 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Clinical management protocol: COVID-19. https://www.mohfw.gov.in/pdf/UpdatedDetailedClinicalManagementProtocolforCOVID19adultsdated24052021.pdf. Accessed February 22, 2021.