Evaluation of the utility of NEWS2 during the COVID-19 pandemic

ABSTRACT

The emergence of the COVID-19 pandemic resulted in a dramatic increase in acutely ill patients presenting to hospitals with life-threatening acute respiratory disease. There was an immediate need for effective triage systems to facilitate clinical decision making. This review assesses the performance of the National Early Warning Score 2 (NEWS2) in two contexts. Firstly, the ability to detect acute illness severity and likely clinical deterioration in patients presenting to hospitals with COVID-19. Secondly, the use of NEWS2 in the longitudinal monitoring to detect acute clinical deterioration in hospitalised patients with COVID-19. NEWS2 appeared to be at least comparable and, often, superior to other scoring systems (such as qSOFA and CURB-65), and provided an earlier alert of deterioration. A NEWS2 of 5 had high short-term sensitivity within and was unlikely to miss patients with COVID-19 who go on to deteriorate, but this comes with moderate specificity. However, the specificity of these systems is likely underestimated because preventing deterioration is their purpose.

NEWS2 is an adjunct to clinical decision making and has served that purpose during the COVID-19 pandemic, playing an important role in communicating illness severity, clinical deterioration, triaging patients to appropriate levels of care and prompting completion of treatment escalation plans for those with high scores and at imminent risk of deterioration.

Introduction

When the COVID-19 pandemic first emerged as a major global healthcare challenge in early 2020, it soon became apparent that some patients experienced a severe pneumonia, often culminating in a rapid deterioration of respiratory function and the requirement for respiratory support. In those developing these severe symptoms in the first wave of the pandemic, mortality was high, typically over 20% in those needing hospitalisation, and much higher in those requiring intensive care and ventilation.¹ This created enormous pressure on the provision of acute hospital care and intensive care bed availability. Triaging patients with COVID-19 in the community or on arrival at emergency departments and in those hospitalised, was challenging. This was not only due to the volume of cases but also the unpredictable and often rapid course of respiratory decline, and the emergent phenomenon termed ‘silent hypoxia’ in which severe and life-threatening hypoxia was seemingly well tolerated by some patients, often preceding a dramatic clinical deterioration.²

The National Early Warning Score (NEWS) had been recommended by the Royal College of Physicians in 2012 as a standardised track and trigger early warning system to grade acute illness severity, detect acute clinical deterioration and help guide clinical decision making.³ The features of NEWS are described elsewhere in this edition.⁴ In 2017, coinciding with the launch of an updated version of NEWS (NEWS2), the NHS in England adopted NEWS2 as a standardised national early warning system and mandated its deployment by ambulance services and acute hospitals.^5,6 By the time COVID-19 infections started to emerge in the UK, NEWS2 was being used in almost all NHS acute hospitals and ambulance services as part of their assessment of acute illness.

COVID-19 was a disease we hadn't seen before and, for those of us managing patients with COVID-19 in early March 2020, as with all acute illnesses, NEWS2 was being used as a means of assessing illness severity at baseline and to detect ongoing clinical deterioration. It was unclear, however, how well NEWS2 would perform in the context of COVID-19. NEWS2 had been shown to predict risk of clinical deterioration in acutely ill patients with undifferentiated disease or sepsis.⁴ It was reasonable to assume, therefore, that NEWS2 scores would be elevated in patients with severe pneumonia associated with COVID-19, the principal features of which were profound hypoxia, a requirement for supplemental oxygen and an increased respiratory rate; accompanied, in some, by a fever and tachycardia, and less commonly acute confusion. All of these parameters form part of the physiological scoring system for NEWS2, thus, one would anticipate an elevated NEWS2 score in patients acutely ill with COVID-19 pneumonia; for example, it was not uncommon for patients presenting to hospital to have oxygen saturations below 92% and require supplemental oxygen, and these two factors alone would result in a NEWS2 score of 5, the trigger threshold for escalation of care. Subsequently, a number of studies have reported evaluations of NEWS2 in patients acutely ill with COVID-19 and these are reviewed later.

Studies evaluating baseline NEWS2 in patients acutely ill with COVID-19

Many studies early in the pandemic were small cohort studies but provided reassuring and important insights. An early study from China in 654 patients hospitalised with COVID-19, the baseline NEWS2 score predicted mortality with an area under the receiver operating characteristic curve (AUROC) of 0.81. When compared with other acute infection scoring systems, this was comparable to CURB-65 (AUROC 0.85) and superior to quick Sequential Organ Failure Assessment Score (qSOFA; AUROC 0.73).⁷ These findings were consistent with a small study from Korea (110 patients) that showed that baseline NEWS in patients admitted to hospital with COVID-19, predicted the requirement for intensive care or death with an AUROC of 0.92, again superior to qSOFA (AUROC 0.76).⁸

A small study from Norway (66 patients) showed that the NEWS2 score at hospital admission, predicted severe disease and in-hospital mortality with 80% sensitivity and 84% specificity.⁹ NEWS2 was also superior to other scores used for this purpose (qSOFA, Systemic Inflammatory Response Syndrome (SIRS) and CURB-65).⁹ Another small study from Italy early in the pandemic showed that NEWS2 at hospital admission was a good predictor of requirement for intensive care unit (ICU) admission with an AUROC analysis of 0.90, and reported that patients were correctly stratified on admission.¹⁰

A much larger study in the UK evaluated the performance of the NEWS/NEWS2 to discriminate the combined outcome of either death or ICU admission within 24 hours of vital signs measurement in patient cohorts admitted to a UK hospital with either proven COVID-19, COVID-19 not detected, those not tested and a large control cohort from 2019 (pre-pandemic).¹¹ The AUROC value for NEWS/NEWS2 in COVID-19 patients was 0.89. They concluded that the performance of NEWS was similar and good in all cohorts and that amendments to NEWS/NEWS2, such as adding new covariates or adjusting the weighting of the scoring system, was unnecessary when evaluating patients with COVID-19. They also noted that their results supported the national and international recommendations for the use of NEWS or NEWS2 for the assessment of acute illness severity in patients with COVID-19.¹¹

Around the same time, another study reported data from the UK, involving 830 patients with COVID-19 during the first wave admitted to seven hospitals, found that a NEWS2 score of <5 accurately identified a group of patients as a low-risk group for early mortality.¹² In these patients, 2.3% died within 72 hours when the NEWS2 score was <5, whereas 10.5% died when the score was 5 or more.¹² Taken together, these studies all suggested that baseline NEWS2 was performing at least as well in predicting the requirement for a major escalation of care or death as other systems designed to detect acute clinical deterioration due to infection, especially within the first 24–72 hours of assessment, with a high negative predictive value but a much more modest positive predictive value. This conclusion is consistent with the findings of another large study involving 70 emergency departments across the UK that concluded that NEWS2, along with other scoring systems for infection provide good but not excellent prediction for adverse outcome in suspected COVID-19.¹³

Studies evaluating longitudinal tracking for clinical deterioration using NEWS2 in hospitalised patients acutely ill with COVID-19

In addition to proving a baseline assessment of acute illness severity, NEWS2 is also used to track physiological wellbeing in hospitalised patients to trigger an alert when physiological disturbance becomes apparent. There are fewer studies of longitudinal monitoring to identify clinical deterioration in patients with COVID-19. In one preliminary report of a study involving only 17 patients admitted to a UK hospital, a highly variable NEWS2, no doubt reflecting physiological instability, was more evident in patients who died (7/10 patients), than in those who survived (0/7 patients).¹⁴ In a similarly small study from France reporting 27 hospitalised COVID-19 patients, the modified Vitalpac Early Warning Score (ViEWS) score predicted clinical deterioration 12 hours before ICU admission with a sensitivity of 94% and specificity of 78%.¹⁵ This is of interest because the scoring system used for NEWS and subsequently NEWS2 is very similar to that used in ViEWS.

A more substantial study from the UK also looked beyond the baseline NEWS2 score and evaluated the ability of NEWS2 to predict clinical deterioration through longitudinal monitoring in a cohort of patients hospitalised with COVID-19.¹⁶ In the study, they compared a NEWS2 threshold score of 5 or more with the equivalent escalation threshold for other scoring systems, such as qSOFA and a modified early warning score (MEWS) during the first wave of the pandemic.^17,18 In the study, a significantly greater number of patients who deteriorated recorded at least one ‘true positive’ NEWS2 trigger (131/133 patients; sensitivity 0.98 (95% confidence interval (CI) 0.96–1.00)) when compared with MEWS (52 patients; sensitivity 0.39 (95% CI 0.31–0.47); p<0.001) and qSOFA (42 patients; sensitivity 0.32 (95% CI 0.24–0.39); p<0.001). Thus, the negative predictive value of NEWS2 (0.96 (95% CI 0.90–1.00)) was significantly greater than that of MEWS (0.59 (95% CI 0.52–0.66); p<0.001) and qSOFA (0.58 (95% CI 0.51–0.65); p<0.001). Importantly, where triggers were recorded within 24 hours before the first event, the first trigger occurred significantly earlier for NEWS2 (median 11.4 hours before the event (interquartile range (IQR) 4.4–20.6)) versus MEWS (median 6.7 hours before the event (IQR 2.9–14.3); p=0.010) and qSOFA (median 5.6 hours before the event (IQR 3.2–12.4); p=0.003; Fig 1).

Fig 1.

Longitudinal trend in National Early Warning Score 2 in the deterioration group prior to occurrence of first serious event. Solid line shows fitted trend (loess regression), teal shading depicts 95% confidence intervals, dashed line shows score trigger threshold. NEWS2 = National Early Warning Score 2. Adapted with permission from Baker KF, Hanrath AT, Schim van der Loeff I et al. National Early Warning Score 2 (NEWS2) to identify inpatient COVID-19 deterioration: a retrospective analysis. Clin Med 2021;21:84–9.

The challenge with all of these systems is the false positive trigger rate, ie a trigger in patients who do not go on to deteriorate. This occurred in 117/163 patients with NEWS2, yielding a specificity of 0.28 (95% CI 0.21–0.35) compared with MEWS (47 patients; specificity of 0.71 (95% CI 0.64–0.78); p<0.001) and qSOFA (37 patients; specificity of 0.77 (95% CI 0.71–0.84); p<0.001). Since NEWS2 scores began to rise many hours prior to occurrence of a serious clinical deterioration event, it is conceivable that in those who remained stable, they did so because an earlier alert resulted in an intervention that caused them to remain stable. The authors acknowledged this, noting the need to determine whether these were true false positives or patients who responded to an appropriate intervention, concluding that their results support the use of NEWS2 monitoring of hospitalised COVID-19 patients as a sensitive method for identifying clinical deterioration and outperforming MEWS and qSOFA in this context.¹⁶

NEWS2 score or component physiology scoring?

One of the largest UK multicentre studies evaluating NEWS2 was undertaken in the west of England.¹⁹ The objective of this study was to examine initial and maximum NEWS2 values and component scores in hospitalised patients with COVID-19 to determine if NEWS2 could be used to identify deterioration in patients with COVID-19 and identify which, if any, of the individual physiological parameters were most predictive of outcomes.¹⁹ As with most other studies, the primary outcome was death in hospital or ICU admission. Patients with higher first NEWS2 values were more likely to deteriorate and require ICU admission and/or die in hospital (Fig 2a). Risk was also related to the change from baseline to maximum score (Fig 2b). Risk of ICU admission and/or death occurred in 7% of patients with a maximum NEWS2 of 0–2, 11% with maximum NEWS2 of 3–6 and 66% of patients with maximum NEWS2 of 7 or more. The length of hospital stay was also proportional to the baseline and maximum NEWS2 score. NEWS2 was shown to be a reasonably good predictor, particularly of short-term (2-day) mortality (AUROC 0.77), declining to AUROC of 0.65 at 30 days.

Fig 2.

a) Outcome by first (baseline) and maximum National Early Warning Score 2 in patients hospitalised with COVID-19. b) Outcome by change in first (baseline) National Early Warning Score 2 to maximum in patients hospitalised with COVID-19. ICU = intensive care unit. Adapted with permission from Scott LJ, Tavaré A, Hill EM, Jordan L et al. Prognostic value of National Early Warning Scores (NEWS2) and component physiology in hospitalised patients with COVID-19: a multicentre study. Emerg Med J 2022;39:589–94 in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license.

No baseline or maximal individual physiological scores were as predictive of 2-day mortality as the aggregate NEWS2 score. The strongest physiological predictor, and the only one meeting an acceptable AUROC level, was a requirement for supplemental oxygen (AUROC 0.71). Oxygen saturation (AUROC 0.65) and respiratory rate (AUROC 0.64) were marginally predictive, whereas systolic blood pressure and temperature were not.¹⁹ These findings are consistent with COVID-19-pneumonia and respiratory failure being the predominant clinical consequence of COVID-19 infection leading to ICU treatment escalation or death in hospital. Interestingly, although concerns have been expressed about ‘silent hypoxia’ masking the seriousness of the clinical condition of patients with COVID-19, this study showed that only 2.6% (28/1071) of patients had oxygen saturations ≤91% with a normal respiratory rate.^2,19

NEWS2 and oxygen requirements in patients with COVID-19

One feature of patients acutely ill with COVID-19-pneumonia was the severity of hypoxia in some patients, the propensity for a rapid decline in respiratory function and a correspondingly rapid escalation in oxygen requirements. The NEWS2 scoring system is a binary system for scoring oxygen supplementation (ie an additional 2 points). It does not account for the amount of oxygen required. Thus, there was potential for oxygen requirements to increase rapidly without this affecting the NEWS2 score. Recognising this, the RCP issued guidance early in the pandemic highlighting that any sudden increase or rapidly changing oxygen requirement should trigger an escalation call to a competent clinical decision maker to review the patient urgently.²⁰

Methods for evaluating performance of early warning scores in the context of COVID-19

There are a plethora of scoring systems and this expanded rapidly during the COVID-19 pandemic in an effort to rapidly produce tools to support clinical decision making.²¹ These scoring systems are challenging to evaluate and compare because many serve different purposes; for example, in predicting prognosis, which is not the primary purpose of early warning scores. Others incorporate the results of investigations, such as blood tests and imaging, which are not usually available at initial assessment or in all of the settings in which these scores are deployed. In most studies, the evaluation tries to link the score to subsequent need for escalation of care or death as an outcome measure. This is potentially flawed because an effective response to the scoring system will eliminate the outcome being assessed as an indicator of the scores’ performance. The effectiveness of this response is influenced by organisational and human factors as well as availability of an effective treatment for the condition in question. A successful clinical response will effectively eliminate the relationship between a high NEWS2 baseline score and the clinical outcome. In effect, a poor relationship between the initial score and the outcome is what we are trying to achieve with our response. This no doubt contributes to the fact that the NEWS/NEWS2 scoring systems were found to be highly sensitive with a high negative predictive value for risk of deterioration >0.9, but with lower specificity with a seemingly high false positive trigger rate (ie a trigger in patients who do not go on to deteriorate). These false positive triggers are almost certainly an overestimate as the purpose of the trigger is to prompt actions to stabilise the patient and prevent the outcome. Furthermore, the outcome will also have been influenced by treatment escalation plans for individual patients; for example, NEWS2 cannot predict escalation of care to ICU if the ceiling of care for a patient has determined that they will not go to ICU. Thus, analysis of the performance of these scores is complex, multifactorial, needs much more granular data than is often available and is often oversimplistic.

Reflections and conclusion

The studies discussed earlier rapidly evaluated NEWS/NEWS2 as a means of identifying patients at risk of clinical deterioration due to COVID-19, principally in the acute hospital setting. More data are needed in the prehospital and community setting. Important points have emerged. Firstly, many authors concluded that the NEWS2 system identified acutely ill patients with COVID-19 and who were likely to need an escalation of care, and were at higher risk of in-hospital mortality. This relationship between NEWS2 and serious outcomes was most evident early, within 24–48 hours of the initial score. This is not surprising as the score is designed to aid assessment of imminent risk of clinical deterioration, rather than as a means of predicting later outcomes. NEWS2 also appeared to be at least comparable and often superior to other scoring systems, such as qSOFA and CURB-65, and importantly provided an earlier alert of deterioration than these others systems.¹⁶ A NEWS2 score of 5 has high sensitivity: it is unlikely to miss patients with COVID-19 who will go on to deteriorate and appears to detect risk earlier than some other scoring systems. However, many studies have emphasised that this comes with moderate specificity, but this is most likely underestimated because preventing deterioration is the raison d’être for NEWS2. In practice, NEWS2 is used as an adjunct to clinical decision making and has served that purpose during the COVID-19 pandemic, playing an important role in communicating illness severity, clinical deterioration, triaging patients to appropriate levels of care and prompting completion of treatment escalation plans for those with high scores and at imminent risk of deterioration.