Abstract
Background
For emergency triage, it is very important to identify patient severity according to their vital signs and chief complaint. Several studies have examined the predictive value of the National Early Warning Score (NEWS) for specific emergency patients and have shown it to be effective. However, few have studied the utility of NEWS in emergency triage for general emergency medicine patients. The aim of this research was to investigate the performance of NEWS in emergency triage with regard to predicting adverse outcomes.
Methods
This was a retrospective cohort study carried out at a tertiary care center hospital in Jinhua, China. A total of 62,403 patients attending the emergency department (ED) from January to December 2018 were included. The NEWS, Modified Early Warning Score (MEWS), and quick Sepsis Related Organ Failure Assessment (qSOFA) score were obtained from emergency triage. Multivariate logistic regression analysis was performed to evaluate the associations between the NEWS, MEWS, and qSOFA, as well as those between other parameters with ED mortality. The predictive performances for emergency observation, death, and intensive care unit (ICU) admission of NEWS, MEWS and qSOFA were compared to the area under the receiver operating characteristic curve (AUROC).
Results
Of the total participants, 6502 were placed under emergency observation, 106 died in the ED, 638 were admitted to the ICU, and 324 died in-hospital. The NEWS, qSOFA, age, and gender were significantly associated with ED mortality. NEWS was significantly better at discriminating all outcomes, and the area under the curve and 95% confidence intervals for ED mortality, observation in ED, composite of ED mortality and ICU admission, and in-hospital mortality were 0.862 (0.859–0.865), 0.691 (0.687–0.695), 0.859 (0.856–0.861), and 0.805 (0.802–0.808), respectively.
Conclusion
NEWS shows good performance in discriminating critical emergency patients in ED triage for emergency medicine patients.
Keywords: National Early Warning Score, triage, emergency department, predict, outcomes
Introduction
The emergency department (ED) of a general hospital has to deal with a large number of patients with varying illness severities every day. Therefore, triage is important in identifying high-risk patients amongst many less urgent patients as ED overcrowding has recently turned into a national crisis.1,2 However, the situation of emergency patients is complex, and information is limited. Overreliance on subjective judgment based on triage nurse experience may lead to results with low reliability and delayed treatment.3,4 Over-triage, where the actual critical level of the patient’s condition is lower than the nurse’s triage, could lead to a waste of medical resources and patient retention; under-triage, where the critical level of the patient’s condition is higher than the nurse’s triage, leads to delayed care for more severely ill patients because they were inappropriately assigned lower triage categories.5 Therefore, a simple tool based on readily available parameters is needed to assist nurses in rapid and accurate triage.
The Modified Early Warning Score (MEWS), based on systolic blood pressure, pulse rate, temperature, respiration rate, and level of consciousness, is a simple bedside tool that nurses use in a busy clinical area. The MEWS calculation for emergency admissions may be useful in triage to identify some of those patients at risk of deterioration and need for more active intervention.6 The quick Sepsis Related Organ Failure Assessment (qSOFA) was recommended by the Third International Consensus Definitions Task Force.7 Its predictive validity for in-hospital mortality was found to be statistically greater than the SOFA and systemic inflammatory response syndrome (SIRS) among encounters with suspected infection outside of the ICU.8 The National Early Warning Score (NEWS), which was developed by the National Early Warning Score Development and Implementation Group (NEWSDIG) in the United Kingdom (UK) in 2012, is used by the National Health Service (NHS) to assess the deterioration of in-patient conditions and to predict in-patient mortality or ICU admissions.9 NEWS measures physiological parameters (systolic blood pressure, pulse rate, respiratory rate, temperature, oxygen saturation), level of consciousness (Alert, Voice, Pain, Unresponsive), and supplement of oxygen—which are all simple and easily accessible.9,10 NEWS is now popular in many countries since it shows a greater capacity than other early warning scores to discriminate patients at risk of the combined outcome of cardiac arrest, unanticipated ICU admission, or death within 24 hours.9,11 In recent years, NEWS use in specific populations of EDs (geriatric patients or patients with suspected sepsis or dyspnea) has also attracted attention.11–16 Some studies focused on the performance of NEWS in severity assessment and mortality prediction for category 2 (Emergency) and 3 (Urgent) patients.17,18 These studies show that NEWS is a beneficial ED tool. However, since the patient cohorts of some of these studies were rather small, the predictive effect of NEWS may not have been accurate. In addition, other studies focused on pre-selected sub-cohorts of patients, which limited the applicability of NEWS scores to other patients.
The aim of this study was to investigate the predictive effect of NEWS on outcomes, such as emergency mortality, ICU admission, and in-hospital mortality in unselected emergency patients (without any selected sub-cohorts). We also compared the prognostic value of NEWS alongside the commonly used MEWS and qSOFA. NEWS performance on sub-cohorts was also evaluated. We hypothesized that the NEWS obtained from triage can reliably predict ED patient outcomes.
Methods
Design and Setting
A retrospective cohort study was performed in the ED of a tertiary care center in Jinhua, China. This hospital has more than 2600 in-patient beds, 26 emergency beds, and 32 observation beds with more than 120,000 ED visits per year. The Ethics Committee of the hospital approved the study; informed consent was waived as the data were anonymous and routine care was not influenced. This study was conducted in accordance with the Declaration of Helsinki.
Study Population
Patients presented with the ED during the year-long study period (January 1, 2018 to December 31, 2018) were eligible to participate in this study if they met the following inclusion criteria: (1) Aged 18 years or older and (2) Emergency medicine patients. The exclusion criteria were as follows: (1) Patients attending the chest pain center, (2) Patients attending the stroke center, and (3) If the data manually inputted into the triage system by nurses (ie, temperature, respiratory rate, and pain score) was significantly abnormal but could not be verified. If the patients were lost to follow-up after ED discharge, they were removed from the study analysis. A total of 62,403 patients were included in this study.
Data Collection
In the sample hospital, the ED includes internal medicine, surgery, obstetrics and gynecology, and pediatrics. The internal medicine emergency department is divided into general emergency medicine, the chest pain center, and the stroke center. The patients come from the following three sources: (1) Delivery by ambulance (about 14 patients per day); (2) Transferred after outpatient treatment (2 to 4 patients per day); and (3) Self-admitted ED patients. All patients must undergo triage before they are admitted to a specific area. After the patient arrives at the ED, a triage nurse with more than three years of ED experience, measures the patient’s pulse rate, blood pressure, and oxygen saturation with a mobile monitor (Mindray V900), which automatically imports the data into the triage information system. The temperature, respiratory rate, consciousness (AVPU), pain score, and reason for visit are recorded manually by nursing staff. After all data is entered, the system automatically calculates the total score according to the calculation rules. The nursing staff determine the triage classification based on the score combined with medical history, symptoms, and vital signs. Only when all of the above items are completed can the patient’s information be entered into the doctor information system. After triage, patients are then classified into four levels, Class I (critical), Class II (emergency), Class III (urgent), and Class IV (sub-urgent), before being admitted to the resuscitation room or the emergency clinic. For example, if a patient’s NEWS≥5 points, combined with medical history, symptoms, and signs, the triage grade of the patient is most likely to be Class I. NEWS 3–4 points, or single score greater than 3 points for Class II; NEWS 2 points for Class III; NEWS 0–1 points for Class IV. The outcomes of patients, including discharge, hospitalization (general ward, ICU) or death, are uniformly summarized in the emergency patient information system and can be directly exported. The above procedures ensure the integrity of patient sources and the accuracy of data. The NEWS, MEWS, and qSOFA values were calculated for each vital signs obtained during the triage (Supplementary Table 1).
However, there was no record of supplemental oxygen in the triage system. In addition, the proportion of patients admitted by pre-hospital emergency care was not large, and the oxygen intake of these patients was mainly based on the patient’s chief complaint, rather than the strict principle of oxygen use. Due to this, the NEWS used in this study did not include the component of supplementary oxygen used. We categorized V, P, or U (Voice, Pain, or Unresponsive levels of consciousness) as having “altered mental status” when calculating qSOFA scores.19
Outcomes
The primary outcome of the study was ED mortality, and the secondary outcomes were observation in the ED; composite of ED mortality and ICU admission; and in-hospital mortality. ED mortality is defined as death in the ED, including death within 24 hours of being discharged from the ED. Emergency observation refers to a patient requiring being observed and treated in the emergency room, resuscitation room, or emergency observation room. The end point of emergency observation is when a patient is discharged or hospitalized from ED or dies in ED. Admission to the ICU includes patients who were admitted directly from the ED to the ICU or who were admitted to the ICU after the emergency intervention. Last, in-hospital mortality refers to death during hospitalization in this visit or death within 24 hours of hospital discharge.
Statistical Analysis
Data were summarized using descriptive statistics (ie, mean, standard deviation, median, and interquartile) when appropriate. To assess group differences, we used the Kruskal–Wallis test for continuous and skewed variables, and the Pearson’s χ2 test for categorical and binary variables.
Multivariate logistic regression analysis was performed to evaluate the associations between the NEWS, MEWS, and qSOFA, as well as those between other parameters with ED mortality. We developed different models with stepwise adjustments for potential confounders. The qSOFA and the main reason leading to ED admission were used as categorical variables. Results of the regression analyses were presented as odds ratios (ORs) with 95% confidence intervals (CIs). The predictive performances of NEWS, MEWS and qSOFA were compared by the area under the receiver operating characteristic curve (AUROC). Sensitivity, specificity, positive and negative predictive values, and Youden’s index were calculated for the different cut-off points. The optimal cut-off point for the different scores was determined by Youden’s index and this was used to assess the data. Calibration was measured by the Hosmer–Lemeshow goodness-of-fit test and expressed as a χ2-value and accessory p-value, which represent how outcome predictions resemble the observed outcomes. A two-tailed p-value of <0.05 was considered statistically significant. Further, we also used the AUROC to evaluate the ability of the NEWS to predict outcomes in different subgroups. Analyses were conducted using Statistical Package for the Social Science (SPSS) version 21.0, R version 3.4.3 (2017-11-30), and EmpowerStats (www.empowerstats.com) and MedCalc version 15 for Windows (MedCalc Software bvba).
Results
Basic Characteristics
There were 70,638 patients who visited the emergency medicine department during the study period (Figure 1). After exclusions, 62,403 patients remained eligible to participate in the study. Only 315 (0.5%) were excluded owing to incomplete data (Figure 1). Out of all those who were admitted, 6502 (10.4%) were under emergency observation, 106 (0.2%) experienced ED death, 638 (1.0%) were for ICU admission, and 324 (0.5%) suffered an in-hospital death (Table 1). Respiratory and digestive system, respectively, were the primary complaints. Since patients visiting the chest pain centers and stroke centers were excluded, the number of patients with cardiovascular and neurological disorders was decreased. Generally, the risk of all outcomes increased with patient age; men were more likely to suffer adverse consequences than women; and diseases of different systems had varying degrees of risk. Among the patients who died in the ED, there were 67 males (63.2%), 71.4 ± 14.3 years old, and 79 (74.52%) with NWS > 3 points. The time from visiting ED to death was 15.0 (6.0–27.0) hours. As for the main reason for referral, 35 (33.0%) was respiratory system, 23 (21.7%) digestive system, and 23 (21.7%) nervous system.
Figure 1.
Emergency department patient flow and the final cohort.
Table 1.
Baseline Characteristics of the Study Population
| Characteristic | Total Population (n =62,403) | Emergency Observation (n =6502) | ED Mortality (n =106) | Composite of ED Mortality and ICU Admission (n =744) | In-Hospital Mortality (n =324) |
|---|---|---|---|---|---|
| Age (years) | 47.4 ± 19.7 | 64.9 ± 17.7* | 71.4 ± 14.3* | 68.8 ± 16.4 * | 70.9 ± 15.7* |
| Sex (n (%)) | |||||
| Male | 29,177 (46.8) | 3981 (61.2)* | 67 (63.2)* | 471 (63.3)* | 210 (64.8)* |
| Female | 33,226 (53.2) | 2521 (38.8) | 39 (36.8) | 273 (36.7) | 114 (35.2) |
| Temperature (°C) | 36.8 ± 0.8 | 36.8 ± 0.9* | 36.4 ± 1.1* | 36.8 ± 1.1 | 36.6 ± 1.1* |
| Respiratory rate (bpm) | 19.2 ± 2.4 | 20.4 ± 4.9* | 23.9 ± 8.4* | 22.8 ± 7.4* | 22.4 ± 7.2* |
| Pulse rate (bpm) | 89.9 ± 19.0 | 94.9 ± 25.6* | 104.3 ± 28.4* | 103.4 ± 26.4* | 99.4 ± 24.5* |
| SBP (mmHg) | 129.7 ± 23.4 | 136.3 ± 31.8* | 120.5 ± 36.1* | 130.7 ± 33.6 | 128.9 ± 34.7 |
| DBP (mmHg) | 74.3 ± 14.6 | 76.6 ± 20.0* | 71.5 ± 26.2* | 74.1 ± 22.4* | 72.5 ± 21.9* |
| Oxygen saturation (%) | 97.5 ± 4.1 | 94.2 ± 9.1* | 88.0 ± 14.4* | 87.9 ± 14.2* | 90.4 ± 12.4* |
| Pain score | 0.0 (0.0–1.0) | 0.0 (0.0–0.0) | 0.0 (0.0–0.0) | 0.0 (0.0–0.0) | 0.0 (0.0–0.0) |
| AVPU (n (%)) | |||||
| Alert | 61,588 (98.7) | 5924 (91.1)* | 69 (65.1)* | 556 (74.7)* | 242 (74.7)* |
| Voice | 391 (0.6) | 207 (3.2) | 6 (5.7) | 61 (8.2) | 18 (5.6) |
| Pain | 272 (0.4) | 232 (3.6) | 19 (17.9) | 70 (9.4) | 40 (12.3) |
| Unresponsive | 152 (0.2) | 139 (2.1) | 12 (11.3) | 57 (7.7) | 24 (7.4) |
| Main reason for referral (n (%)) | |||||
| Circulatory system | 2012 (3.2) | 633 (9.7)* | 7 (6.6)* | 32 (4.3)* | 21 (6.5)* |
| Respiratory system | 26,427 (42.3) | 2388 (36.7) | 35 (33.0) | 359 (48.3) | 125 (38.6) |
| Digestive system | 15,048 (24.1) | 1169 (18.0) | 23 (21.7) | 89 (12.0) | 66 (20.4) |
| Endocrine System | 1505 (2.4) | 326 (5.0) | 6 (5.7) | 30 (4.0) | 12 (3.7) |
| Blood system | 405 (0.6) | 181 (2.8) | 2 (1.9) | 17 (2.3) | 8 (2.5) |
| Genitourinary system | 819 (1.3) | 78 (1.2) | 0 (0.0) | 23 (3.1) | 4 (1.2) |
| Nervous system | 5101 (8.2) | 973 (15.0) | 23 (21.7) | 110 (14.8) | 58 (17.9) |
| Others | 11,086 (17.8) | 754 (11.6) | 10 (9.4) | 84 (11.3) | 30 (9.3) |
| Emergency observation time | 11.0 (5.0–23.0) | 11.0 (5.0–23.0) | 15.0 (6.0–27.0) | 14.0 (6.0–27.0) * | 15.0 (6.0–26.5) * |
| MEWS | 2.0 (1.0–2.0) | 2.0 (1.0–4.0) * | 4.0 (2.2–6.0) * | 4.0 (2.0–5.0) * | 3.0 (2.0–5.0) * |
| NEWS | 1.0 (0.0–2.0) | 3.0 (1.0–5.0) * | 7.0 (3.2–10.0) * | 6.0 (3.0–8.0) * | 5.0 (2.0–8.0) * |
| QSOFA | 0.0 (0.0–0.0) | 0.0 (0.0–1.0) * | 1.0 (0.0–2.0) * | 1.0 (0.0–1.0) * | 1.0 (0.0–1.0) * |
Notes: *Compared to no-experience the outcome, P <0.05.
Abbreviations: ED, emergency department; ICU, intensive care unit; SBP, systolic blood pressure; DBP, diastolic blood pressure; ICU, intensive care unit; NEWS, National Early Warning Score; MEWS, Modified Early Warning Score; qSOFA, quick Sequential Organ Failure Assessment.
In the multivariate logistic regression analysis, we found a significant association between ED mortality and the NEWS (OR 1.5, 95% CI: 1.5 to 1.6, p < 0.001), MEWS (OR 1.8, 95% CI: 1.6 to 1.9, p < 0.001), and qSOFA (OR 126.4, 95% CI: 51.3 to 311.1, p < 0.001, qSOFA=3 compared to qSOFA=1). Additionally, age (OR 1.014, 95% CI: 1.029 to 1.053, p < 0.001) and gender (OR 0.7, 95% CI: 0.4 to 1.0, p < 0.001) were also associated with ED mortality (Table 2).
Table 2.
Multivariate Logistic Regression Analysis of Factors Independently Associated with ED Mortality
| Term | Unadjusted | Fully Adjusted | ||||
|---|---|---|---|---|---|---|
| Odds Ratio | 95% CI | P-value | Odds Ratio | 95% CI | P-value | |
| NEWS | 1.7 | 1.6–1.7 | <0.001 | 1.5 | 1.5–1.6 | <0.001 |
| MEWS | 2.1 | 1.9–2.2 | <0.001 | 1.8 | 1.6–1.9 | <0.001 |
| qSOFA | ||||||
| 0 | 1.0 | 1.0 | ||||
| 1 | 7.4 | 4.6–11.8 | <0.001 | 5.5 | 3.4–8.9 | <0.001 |
| 2 | 85.8 | 51.5–142.8 | <0.001 | 31.9 | 17.6–57.8 | <0.001 |
| 3 | 433.2 | 199.6–940.4 | <0.001 | 126.4 | 51.3–311.1 | <0.001 |
| Female | 0.5 | 0.3–0.8 | 0.001 | 0.7 | 0.4–1.0 | 0.038 |
| Age (years) | 1.1 | 1.1–1.1 | <0.001 | 1.1 | 1.0–1.1 | <0.001 |
| DBP (mmHg) | 1.0 | 1.0–1.0 | 0.050 | 1.0 | 1.0–1.0 | 0.150 |
| Main reason for referral | ||||||
| Circulatory system | 1.0 | 1.0 | ||||
| Respiratory system | 0.4 | (0.2–0.9) | 0.020 | 0.5 | (0.2–1.1) | 0.078 |
| Digestive system | 0.4 | (0.2–1.0) | 0.057 | 1.1 | (0.4–2.7) | 0.848 |
| Endocrine System | 1.1 | (0.4–3.4) | 0.806 | 1.1 | (0.3–3.5) | 0.911 |
| Blood system | 1.4 | (0.3–6.9) | 0.662 | 1.7 | (0.3–8.9) | 0.516 |
| Genitourinary system | 0.0 | (0.0 - -) | 0.971 | 0.0 | (0.0 - -) | 0.971 |
| Nervous system | 1.3 | (0.6–3.0) | 0.547 | 0.9 | (0.3–2.4) | 0.839 |
| Others | 0.3 | (0.1–0.7) | 0.006 | 0.5 | (0.2–1.5) | 0.232 |
Abbreviations: NEWS, National Early Warning Score; MEWS, Modified Early Warning Score; qSOFA, quick Sequential Organ Failure Assessment; DBP, diastolic blood pressure; NEWS, National Early Warning Score; CI, confidence intervals.
Prognostic Value of NEWS, MEWS and qSOFA
ED Mortality
Tables 3 and 4 and Figure 2 show the prognostic values and calibration of the NEWS, MEWS, and qSOFA for the prediction of ED mortality. The area under the curve (AUC) of the NEWS, MEWS, and qSOFA to predict ED mortality was 0.862 (95% CI: 0.859–0.865), 0.806 (95% CI: 0.803–0.809), and 0.792 (95% CI: 0.789–0.795), respectively. Using Youden’s J statistic, the optimal cut-off points for ED mortality were found to be: NEWS >3, MEWS >2, and qSOFA >0. The pairwise comparison of receiver operating characteristic (ROC) curves showed that the NEWS was better at predicting ED mortality than the MEWS and qSOFA. Calibration for the NEWS showed a χ2 = 0.00 and p = 1.0. The non-significant p-value indicates that the mortality rates between the observed and the predicted values were statistically equivalent.
Table 3.
Optimal Cut-Off Values, Sensitivity, Specificity, PPV, NPV, Positive Likelihood Ratio, Negative Likelihood Ratio and Corresponding 95% Confidence Intervals of Each Scale for Outcomes
| Optimal Cut-Off Values (95% CI) | Sensitivity (%) (95% CI) | Specificity (%) (95% CI) | PPV (%) (95% CI) | NPV (%) (95% CI) | Positive Likelihood Ratio (95% CI) | Negative Likelihood Ratio (95% CI) | |
|---|---|---|---|---|---|---|---|
| ED mortality | |||||||
| MEWS | >2 (>2 - >3) | 74.53 (65.1–82.5) | 78.63 (78.3–78.9) | 0.6 (0.5–0.7) | 99.9 (99.9–100.0) | 3.49 (3.1–3.9) | 0.32 (0.2–0.4) |
| NEWS | >3 (>2 - >3) | 74.53 (65.1–82.5) | 88.39 (88.1–88.6) | 1.1 (0.9–1.3) | 100.0 (99.9–100.0) | 6.42 (5.7–7.2) | 0.29 (0.2–0.4) |
| QSOFA | >0 (>0 - >0) | 66.98 (57.2–75.8) | 87.62 (87.4–87.9) | 0.9 (0.7–1.1) | 99.9 (99.9–100.0) | 5.41 (4.7–6.2) | 0.38 (0.3–0.5) |
| Emergency observation | |||||||
| MEWS | >2 (>2 - >2) | 43.36 (42.1–44.6) | 81.08 (80.8–81.4) | 21.0 (20.4–21.7) | 92.5 (92.2–92.7) | 2.29 (2.2–2.4) | 0.7 (0.7–0.7) |
| NEWS | >2 (>2 - >2) | 51.26 (50.0–52.5) | 81.49 (81.2–81.8) | 24.4 (23.6–25.1) | 93.5 (93.3–93.7) | 2.77 (2.7–2.9) | 0.6 (0.6–0.6) |
| QSOFA | >0 (>0 - >0) | 35.22 (34.1–36.4) | 90.17 (89.9–90.4) | 29.4 (28.4–30.4) | 92.3 (92.1–92.5) | 3.58 (3.4–3.7) | 0.72 (0.7–0.7) |
| Composite of ED mortality and ICU admission | |||||||
| MEWS | >2 (>2 – >2) | 67.34 (63.8–70.7) | 79.09 (78.8–79.4) | 3.7 (3.4–4.1) | 99.5 (99.4–99.6) | 3.22 (3.1–3.4) | 0.41 (0.4–0.5) |
| NEWS | >3 (>2 – >3) | 71.37 (68.0–74.6) | 89.00 (88.8–89.2) | 7.3 (6.7–7.9) | 99.6 (99.6–99.7) | 6.49 (6.2–6.8) | 0.32 (0.3–0.4) |
| QSOFA | >0 (>0 – >0) | 61.83 (58.2–65.3) | 88.12 (87.9–88.4) | 5.9 (5.4–6.5) | 99.5 (99.4–99.5) | 5.21 (4.9–5.5) | 0.43 (0.4–0.5) |
| In-hospital mortality | |||||||
| MEWS | >2 (>2 – >2) | 62.96 (57.5–68.2) | 78.75 (78.4–79.1) | 1.5 (1.3–1.7) | 99.8 (99.7–99.8) | 2.96 (2.7–3.2) | 0.47 (0.4–0.5) |
| NEWS | >3 (>2 – >3) | 62.04 (56.5–67.3) | 88.55 (88.3–88.8) | 2.7 (2.4–3.1) | 99.8 (99.7–99.8) | 5.42 (5.0–5.9) | 0.43 (0.4–0.5) |
| QSOFA | >0 (>0 – >0) | 56.17 (50.6–61.7) | 87.75 (87.5–88.0) | 2.3 (2.0–2.7) | 99.7 (99.7–99.8) | 4.59 (4.2–5.1) | 0.50 (0.4–0.6) |
Abbreviations: ED, emergency department; PPV, positive predictive value; NPV, negative predictive value; CI, confidence intervals; NEWS, National Early Warning Score; MEWS, Modified Early Warning Score; qSOFA, quick Sequential Organ Failure Assessment; ICU, intensive care unit.
Table 4.
AUC and Hosmer–Lemeshow of Fit Test for the Prediction of Outcomes
| Score | AUC (95% CI) | Hosmer–Lemeshow C Statistic (Chi-Square) | ||||||
|---|---|---|---|---|---|---|---|---|
| Emergency Observation | ED Mortality | Composite of ED Mortality and ICU Admission | In-Hospital Mortality | Emergency Observation | ED Mortality | Composite of ED Mortality and ICU Admission | In-Hospital Mortality | |
| NEWS | 0.691* (0.687–0.695) | 0.862* (0.859–0.865) | 0.859* (0.856–0.861) | 0.805* (0.802–0.808) | 0.00※ | 0.00※ | 0.00※ | 0.00※ |
| MEWS | 0.638 (0.634–0.642) | 0.806 (0.803–0.809) | 0.781 (0.778–0.785) | 0.739 (0.735–0.742) | 0.00 | 0.00 | 0.00 | 0.00 |
| qSOFA | 0.629 (0.626–0.633) | 0.792 (0.789–0.795) | 0.760 (0.757–0.764) | 0.730 (0.727–0.734) | 0.00 | 0.00 | 0.00 | 0.00 |
Notes: *The p-value is less than 0.05 between NEWS and MEWS or NEWS and qSOFA. ※The p-value is 1.0. The non-significant p-value indicates that the mortality rates between the observed and the predicted values were statistically equivalent.
Abbreviations: AUC, area under the curve; CI, confidence intervals; ED, emergency department; ICU, intensive care unit; NEWS, National Early Warning Score; MEWS, Modified Early Warning Score; qSOFA, quick Sequential Organ Failure Assessment.
Figure 2.
ROC curves of NEWS, MEWS and qSOFA for predicting outcomes. (A) ROC curves for ED mortality; (B) ROC curves for predicting emergency observation; (C) ROC curves for predicting composite of ED mortality and ICU admission; (D) ROC curves for predicting in-hospital mortality.
Abbreviations: NEWS, National Early Warning Score; MEWS, Modified Early Warning Score; qSOFA, quick Sequential Organ Failure Assessment.
Emergency Observation
The prognostic values of the NEWS, MEWS, and qSOFA for predicting emergency observation were also analyzed as shown in Tables 3 and 4 and Figure 2. The prognostic values for the prediction of emergency observation with the AUCs for NEWS, MEWS, and qSOFA were 0.691 (95% CI: 0.687–0.695), 0.638 (95% CI: 0.634–0.642), and 0.629 (95% CI: 0.626–0.633), respectively. The optimal cut-off points of the NEWS, MEWS and qSOFA were >2, >2, and >0, respectively. In addition, when the ROC curves were compared, the AUC of the NEWS was better than MEWS and qSOFA for predicting emergency observation in ED patients. The Hosmer–Lemeshow C statistic also showed that the NEWS has good calibration for predicting emergency observation.
Composite of ED Mortality and ICU Admission
The AUC of NEWS, MEWS, and qSOFA to predict the composite of ED mortality and ICU admission was 0.859 (95% CI: 0.856–0.861), 0.781 (95% CI: 0.778–0.785), and 0.760 (95% CI: 0.757–0.764), respectively (Tables 3 and 4; Figure 2). The optimal cut-off points were as follows: NEWS >3, MEWS >2, and qSOFA >0. NEWS also showed the best performance out of the three scores to predict the composite of ED mortality and ICU admission in ED patients, further proving NEWS’ good calibration.
In-Hospital Mortality
As for in-hospital mortality, the AUC for the NEWS, MEWS, and qSOFA was 0.805 (95% CI: 0.802–0.808), 0.739 (95% CI: 0.735–0.742), and 0.730 (95% CI: 0.727–0.734), respectively (Tables 3 and 4; Figure 2). The optimal cut-off points for in-hospital mortality of the NEWS, MEWS, and qSOFA were >2, >2, and >0, respectively. Compared to MEWS and qSOFA, NEWS had the best predictive effect.
Sensitivity Analyses
The prognostic values of the NEWS for the prediction of outcomes in subgroups are shown in Supplementary Table 2. For the primary outcome (ED mortality), the AUC of the NEWS was 0.854 (95% CI: 0.850–0.858) in male patients, 0.874 (95% CI: 0.871–0.878) in female patients, 0.864 (95% CI: 0.858–0.870) when patient age was ≥65 years, 0.800 (95% CI: 0.796–0.804) when patient age was <65 years, and 0.899 (95% CI: 0.896–0.903) in patients with respiratory system disorders. Additionally, the NEWS predictions of all outcomes appeared to be better for patients with respiratory diseases than for the total number of patients.
Discussion
To the best of our knowledge, this is the first retrospective study conducted to evaluate the performance of NEWS for unselected emergency medicine patients in a predominantly Chinese population. We found that NEWS correlates significantly with patient outcomes, including ED mortality, ICU admission, and in-hospital mortality. Irrespective of different outcomes, NEWS was the most accurate tool for prediction of outcomes when compared with the MEWS and qSOFA. The results showed that NEWS was the model with the highest agreement between the expected and observed outcomes. NEWS also performed well in pre-selected sub-cohorts of patients, especially those with respiratory system conditions.
There are several scoring systems that can be used to identify patients at risk of deterioration and predict clinical outcomes. The AIMS65 score, and Glasgow Blatchford score for example, can be used to assess severity in patients with acute gastrointestinal bleeding.20–22 Systemic inflammatory response syndrome (SIRS) is often used to predict infection or septic shock.23–27 However, many of these scores include laboratory results and are used primarily for specific patients, limiting their value in emergency triage. However, general early warning scores, such as the MEWS and NEWS, are based only on readily available parameters, and thus have a good range of applicability.
An obvious advantage of NEWS, MEWS, and qSOFA is that they do not require any biological tests, which together with their good performance make their ED use possible. In recent years, several studies have focused on their value in predicting the severity and prognosis of emergency patients. MEWS is used as one of the standard bases for triage classification judgment.28 NEWS on ED arrival performs well in recognizing severity and prognosis of patients with severe sepsis and septic shock, patients with respiratory distress, elderly patients, and unselected emergency patients.10,12,17,29 NEWS can serve as a useful tool for managing patients in a complex ED environment, enhancing patient care quality.18 Compared with MEWS, qSOFA, and other warning scores, NEWS is generally more accurate in predicting admission and mortality for all patient types, showing that NEWS may have greater ED application value.10,12,15,17,18,30
However, these studies contain several limitations. The patient cohorts of these studies were rather small, from several hundreds to thousands;14,17,30,31 thus, the prediction effect for NEWS may not have been stable. Furthermore, most focused on a pre-selected sub-cohort of patients, such as patients suspected of sepsis,14,15,30 elderly patients,12 or category 2 (Emergency) and 3 (Urgent) patients,13 which limits the applicability of NEWS scores to other patients. However, it is impossible to divide patients in sub-cohorts ahead of emergency triage. With the exception of one study conducted in Hong Kong,17 none of the aforementioned studies focused on a predominantly Chinese population.
The participants for the present study were continuously recruited during the entire year and were from both urban and rural areas to minimize selection bias.32 In order to guarantee the representativeness of the sample and improve the stability of results, more than 60,000 participants were included. At the same time, all emergency patients needed to go through triage before they could see a doctor, which ensured the integrity of the patient queue. In the present study, only a small number of patients (315) were excluded owing to abnormal data or were lost to follow-up after discharge. Moreover, this study was aimed at unselected emergency department patients, which is in line with the actual situation of emergency triage. We chose the particular outcomes over longer term mortality (90 days) because, for emergency care, it is essential to predict both the current level of severity and the immediate risk of death to provide more active and effective interventions. Moreover, the longer the follow-up time, the higher the incidence of other influential factors affecting the prognosis of patients.
Our results showed that NEWS had a high predictive effect for emergency death. The predictive effect of ICU admission and in-hospital mortality was also good. Whether patients need to be observed in the ED depends not only on the disease severity but also on many other factors, such as the need for further evaluation or diagnosis, the need to wait for crowded hospital beds, etc., so the prediction effect of NEWS on emergency observation was only near moderate. The optimal cut-off points of NEWS for ED mortality and in-hospital mortality were >3, which differ from Brinks et al’s research findings.15 This may be because the subjects were different, and the NEWS of our research did not include oxygen supplementation. In general, all acute deteriorating patients would not be checked and screened using the oxygen supplement category. If patients have oxygen supplementation, they were assigned a score of 2 on the NEWS score. Our results showed that NEWS can serve as a useful tool to help triage nurse’s grade patients more quickly, objectively, and accurately. Based on the triage level, then medical staff can determine the patient’s zoning, emergency response time, and implementation of first aid measures. For example, for patients with Class I (such as cardiac, respiratory arrest, heart failure, respiratory failure, etc.), emergency staff should immediately take emergency measures such as cardiopulmonary resuscitation, respiratory support, circulatory support, various tests, etc., to prevent acute deterioration.
The Hosmer–Lemeshow test showed that the results between observed and predicted values were statistically equivalent, which was consistent with Brinks et al’s results.15 The results of multiple regression analysis showed that in addition to warning scores, age and gender were independent risk factors for emergency death. The effect of gender on mortality has been a concern.33 Studies have shown that males are significantly associated with a higher risk of death in critically ill or elderly patients.34,35 In our cohort, the male patients were older than female patients (49.2 ± 20.0 VS 45.8 ± 19.3). However, the male patients who died in the emergency department were younger than female patients (69.6 ± 14.0 VS 74.6 ± 14.5). Males also had higher BP (132.9 ± 22.7 VS 126.9 ± 23.6). In addition, the incidence of circulatory disease was higher among males (1146, 3.9% VS 866, 2.6%). These factors may lead to gender being an independent risk factor for emergency death. The predictive effect of NEWS on some specific groups was also studied, and the results were stable. Therefore, we conducted studies on sub-cohorts of different ages and genders, and the results showed that NEWS still had a good predictive effect on outcomes. At the same time, it was found that NEWS had a better predictive effect for patients with respiratory diseases, possibly because the score increased the respiratory-related parameter, that is, oxygen saturation. In the research cohort, NEWS predictive performance on all outcomes was superior compared to MEWS and qSOFA, which is consistent with other studies.18,24,27,29,31
Limitations
Our study had several limitations. First, this was a single-center investigation in a tertiary care center hospital in China; therefore, the results may not be generalizable to other settings and populations. Second, for ED mortality, we only followed critically ill patients who were discharged from the ED, owing to the large volume of patients. Therefore, a small percentage of “mild” patients who died within 24 hours of discharge may be missing. Third, because there are many influencing factors, there may be some selection bias in the definition of emergency observation. In addition, the NEWS we studied did not include oxygen supplementation, so its results may differ from the standard NEWS. It should be noted that excluding the oxygen parameters only weakens the predictive effect. Therefore, the value of the standard NEWS should be better than the NEWS in this study. Moreover, because of differing intervention processes, the patients admitted to the chest pain center and stroke center were excluded. Hence, the predictive value of the NEWS for these patients needs to be examined further. Finally, we did not stratify NEWS in more detail and compare it with the triage categories. Thus, further studies are needed to accurately determine the role of the NEWS in reducing the under-triage or over-triage rates of patients.
Conclusion
The study found that NEWS calculated according to the data obtained during the triage was significantly correlated with the prognosis of emergency medicine patients, including ED mortality, emergency observation, ICU admission, and in-hospital mortality. The study revealed that NEWS was the most accurate tool for predicting adverse outcomes when compared with MEWS and qSOFA. These findings have important implications for optimizing the triage process in EDs of general hospitals. Through the use of NEWS, the ED staff have a better indication of which patients are at greater risk. Moreover, NEWS can serve as a useful tool to help triage nurse’s grade patients more quickly, objectively, and accurately, and thus manage patients more effectively in a complex ED environment, enhancing patient care quality.
Future studies should focus on the value of NEWS in multi-center EDs. At the same time, the predictive value of NEWS for emergency patients admitted to chest pain centers or stroke centers, as well as its effect on improving triage accuracy, should be studied.
Funding Statement
No funds, grants, or other support were received.
Data Sharing Statement
The datasets generated during and/or analysed during the current study are available from the corresponding author on request.
Patient Consent and Ethical Approval
The study was approved by the Ethics Committee of Jinhua Municipal Central Hospital (No.2020-18). The data were anonymous, and the requirement for informed consent was therefore waived.
Disclosure
The authors declare they have no competing or conflicting financial interests.
References
- 1.Kwon JM, Lee Y, Lee Y, et al. Validation of deep-learning-based triage and acuity score using a large national dataset. PLoS One. 2018;13(10):e0205836. doi: 10.1371/journal.pone.0205836 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Sharma S, Rafferty AM, Boiko O. The role and contribution of nurses to patient flow management in acute hospitals: a systematic review of mixed methods studies. Int J Nurs Stud. 2020;110:103709. doi: 10.1016/j.ijnurstu.2020.103709 [DOI] [PubMed] [Google Scholar]
- 3.Farrohknia N, Castrén M, Ehrenberg A, et al. Emergency department triage scales and their components: a systematic review of the scientific evidence. Scand J Trauma Resusc Emerg Med. 2011;19(1):42. doi: 10.1186/1757-7241-19-42 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Welch SJ, Davidson SJ. The performance limits of traditional triage. Ann Emerg Med. 2011;58(2):143–144. doi: 10.1016/j.annemergmed.2011.04.017 [DOI] [PubMed] [Google Scholar]
- 5.Mullan PC, Torrey SB, Chandra A, et al. Reduced overtriage and undertriage with a new triage system in an urban accident and emergency department in Botswana: a cohort study. Emerg Med J. 2014;31(5):356–360. doi: 10.1136/emermed-2012-201900 [DOI] [PubMed] [Google Scholar]
- 6.Subbe CP, Kruger M, Rutherford P, et al. Validation of a modified early warning score in medical admissions. QJM. 2001;94(10):521–526. doi: 10.1093/qjmed/94.10.521 [DOI] [PubMed] [Google Scholar]
- 7.Singer M, Deutschman CS, Seymour CW, et al. The third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA. 2016;315(8):801–810. doi: 10.1001/jama.2016.0287 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA. 2016;315(8):762–774. doi: 10.1001/jama.2016.0288 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Smith GB, Prytherch DR, Meredith P, et al. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation. 2013;84(4):465–470. doi: 10.1016/j.resuscitation.2012.12.016 [DOI] [PubMed] [Google Scholar]
- 10.Bilben B, Grandal L, Søvik S. National Early Warning Score (NEWS) as an emergency department predictor of disease severity and 90-day survival in the acutely dyspneic patient - a prospective observational study. Scand J Trauma Resusc Emerg Med. 2016;24(1):80. doi: 10.1186/s13049-016-0273-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.National Early Warning Score (NEWS). Standardising the assessment of acute illness severity in the NHS. Report of a working party. London: Royal College of Physicians; 2012. [Google Scholar]
- 12.Mitsunaga T, Hasegawa I, Uzura M, et al. Comparison of the National Early Warning Score (NEWS) and the Modified Early Warning Score (MEWS) for predicting admission and in-hospital mortality in elderly patients in the pre-hospital setting and in the emergency department. PeerJ. 2019;7:e6947. doi: 10.7717/peerj.6947 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Dundar ZD, Kocak S, Girisgin AS. Lactate and NEWS-L are fair predictors of mortality in critically ill geriatric emergency department patients. Am J Emerg Med. 2020;38(2):217–221. doi: 10.1016/j.ajem.2019.02.006 [DOI] [PubMed] [Google Scholar]
- 14.Goulden R, Hoyle MC, Monis J, et al. qSOFA, SIRS and NEWS for predicting inhospital mortality and ICU admission in emergency admissions treated as sepsis. Emerg Med J. 2018;35(6):345–349. doi: 10.1136/emermed-2017-207120 [DOI] [PubMed] [Google Scholar]
- 15.Brink A, Alsma J, Verdonschot RJCG, et al. Predicting mortality in patients with suspected sepsis at the emergency department; A retrospective cohort study comparing qSOFA, SIRS and National Early Warning Score. PLoS One. 2019;14(1):e0211133. doi: 10.1371/journal.pone.0211133 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Churpek MM, Snyder A, Han X, et al. Quick sepsis-related organ failure assessment, systemic inflammatory response syndrome, and early warning scores for detecting clinical deterioration in infected patients outside the intensive care unit. Am J Respir Crit Care Med. 2017;195(7):906–911. doi: 10.1164/rccm.201604-0854OC [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Graham CA, Leung LY, Lo RSL, et al. NEWS and qSIRS superior to qSOFA in the prediction of 30-day mortality in emergency department patients in Hong Kong. Ann Med. 2020;52(7):403–412. doi: 10.1080/07853890.2020.1782462 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Alam N, Vegting IL, Houben E, et al. Exploring the performance of the National Early Warning Score (NEWS) in a European emergency department. Resuscitation. 2015;90:111–115. doi: 10.1016/j.resuscitation.2015.02.011 [DOI] [PubMed] [Google Scholar]
- 19.Jiang X, Jiang P, Mao Y. Performance of Modified Early Warning Score (MEWS) and Circulation, Respiration, Abdomen, Motor, and Speech (CRAMS) score in trauma severity and in-hospital mortality prediction in multiple trauma patients: a comparison study. PeerJ. 2019;7:e7227. doi: 10.7717/peerj.7227 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Abougergi MS, Charpentier JP, Bethea E, et al. A prospective, multicenter study of the AIMS65 score compared with the Glasgow-Blatchford score in predicting upper gastrointestinal hemorrhage outcomes. J Clin Gastroenterol. 2016;50(6):464–469. doi: 10.1097/MCG.0000000000000395 [DOI] [PubMed] [Google Scholar]
- 21.Kim D, Jo S, Lee JB, et al. Comparison of the national early warning score+lactate score with the pre-endoscopic Rockall, Glasgow-Blatchford, and AIMS65 scores in patients with upper gastrointestinal bleeding. Clin Exp Emerg Med. 2018;5(4):219–229. doi: 10.15441/ceem.17.268 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Stanley AJ, Laine L, Dalton HR, et al. Comparison of risk scoring systems for patients presenting with upper gastrointestinal bleeding: international multicentre prospective study. BMJ. 2017;356:i6432. doi: 10.1136/bmj.i6432 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Gando S, Shiraishi A, Abe T, et al. The SIRS criteria have better performance for predicting infection than qSOFA scores in the emergency department. Sci Rep. 2020;10(1):8095. doi: 10.1038/s41598-020-64314-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Lind ML, Phipps AI, Mooney S, et al. Predictive value of 3 clinical criteria for sepsis (quick sequential organ failure assessment, systemic inflammatory response syndrome, and national early warning score) with respect to short-term mortality in allogeneic hematopoietic cell transplant recipients with suspected infections. Clin Infect Dis. 2021;72(7):1220–1229. doi: 10.1093/cid/ciaa214 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Peng Y, Zhang W, Xu Y, et al. Performance of SOFA, qSOFA and SIRS to predict septic shock after percutaneous nephrolithotomy. World J Urol. 2021;39(2):501–510. doi: 10.1007/s00345-020-03183-2 [DOI] [PubMed] [Google Scholar]
- 26.Green SL, Smith MTD, Cairns C, et al. The combined SIRS+ qSOFA (qSIRS) score is more accurate than qSOFA alone in predicting mortality in patients with surgical sepsis in an LMIC emergency department. World J Surg. 2020;44(1):21–29. doi: 10.1007/s00268-019-05181-x [DOI] [PubMed] [Google Scholar]
- 27.Usman OA, Usman AA, Ward MA. Comparison of SIRS, qSOFA, and NEWS for the early identification of sepsis in the emergency department. Am J Emerg Med. 2019;37(8):1490–1497. doi: 10.1016/j.ajem.2018.10.058 [DOI] [PubMed] [Google Scholar]
- 28.Jingfen J, Shuihong C, Zhang MAO, et al. Construction of grading triage criteria for emergency pre-examination. Chin J Emerg Med. 2016;25(4):527–531. doi: 10.3760/cma.j.issn.1671-0282.2016.04.030 [DOI] [Google Scholar]
- 29.Keep JW, Messmer AS, Sladden R, et al. National early warning score at emergency department triage may allow earlier identification of patients with severe sepsis and septic shock: a retrospective observational study. Emerg Med J. 2016;33(1):37–41. doi: 10.1136/emermed-2014-204465 [DOI] [PubMed] [Google Scholar]
- 30.Nieves Ortega R, Rosin C, Bingisser R, et al. Clinical scores and formal triage for screening of sepsis and adverse outcomes on arrival in an emergency department all-comer cohort. J Emerg Med. 2019;57(4):453–460. doi: 10.1016/j.jemermed.2019.06.036 [DOI] [PubMed] [Google Scholar]
- 31.Kivipuro M, Tirkkonen J, Kontula T, et al. National early warning score (NEWS) in a Finnish multidisciplinary emergency department and direct vs. late admission to intensive care. Resuscitation. 2018;128:164–169. doi: 10.1016/j.resuscitation.2018.05.020 [DOI] [PubMed] [Google Scholar]
- 32.Medina-Lozano E, Martín-Rodríguez F, Castro-Villamor MÁ, et al. Accuracy of early warning scores for predicting serious adverse events in pre-hospital traumatic injury. Injury. 2020;51(7):1554–1560. doi: 10.1016/j.injury.2020.04.042 [DOI] [PubMed] [Google Scholar]
- 33.Arrich J, Sterz F, Fleischhackl R, et al. Gender modifies the influence of age on outcome after successfully resuscitated cardiac arrest: a retrospective cohort study. Medicine. 2006;85(5):288–294. doi: 10.1097/01.md.0000236954.72342.20 [DOI] [PubMed] [Google Scholar]
- 34.Moran JL, Bristow P, Solomon PJ, et al. Mortality and length-of-stay outcomes, 1993–2003, in the binational Australian and New Zealand intensive care adult patient database. Crit Care Med. 2008;36(1):46–61. doi: 10.1097/01.CCM.0000295313.08084.58 [DOI] [PubMed] [Google Scholar]
- 35.Chalfin DB, Trzeciak S, Likourezos A, et al. Impact of delayed transfer of critically ill patients from the emergency department to the intensive care unit. Crit Care Med. 2007;35(6):1477–1483. doi: 10.1097/01.CCM.0000266585.74905.5A [DOI] [PubMed] [Google Scholar]