Abstract
The National Early Warning Score (NEWS)-2 is an early warning scale that is used in emergency departments to identify patients at risk of clinical deterioration and to help establish rapid and timely management. The objective of this study was to determine the validity and prediction of mortality using the NEWS2 scale for adults in the emergency department of a tertiary clinic in Colombia.
A prospective observational study was conducted between August 2018 and June 2019 at the Universidad de La Sabana Clinic.
The nursing staff in the triage classified the patients admitted to the emergency room according to Emergency Severity Index and NEWS2. Demographic data, physiological variables, admission diagnosis, mortality outcome, and comorbidities were extracted.
Three thousand nine hundred eighty-six patients were included in the study. Ninety-two (2%) patients required intensive care unit management, with a mean NEWS2 score of 7. A total of 158 patients died in hospital, of which 63 were women (40%). Of these 65 patients required intensive care unit management. The receiver operating characteristic curve for NEWS2 had an area of 0.90 (CI 95%: 0.87–0.92). A classification and score equivalency analysis was performed between triage and the NEWS2 scale in terms of mortality. Of the patients classified as triage I, 32.3% died, and those who obtained a NEWS2 score greater than or equal to 10 had a mortality of 38.6%.
Among our population, NEWS2 was not inferior in its area under the receiver operating characteristic curve when predicting mortality than triage, and the cutoff point for NEWS2 to predict in-hospital mortality was higher.
Keywords: adult, clinical deterioration, early warning systems, emergency service in hospital, hospital mortality, National Early Warning Score 2, TRIAGE, triage
1. Introduction
The severity and risk of the deterioration of patients admitted to emergency services should be assessed by a health professional with the ability to detect the health condition early, efficiently, and effectively and, through clinical criteria, classify the level of care required.[1] Classifying patients into an appropriate level of care will allow the timely treatment of complications derived from their clinical conditions and the creation of multivariate assessment methods. When there is no adequate assessment or classification, the response to the deterioration of a condition is difficult, except for emergencies that must be addressed at any level.[2]
Thus, early warning scales are used worldwide. In this way, the triage classification system has facilitated the prioritization of patients; for this, several systems have been implemented worldwide, including the Australian Triage Scale,[3] Canada Emergency Department Triage,[4] the Manchester Triage System,[5–7] the Emergency Severity Index (ESI),[8,9] and the Andorran Triage Model,[2] which have shown favorable results in the targeting and care times of patients. However, the use of different scales to assess patients can be detrimental, as not all staff are familiar with the necessary language, which could increase the risk of missing critical signs in the initial assessments of patients.[10]Table 1 shows the differences in the categories, care times, and countries in which the different patient assessment scales are used.
Table 1.
Disease severity scales such as Triage, National Early Warning Score (NEWS)-2, Sequential Organ Failure,[11] Acute Physiology And Chronic Health Evaluation II,[12] the Hamilton Early Warning Score,[13] the Modified Early Warning Score,[14] and the Simplified Acute Physiology Score[12] have been extensively studied among acutely critical patients such that in clinical practice, they have been gaining popularity in recent years. Moreover, when they have been used, a statistically significant difference has been demonstrated since the activation of these scales makes it possible to identify the clinical deterioration of patients, the demand for more hospital resources, and even predict mortality.[15–17]
In emergency services worldwide, a triage system is used, which includes several models consisting of 3 or 5 levels of classification depending on the scale adopted by each institution. This requires the nursing staff to have expertise and evaluate not only vital signs but also the clinical condition to determine the level of care, which makes triage a subjective scale.[18] Studies have shown that triage assessment has better validity and reliability when performed with 5-level scales instead of 3 (P < .001). Even so, it has been shown that this type of assessment for patients older than 65 years has a sensitivity as low as up to 42%, although the specificity reaches 99.2%.[19] On the other hand, for 3-level classification, triage presents lower sensitivity corresponding to 60.2%.[20,21] Despite this variability in sensitivity, the ESI is currently the best triage model to predict hospital mortality in terms of the 5-level scales.[2,22–25] This indicates a high variability in sensitivity, regardless of whether the patient is evaluated at 3 or 5 levels, with an increased risk of overlooking patients featuring a risk of deterioration as high as 50%.
In the United Kingdom, the NEWS2 assessment scale has been implemented in recent years as a tool for national use to standardize the recording, scoring, and interpretation of results in the face of physiological parameters routinely measured in the emergency room, especially those that require transfer to an intensive care unit (ICU), which allows objective analysis of patients with acute diseases and a risk of clinical deterioration at any level of care.[26–30] This scale has shown favorable results and is not inferior to triage scales. Given its ability to predict deterioration and mortality objectively based on easily measurable physiological variables at any level of care, we considered the use of the NEWS2 scale relevant for our study. Likewise, data on their performance compared to triage are scarce.
The objective of this study is to determine the reproducibility and validity of the NEWS2 scale in the emergency department for the prediction of in-hospital mortality.
2. Materials and methods
2.1. Study design
Under the objectives set out in this study, the methodology was carried out in 2 phases: Phase I, determination of the reproducibility inter- and intra-observer of the NEWS2, and Phase II, the validity and prediction of mortality using the NEWS 2 scale for adults in the emergency department.
For the development of both phases, a prospective observational study was carried out between August 2018 and June 2019 on patients older than 18 years who were admitted to the emergency department of the Universidad de la Sabana Clinic (Chía, Cundinamarca, Colombia), where the nursing triage staff classifies each patient to the necessary level of care, depending on the ESI. Excluded from the study who did not have a record of diagnoses of admission to the emergency room, as well as patients who presented errors when filling in the NEWS scale (incomplete data).
During patient care, demographic data were recorded, such as age, sex, living or dead status, date and time of admission and discharge, vital signs, admission and discharge diagnosis, ICU requirements, and comorbidities such as arterial hypertension, type 2 diabetes mellitus, chronic obstructive pulmonary disease, chronic kidney disease, and cerebrovascular disease. The patients were assessed and classified according to the triage and NEWS2 scale on 3 occasions (twice by the same nurse); these assessments were performed at different times but with time intervals no greater than 1 hour. The outcome variable was mortality.
The triage system was divided into 5 levels, where level 1 represents a vital urgency that requires immediate medical attention, level 4 corresponds to patients who do not require a medical evaluation in the emergency room but do require a priority appointment for evaluation within 48 hours, and level 5 is directed toward non-priority external consultation[26] (Table 1). Likewise, staff are trained to evaluate patients with the NEWS2 scale, which is based on physiological variables such as blood pressure, heart rate, respiratory rate, temperature, oxygen saturation, and level of consciousness, providing a score of 0 to 20. The patients are classified into 1 of 2 categories: 1 to 4 points indicates a low risk of clinical deterioration, 5 to 6 indicates a moderate risk of clinical deterioration, and greater than or equal to 7 indicates a high risk of clinical deterioration requiring attention immediate[5,10] (Table 1).
The sample size was calculated according to the data of the Keep study,[33] where a sensitivity of 92.6% and specificity of 77% are reported for a cutoff point of 3, for an expected mortality of 10%, confidence level of 95%, and precision of 5% required a minimum of 562 subjects. For reproducibility, an intra-class correlation coefficient (ICC) value of 0.9 was considered, and amplitude of 10%, number of observers 2, requiring a minimum of 57 subjects for this analysis.
2.2. Data collection
For the application of the scale, a questionnaire electronic with the variables of the NEWS2 scale and a specific score for each variable. The nursing staff filled out each questionnaire for each patient admitted to the emergency department during the triage assessment. The calculation of the final score using the NEWS2 scale was automatically sent to an Excel database designed for the study. In order to reduce the selection bias, the admission of patients with all degrees of severity was sought, and to reduce the information bias, the data were obtained by trained personnel and verified by the research team the clinical records and double use was used for transcription fingering
2.3. Statistical analysis
The database was collected in a Microsoft Excel spreadsheet and subsequently analyzed with the statistical program IBM SPSS statistics 25. The qualitative variables were summarized as frequencies and percentages, using the average quantitative variables and standard deviation to determine if the data were normally distributed or the median and interquartile range if the data were not normally distributed. A description was made of the clinical characteristics of the studied population in terms of sex, age, vital signs and mortality outcomes, ICU requirements, hospitalization, or readmission in the first 48 hours. Additionally, a bivariate analysis was performed between the positive outcomes with a chi-square test for the qualitative variables, a Student t distribution test to determine the normal distribution, and a non-parametric Mann–Whitney U test if the distribution was non-normal. Reproducibility was evaluated with the intra-observer ICC with the data taken from the same evaluator and inter-observer with the data obtained from the scores by 2 different evaluators. Subsequently, the area under the receiver operating characteristic (ROC) curve for the quantitative NEWS2 score and the triage with mortality outcomes and ICU admission was used to calculate the confidence interval (CI) and the P value. A comparison of the areas under the receiver operating curve was made, using DeLong method. On the other hand, the sensitivity and specificity analyses were performed for the cutoff point with the highest Youden index (YI) and the best discrimination of the area under the ROC curve, calculating sensitivity, specificity, and the positive and negative likelihood ratio. Finally, a table was made to approximate the equivalences of triage levels 1, 2, 3, 4, and 5 to the different scores of the YI, considering significance at P < .005.
2.4. Ethical considerations
The project was approved by the ethics committee of the Universidad de La Sabana Clinic (Acta 17, Agosto 2019). This study was conducted as per the Declaration of Helsinki. According to the results, the decision to inform the volunteers about the details of the findings of this study was made by the research group in a prior meeting.
3. Results
During the study, 4984 patients admitted to the emergency room were registered in the database. Of these, 84 patients were excluded because they were under 18 years of age, 315 patients were excluded because they had been readmitted to the emergency department, 56 were excluded because they had no record of diagnoses with an admission to the emergency room, and 543 produced errors when filling in the NEWS2 scale (Fig. 1). The final number of patients in the study was 3986. File 1, Supplemental Digital Content shows the summary of missing values.
Figure 1.
Flowchart of the selection of patients who entered the study.
Table 2 shows the characteristics of the population that participated in the study. In total, 2230 women (56%) and 1756 men (44%) were admitted to the emergency room, with a mean age of 44.6 years. The most frequent comorbidities were arterial hypertension (n = 777) and type 2 diabetes mellitus (n = 265), followed by chronic obstructive pulmonary disease (n = 129), chronic kidney disease (n = 99), and cerebrovascular disease (n = 69). Regarding admission diagnoses, 3172 patients were admitted for medical illness. Among these, 954 patients had an infectious medical diagnosis, followed by 625 with a traumatic disease, and 321 had a surgical disease. It is also important to highlight the patients who were admitted with simultaneous diagnoses. There were 92 (2%) patients admitted to the ICU, with a mean NEWS2 of 7.
Table 2.
General population characteristics.
| Characteristics | General population n = 3986 | Alive n = 3828 | Deaths n = 158 | P value |
| Sex in (%) | ||||
| Female | 2230 (56) | 2167 (57) | 63 (40) | <.001 |
| Male | 1756 (44) | 1661 (43) | 95 (60) | |
| Age × (SD) | ||||
| 44.6 (19.5) | 43.4 (18.6) | 75.2 (14.6) | <.001 | |
| Vital signs (SD) | ||||
| HR | 84.3 (17.8) | 84.3 (16.7) | 83 (34.6) | .802 |
| RR | 19.1 (2.5) | 19,1 (2.0) | 19.8 (7.8) | .252 |
| SBP | 124.9 (22.5) | 125.5 (20.4) | 108 (49.2) | <.001 |
| T (°C) | 36.4 (1.91) | 36.5 (1.0) | 34.5 (8.1) | .003 |
| SAT O2 | 92.5 (6.4) | 93.1 (3.4) | 79.5 (24.1) | <.001 |
| Supplemental oxygen | 119 (3) | 60 (2) | 59 (2) | <.001 |
| State of consciousness n (%) | 123 (3) | 45 (1) | 78 (49) | <.001 |
| Comorbidities n (%) | ||||
| HA | 777 (19) | 689 (18) | 88 (56) | <.001 |
| DM type 2 | 265 (7) | 238 (6) | 27 (17) | <.001 |
| COPD | 129 (3) | 89 (2) | 40 (25) | <.001 |
| CKD | 99 (2) | 78 (2) | 21 (13) | <.001 |
| CD | 69 (2) | 49 (1) | 20 (13) | <.001 |
| Admission diagnosis n (%) | ||||
| Traumatic illness | 625 (16) | 621 (16) | 4 (3) | <.001 |
| Medical illness | 3172 (80) | 3038 (79) | 134 (85) | .097 |
| Infectious medical pathology | 954 (24) | 919 (24) | 35 (22) | .31 |
| Surgical disease | 321 (8) | 296 (8) | 25 (16) | <.001 |
| Admission to ICU n (%) | 92 (2) | 27 (1) | 65 (41) | |
| NEWS2 score of patients admitted to ICU × (SD) | 7 (4,3) | 5 (3) | 8 (4,5) | |
CD = cerebrovascular disease, CKD = chronic kidney disease, COPD = chronic obstructive pulmonary disease, DM type 2 = type 2 diabetes mellitus, HA = arterial hypertension, HR = heart rate, ICU = intensive care unit, NEWS = National Early Warning Score, RR = respiratory rate, SAT O2 = oxygen saturation, SBP = systolic blood pressure, SD = Standard Deviation, T = temperature (centigrade).
A total of 158 patients died in hospital, of which 95 were men (60%) with a mean age of death at 75.2 years. Of these, 65 patients required ICU management according to factors related to vital signs upon admission to the emergency room, with an average heart rate of 83 beats per minute, a respiratory rate of 19 breaths per minute, systolic blood pressure of 108 mm Hg, a temperature of 34.5°C, and oxygen saturation of 79.5%. Moreover, 78 patients (49%) were admitted with an altered state of consciousness, including delirium, drowsiness, stupor, and a superficial or deep coma. The most frequent comorbidities were arterial hypertension (56%), chronic obstructive pulmonary disease (25%), type 2 diabetes mellitus (17%), chronic kidney disease (13%), and cerebrovascular disease (13%).
The area under the ROC curve for triage and NEWS2 in predicting in-hospital mortality due to any cause of admission to the emergency room is shown in Figure 2 and Table 3. NEWS2 had an area of 0.90 (95% CI: 0.87–0.92), and triage had an area of 0.89 (95% CI: 0.87–0.91). DeLong method P value = .544.
Figure 2.
The area under the receiver operating characteristic (ROC) curve for (A) NEWS2 and (B) TRIAGE. NEWS = National Early Warning Score.
Table 3.
Data obtained for the NEWS2 scale in comparison with triage.
| ROC | CI 95% | P value | |
| NEWS2 | |||
| Mortality | 0.90 | 0.87–0.92 | <.001 |
| Hospitalization | 0.67 | 0.76–0.86 | <.001 |
| ICU | 0.81 | 0.63–0.69 | <.001 |
| Triage | |||
| Mortality | 0.89 | 0.87–0.91 | <.001 |
CI = confidence interval, ICU = intensive care unit, NEWS = National Early Warning Score, ROC = receiver operating characteristic.
In the same way, the precision of the area under the ROC curve was calculated to predict mortality. For the most exact point of NEWS2, the score was greater than or equal to 7, presenting a sensitivity of 66%, specificity of 96%, a positive likelihood ratio of 17.5, a negative likelihood ratio of 0.34, and a YI of 0.627. Finally, a classification and score equivalency analysis was performed between triage and the NEWS2 scale in terms of mortality. Of the patients classified as triage 1, 32.3% died, and those who obtained a NEWS2 score greater than or equal to 10 had a mortality of 38.6%. For triage 5, mortality was 0%, and on the NEWS2 scale of 0 points, mortality was 1.3%. This analysis is presented in Table 4.
Table 4.
Classification and score equivalency analysis between triage and NEWS2.
| Triage | Alive n (%) | Deaths n (%) | Puntaje NEWS2 | Alive n (%) | Deaths n (%) |
| 1 | 21 (0.5) | 51 (32.3) | ≥10 | 22 (0.6) | 61 (38.6) |
| 2 | 1118 (29) | 105 (66.5) | 4–9 | 1775 (46.4) | 88 (55.7) |
| 3 | 2231 (58.3) | 2 (1.3) | 2–3 | 803 (21) | 3 (1.9) |
| 4 | 341 (8.9) | 0 (0) | 1 | 528 (13.8) | 4 (2.5) |
| 5 | 116 (3) | 0 (0) | 0 | 699 (18,3) | (1.3) |
NEWS = National Early Warning Score.
From the database selected for the study (3986 patients admitted to the emergency room), 77 patients were selected for a reproducibility analysis. Regarding the characteristics of the general population, among the 77 patients, 10 exhibited mortality, with a mean age of 69 years, compared to the general population, which was 48 years. Mortality was higher for men (n = 7); the mean NEWS2 score at admission among patients who had a mortality outcome was 10 points, and for triage, it was 1. Finally, the mean of the patient's vital signs was a heart rate of 88 beats per minute, a respiratory rate of 22 breaths per minute, systolic blood pressure of 132 mm Hg, temperature of 36.4°C, and oxygen saturation of 81%.
These patients had to be assessed by nursing personnel with experience in triage classification and the use of the NEWS 2 score. Table 5 shows the intra-observer variability determined by a nurse who classified the same patient at observation times, providing means of 3.16 and 2.96 in the NEWS 2 score. Likewise, for triage, the mean was 2.88. Additionally, for NEWS2, a very high ICC of 0.91 was observed (95% CI: 0.880–0.934) with a P value of <.001 in contrast to triage, which presented a high correlation coefficient of 0.709 (95% CI: 0.705–0.832) and a P value <.001.
Table 5.
Data obtained for intra-observer variability.
| Variables | Población general | Vivos | Muertos |
| Age (yrs) | 48 | 46 | 69 |
| Female (n) | 39 | 36 | 3 |
| Score NEWS2 (x) | 3 | 2 | 10 |
| Triage (x) | 3 | 2 | 1 |
| Vital signs | |||
| HR (lpm) | 82 | 81 | 88 |
| RR (rpm) | 17 | 17 | 22 |
| SBP | 127 | 126 | 132 |
| Temperature (°C) | 36.5 | 36.5 | 36.4 |
| OS % | 92 | 93 | 81 |
HR = heart rate, lpm = litres per minute, n = 77, NEWS = National Early Warning Score, rpm = revolutions per minute, OS = Oxygen Saturation, RR = respiratory rate, SBP = systolic blood pressure.
The concordance analysis was performed at the inter-observer level (Table 6), where 2 nurses classified the same patient using triage and NEWS2 at equal observation times, resulting in means of 3.11 and 3.10 for NEWS2. In the triage, a mean of 2.88 was provided by both observers. The ICC was very high for NEWS2, with a result of 0.953 (95% CI: 0.945–0.960) compared to triage, with a moderate value of 0.709 (95% CI: 0.668–0.747).
Table 6.
Intra- and inter-observer TRIAGE and NEWS2 correlation and concordance.
| Intra-class correlation coefficient | CI 95% | P value | |
| Intra-observe concordance correlation | |||
| NEWS2 | 0.91 | 0.88–0.93 | <.001 |
| Triage | 0.77 | 0.70–0.83 | <.001 |
| Inter-observe concordance correlation | |||
| NEWS2 | 0.95 | 0.94–0.96 | <.001 |
| Triage | 0.70 | 0.66–0.74 | <.001 |
CI = confidence interval, NEWS = National Early Warning Score.
The means and correlations of the vital signs of the patients were evaluated by both intra-observers and did not have significant variability, even though the evaluation time was prolonged. Likewise, in the inter-observer observations, the vital signs did not have variability because they were assessed at the same time (Table 7).
Table 7.
Comparison intra-observe and inter-observer reliability for NEWS2 score and triage.
| Intra-observe reliability n = 77 | Inter-observer reliability n = 61 | ||||
| Nurse A1 and A2 | Media | Intra-class correlation CI 95% | Pearson correlation | Media | Intra-class correlation CI 95% |
| NEWS2 score | 3.16–2.96 | 0.911 (0.880–0.934) P < .0001 | 0.925 P < .0001 | 3.11–3.10 | 0,953 (0.945–0.960) P < .0001 |
| Triage | 2.88–2.88 | 0.777 (0.705–0.832) P < .0001 | 0.586 P < .0001 | 2.88–2.88 | 0.709 (0.668–0.747) P < .0001 |
CI = confidence interval, NEWS = National Early Warning Score.
4. Discussion
The NEWS2 scale patient assessment has been used in specific scenarios, such as for those with sepsis, where this scale has been shown to have a good correlation in the early identification of cases with a risk of deterioration.[34,35] In our study, patients with various pathologies were evaluated. The validity of NEWS2 in the emergency department and the prediction of in-hospital mortality were evaluated for both triage and NEWS2 among admitted adult patients, indicating good performance for both. NEWS2 showed a greater area under the ROC curve (0.90). The point with the highest precision in the area under the curve (AUC) for NEWS2 was a score greater than 7, as confirmed by the YI. The results found in our study correspond to those reported in a prospective observational study that evaluated NEWS2 mortality predictions at 2, 7, and 30 days, which are reflected in the AUC values of 0.88, 0.86, and 0.82 respectively.[1]
Studies carried out by Martín-Rodriguez et al,[36] evaluated the abilities of different pre-hospital triage systems based on physiological parameters (Shock Index, Glasgow-Age-Pressure Score, Revised Trauma Score, and NEWS2) to predict early mortality (within 48 hours), finding that the system with the best predictive capacity was NEWS2, with an AUC of 0.891 (95% CI, 0.84–0.94), a sensitivity of 79.7% (95% CI, 68.8–87.5), and a specificity of 84.5% (95% CI, 82.4–86.4) for a cutoff point of 9 points, with a positive probability ratio of 5.14 (95% CI, 4.31–6.14) and a negative predictive value of 98.7% (95% CI, 97.8–99.2). On the other hand, the main implementation of NEWS2 compared to NEWS yielded a better characterization of patients with hypercapnic respiratory failure by calculating the value of the score obtained from oxygen saturation using 2 sub-scores, but few studies have evaluated the usefulness of NEWS2 in the prediction of mortality from triage. One such study was also carried out by Martín-Rodriguez et al,[37] where the authors found an AUC for mortality of 0.862 (95% CI: 0.78–0.93) at 1 day, of 0.885 at 2 days (95% CI: 0.84–0.92), and of 0.835 at 7 days (95% CI: 0.79–0.87) (in all cases, P < .001). NEWS 2 has proven to be a highly useful score for the early identification of patients with sepsis in the emergency department, even when compared to new initiatives such as the Rapid Emergency Triage and Treatment System for sepsis Rapid Emergency Triage and Treatment System.[38] In other contexts, mainly Latin America, there was an initiative by de Oliveira et al[39] in Brazil, which used a prospective cohort study to evaluate nursing records to adapt and validate NEWS2 for the population. Our study is the first to validate NEWS2 in a Latin American population. These results are similar to other scores such as the Mortality in Emergency Department Sepsis, which shows an area under the ROC curve of 0.83 with the difference that Mortality in Emergency Department Sepsis include variables such as age and comorbidities that influence mortality, regardless of the clinical condition of the patient. Likewise, it uses values from laboratory results that can delay a rapid classification of the patient.[40,41]
Regarding the approximation of equivalence between the triage classification and the NEWS2 score to determine mortality, it was found that the mortality percentage is similar between triage 1 and a NEWS2 greater than or equal to 10 with an inverse relationship since the classification system is different. For triage, the lower the score, the greater the severity, while for NEWS2, the higher the score, the greater the severity. Furthermore, the mortality ratios for triage and other scales are similar. These findings suggest that the cutoff point for our population is higher compared to that established in the United Kingdom, where a NEWS2 of 7 was established to indicate a risk of mortality.[10,42] However, this cutoff is not counted among the evidence in the literature that corroborates these results.
In a study by Corfield et al[34] in 2013, the predictability of mortality and ICU admission was evaluated among patients who were admitted to the emergency department with a diagnosis of sepsis, where the percentage of patients admitted to the ICU and/or dead at 30 days was 35% for a NEWS2 score on admission greater than or equal to 9, presenting mortality similar to that found in our study for patients with a NEWS2 score on admission greater than or equal to 10 points, which totaled 37%. In another study, Engebretsen et al,[43] for 1586 patients, reported a cutoff point of 4 or higher as a predictor of admission to the ICU, while for admission to the ICU in our study, the cutoff point was 7 points or higher; these results require further analysis. NEWS2 was developed through widely used statistical methods for the development of predictive models. These can be comparable to other predictive analytics methods.[44,45] In addition, NEWS2 uses physiological variables that can change rapidly according to the clinical condition of the patient, which facilitates its follow-up.
It should be noted that although an alert scale guarantees the detection of critical patients and a favorable outcome, an incorrect choice or poor understanding of the chosen scale can also result in an overload of procedures and evaluations by the staff, which could generate unnecessary deviation from their work schedules. It is necessary to engage in a periodic assessment of the teaching materials and applications these scales, as well as the training of the personnel (nurses, doctors, and other members) who involved in the intervention of the detected patient using the application of the scale.[17]
Ming et al[46] found no differences in mortality outcomes and hospital stay when a rapid response team was implemented for emergency care. Other factors have a greater impact, such as economic, organizational, and personnel education factors. The use of the NEWS2 scale showed superior validity, managing to improve the objective care of these patients and optimize resources. In this case, NEWS2 provides support not only for rapid response teams but also for a large portion of other healthcare personnel (doctors, nurses, and nursing assistants).
In 2016, Bilben et al[47] compared NEWS with the Manchester triage scale in an emergency department. These results showed a close correlation between both scales in predicting mortality and the highest level of hospital care (emergency room, ICU, or intermediate care). In our study, the results not only show a correlation but also better sensitivity and specificity for mortality among adult patients treated in the emergency department, in addition to highlighting the need for a higher level of care (ICU).
Importantly, NEWS2 does not replace clinical criteria; instead, it is a tool that complements the assessment of patients in general practice, thereby consolidating the results of said assessment for both critical patients who require quick decisions and for patients with the possibility of outpatient management, without the need for additional studies, such as laboratory tests. Past studies have explored combinations of NEWS and biomarkers (lactate, glucose, and excess base); however, the performance has not been shown to improve.[30]
On the other hand, regarding the reproducibility of the NEWS2 scale, Hernández Ruipérez et al[26] evaluated the applicability of triage among 410 patients admitted to the emergency department who were classified by various observers using the ESI, which is a gold standard among groups of medical experts. The kappa index of this study between the comparison of the nurse's classification compared and that of the group of doctors was 0.68 (95% CI 0.65–0.71), which is similar to the results found here. On the other hand, another group of doctors compared with expert physicians presented a kappa index of 0.85 (95% CI 0.81–0.90). Clinical studies have shown that the ESI has good validity and reliability for special groups, such as children and older adults, with very good inter-observer kappa values (k = 0.82 and 0.46 to 0.91, respectively).[2,19,48,49] For these groups, adequate classification has been shown in the relevant emergency departments.
In our study, however, the average age of mortality among patients was 69 years, showing that with the NEWS2 scale, inter-observer correlation and agreement are higher than those of triage. This may be because NEWS2 is based on stipulated physiological criteria that determine severity in any adult age range, while triage may fail due to unconscious bias related to the multiple comorbidities of these patients.[19,49]
It should be noted that the correlation and concordance of triage at the intra-observer level are also low compared to those of NEWS2. This suggests that, according to the time at which the patient is reassessed, the patient will be given a different classification of greater or lesser complexity. Therefore, it is suggested to apply NEWS2 in this group of patients to identify whether they require immediate attention due to a high risk of clinical deterioration and to expand relevant studies with a larger sample to evaluate the intra-observer variability between these scales. The main limitation of the study is that we applied validation in a single healthcare center; however, the number of patients included, and the methodology applied partially overcame this limitation. Another limitation is that we carried out the validation in a hospital located at an altitude of 2650 m above sea level, which may limit its usefulness only to similarly located hospital centers.
5. Conclusion
NEWS2 is no worse than triage in the area under the ROC curve for predicting mortality. On the other hand, the cutoff points of NEWS2 were higher in our population compared to the cutoffs stipulated in the United Kingdom. In addition, a score greater than or equal to 10 for the NEWS2 was considered to indicate a correlation with triage 1. However, we require more studies to corroborate these results. Additionally, the intra-class correlation was higher than that of triage. Therefore, the intra- and inter-observer reproducibility of the NEWS2 scale should be analyzed with a larger sample. Finally, intra-observer studies are required for patients admitted to the emergency room with the ESI system.
Acknowledgment
We would like to thank the Universidad de La Sabana.
Author contributions
Conceptualization: Peter Vergara, Daniela Forero, Alirio Bastidas, Julio-Cesar Garcia.
Data curation: Daniela Forero, Alirio Bastidas.
Formal analysis: Peter Vergara, Daniela Forero, Alirio Bastidas, Jhosep Blanco, Jorge Azocar.
Investigation: Peter Vergara, Daniela Forero, Alirio Bastidas, Jhosep Blanco, Jorge Azocar.
Methodology: Peter Vergara, Daniela Forero, Alirio Bastidas.
Supervision: Peter Vergara, Julio-Cesar Garcia.
Visualization: Peter Vergara.
Writing – original draft: Peter Vergara, Daniela Forero, Alirio Bastidas, Julio-Cesar Garcia, Rosa-Helena Bustos, Hans Liebisch.
Writing – review & editing: Peter Vergara, Daniela Forero, Alirio Bastidas, Julio-Cesar Garcia, Rosa-Helena Bustos.
Supplementary Material
Footnotes
Abbreviations: AUC = area under the curve, CI = confidence interval, ESI = Emergency Severity Index, ICC = intra-class correlation coefficient, ICU = intensive care unit, NEWS = National Early Warning Score, ROC = receiver operating characteristic, YI = Youden index.
How to cite this article: Vergara P, Forero D, Bastidas A, Garcia JC, Blanco J, Azocar J, Bustos RH, Liebisch H. Validation of the National Early Warning Score (NEWS)-2 for adults in the emergency department in a tertiary-level clinic in Colombia: cohort study. Medicine. 2021;100:40(e27325).
The authors have no funding and conflicts of interest to disclose.
The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.
Supplemental digital content is available for this article.
References
- [1].Martín-Rodríguez F, López-Izquierdo R, Del Pozo Vegas C, et al. A multicenter observational prospective cohort study of association of the prehospital National Early Warning Score 2 and hospital triage with early mortality. Emerg Med Int 2019;2019:01–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [2].Christ M, Grossmann F, Winter D, Bingisser R, Platz E. Modern triage in the emergency department. Dtsch Arztebl Int 2010;107:892–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [3].ACEM, A.C.f.E.M. Guidelines on the implementation of the Australasian Triage Scale in emergency departments. Available at: https://acem.org.au/getmedia/51dc74f7-9ff0-42ce-872a-0437f3db640a/G24_04. Accessed September 24, 2020. [Google Scholar]
- [4].CAEP, C.A.o.E.P. Canadian Triage and Acuity Scale (CTAS). Available at: tas-phctas.ca/wp-content/uploads/2018/05/participant_manual_v2.5b_november_2013_0.pdf. Accessed September 23, 2020. [Google Scholar]
- [5].van der Wulp I, van Baar ME, Schrijvers AJ. Reliability and validity of the Manchester Triage System in a general emergency department patient population in the Netherlands: results of a simulation study. Emerg Med J 2008;25:431–4. [DOI] [PubMed] [Google Scholar]
- [6].Gräff I, Latzel B, Glien P, Fimmers R, Dolscheid-Pommerich RC. Validity of the Manchester Triage System in emergency patients receiving life-saving intervention or acute medical treatment—a prospective observational study in the emergency department. J Eval Clin Pract 2019;25:398–403. [DOI] [PubMed] [Google Scholar]
- [7].Lähdet EF, Suserud BO, Jonsson A, Lundberg L. Analysis of triage worldwide. Emerg Nurse 2009;17:16–9. [DOI] [PubMed] [Google Scholar]
- [8].Gilboy N, Tanabe T, Travers D, Rosenau A. Quality AfHRa. Emergency Severity Index (ESI): A Triage Tool for Emergency Department Care, Version 4. Implementation Handbook 2012 Edition. Rockville, MD: Agency for Healthcare Research and Quality; 2011. 114. [Google Scholar]
- [9].Wessex Academic Health Science Network. Emergency Severity Index. Available at: https://www.northeasthampshireandfarnhamccg.nhs.uk/documents/north-east-hampshire-and-farnham-vanguard/evaluation/838-esi-evaluation-report-final/file. Accessed September 23, 2020. [Google Scholar]
- [10].Physicians, R.C.o. National Early Warning Score (NEWS) 2. Standardising the assessment of acute-illnessseverity in the NHS. Available at: https://www.rcplondon.ac.uk/projects/outputs/national-early-warning-score-news-2. Accessed July 15, 2020. [Google Scholar]
- [11].Caramello V, Marulli G, Reimondo G, Fanto F, Boccuzzi A. Comparison of Reverse Triage with National Early Warning Score, Sequential Organ Failure Assessment and Charlson Comorbidity Index to classify medical inpatients of an Italian II level hospital according to their resource's need. Intern Emerg Med 2019;14:1073–82. [DOI] [PubMed] [Google Scholar]
- [12].Salluh JI, Soares M. ICU severity of illness scores: APACHE, SAPS and MPM. Curr Opin Crit Care 2014;20:557–65. [DOI] [PubMed] [Google Scholar]
- [13].Fernando SM, Fox-Robichaud AE, Rochwerg B, et al. Prognostic accuracy of the Hamilton Early Warning Score (HEWS) and the National Early Warning Score 2 (NEWS2) among hospitalized patients assessed by a rapid response team. Crit Care (London, England) 2019;23:60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [14].Fullerton JN, Price CL, Silvey NE, Brace SJ, Perkins GD. Is the Modified Early Warning Score (MEWS) superior to clinician judgement in detecting critical illness in the pre-hospital environment? Resuscitation 2012;83:557–62. [DOI] [PubMed] [Google Scholar]
- [15].Capan M, Ivy JS, Rohleder T, Hickman J, Huddleston JM. Individualizing and optimizing the use of early warning scores in acute medical care for deteriorating hospitalized patients. Resuscitation 2015;93:107–12. [DOI] [PubMed] [Google Scholar]
- [16].Jäderling G, Bell M, Martling CR, Ekbom A, Bottai M, Konrad D. ICU admittance by a rapid response team versus conventional admittance, characteristics, and outcome. Crit Care Med 2013;41:725–31. [DOI] [PubMed] [Google Scholar]
- [17].Smith ME, Chiovaro JC, O’Neil M, et al. Early warning system scores for clinical deterioration in hospitalized patients: a systematic review. Ann Am Thorac Soc 2014;11:1454–65. [DOI] [PubMed] [Google Scholar]
- [18].G.N. V, González MF. Comparación de parámetros de atención en una escala de triage de 5 niveles con los parámetros estándar. Bogotá, Colombia: Universidad Colegio Mayor de Nuestra Señora del Rosario; 2015. [Google Scholar]
- [19].Platts-Mills TF, Travers D, Biese K, et al. Accuracy of the Emergency Severity Index triage instrument for identifying elder emergency department patients receiving an immediate life-saving intervention. Acad Emerg Med 2010;17:238–43. [DOI] [PubMed] [Google Scholar]
- [20].Mendoza C, Elguero P. Sensibilidad del triage clínico en el Servicio de Urgencias Adultos del HRLALM del ISSSTE. Arch Med Urg Mex 2011;3:96–8. [Google Scholar]
- [21].Wuerz RC, Milne LW, Eitel DR, Travers D, Gilboy N. Reliability and validity of a new five-level triage instrument. Acad Emerg Med 2000;7:236–42. [DOI] [PubMed] [Google Scholar]
- [22]. Agency for Healthcare Research and Quality. Emergency Severity Index (ESI): a triage tool for emergency departments. Available at: https://www.ahrq.gov/professionals/systems/hospital/esi/index.html. Accessed September 25, 2020. [Google Scholar]
- [23]. Agency for Healthcare Research and Quality. Emergency Severity Index implementation handbook. Available at: https://www.ahrq.gov/professionals/systems/hospital/esi/esi1.html. Accessed September 25, 2020. [Google Scholar]
- [24].Eitel DR, Travers DA, Rosenau AM, Gilboy N, Wuerz RC. The emergency severity index triage algorithm version 2 is reliable and valid. Acad Emerg Med 2003;10:1070–80. [DOI] [PubMed] [Google Scholar]
- [25].Tanabe P, Gimbel R, Yarnold PR, Kyriacou DN, Adams JG. Reliability and validity of scores on The Emergency Severity Index version 3. Acad Emerg Med 2004;11:59–65. [DOI] [PubMed] [Google Scholar]
- [26].Hernández Ruipérez T, Leal Costa C, Adánez Martínez MG, García Pérez B, Nova López D, Díaz Agea JL. Evidence of the validity of the Emergency Severity Index for triage in a general hospital emergency department. Emergencias 2015;27:301–6. [PubMed] [Google Scholar]
- [27].Jones D, Mitchell I, Hillman K, Story D. Defining clinical deterioration. Resuscitation 2013;84:1029–34. [DOI] [PubMed] [Google Scholar]
- [28].McNeill G, Bryden D. Do either early warning systems or emergency response teams improve hospital patient survival? A systematic review. Resuscitation 2013;84:1652–67. [DOI] [PubMed] [Google Scholar]
- [29].Mohammed MA, Faisal M, Richardson D, et al. The inclusion of delirium in version 2 of the National Early Warning Score will substantially increase the alerts for escalating levels of care: findings from a retrospective database study of emergency medical admissions in two hospitals. Clin Med (Lond) 2019;19:104–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [30].Abbott TEF, Torrance HDT, Cron N, Vaid N, Emmanuel J. A single-centre cohort study of National Early Warning Score (NEWS) and near patient testing in acute medical admissions. Eur J Intern Med 2016;35:78–82. [DOI] [PubMed] [Google Scholar]
- [31].Ministerio de Salud y Protección Social. Resolución 5596 del 24 de diciembre de 2015. Available at: https://www.minsalud.gov.co/salud/PServicios/Paginas/triage.aspx. Accessed September 21, 2020. [Google Scholar]
- [32].Soler W, Gómez Muñoz M, Bragulat E, Álvarez A. El triaje: herramienta fundamental en urgencias y emergencias. An Sist Sanit Navar 2010;33:55–68. [PubMed] [Google Scholar]
- [33].Keep JW, Messmer AS, Sladden R, et al. National early warning score at Emergency Department triage may allow earlier identification of patients with severe sepsis and septic shock: a retrospective observational study. Emerg Med J 2016;33:37–41. [DOI] [PubMed] [Google Scholar]
- [34].Corfield AR, Lees F, Zealley I, et al. Utility of a single early warning score in patients with sepsis in the emergency department. Emerg Med J 2014;31:482. [DOI] [PubMed] [Google Scholar]
- [35].Geier F, Popp S, Greve Y, et al. Severity illness scoring systems for early identification and prediction of in-hospital mortality in patients with suspected sepsis presenting to the emergency department. Wien Klin Wochenschr 2013;125:508–15. [DOI] [PubMed] [Google Scholar]
- [36].Martín-Rodríguez F, López-Izquierdo R, Del Pozo Vegas C, et al. Accuracy of National Early Warning Score 2 (NEWS2) in prehospital triage on in-hospital early mortality: a multi-center observational prospective cohort study. Prehosp Disaster Med 2019;34:610–8. [DOI] [PubMed] [Google Scholar]
- [37].Martín-Rodríguez F, López-Izquierdo R, Del Pozo Vegas C, et al. Can the prehospital National Early Warning Score 2 identify patients at risk of in-hospital early mortality? A prospective, multicenter cohort study. Heart Lung 2020;49:585–91. [DOI] [PubMed] [Google Scholar]
- [38].Mellhammar L, Linder A, Tverring J, et al. Scores for sepsis detection and risk stratification – construction of a novel score using a statistical approach and validation of RETTS. PLoS One 2020;15:e0229210. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [39].De Oliveira AP, Urbanetto J, Caregnato R. Transcultural adaptation and validation of the National Early Warning Score 2 for Brazil. J Nurs 2018;12:3154–7. [Google Scholar]
- [40].Zhang G, Zhang K, Zheng X, Cui W, Hong Y, Zhang Z. Performance of the MEDS score in predicting mortality among emergency department patients with a suspected infection: a meta-analysis. Emerg Med J 2020;37:232–9. [DOI] [PubMed] [Google Scholar]
- [41].Shapiro NI, Wolfe RE, Moore RB, Smith E, Burdick E, Bates DW. Mortality in Emergency Department Sepsis (MEDS) score: a prospectively derived and validated clinical prediction rule. Crit Care Med 2003;31:670–5. [DOI] [PubMed] [Google Scholar]
- [42].Kuriyama A, Urushidani S, Nakayama T. Five-level emergency triage systems: variation in assessment of validity. Emerg Med J 2017;34:703–10. [DOI] [PubMed] [Google Scholar]
- [43].Engebretsen S, Bogstrand ST, Jacobsen D, Vitelli V, Rimstad R. NEWS2 versus a single-parameter system to identify critically ill medical patients in the emergency department. Resusc Plus 2020;3:100020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [44].Zhang Z, Liu J, Xi J, Gong Y, Zeng L, Ma P. Derivation and validation of an ensemble model for the prediction of agitation in mechanically ventilated patients maintained under light sedation. Crit Care Med 2021;49:e279–90. [DOI] [PubMed] [Google Scholar]
- [45].Zhang Z, Zheng B, Liu N, Ge H, Hong Y. Mechanical power normalized to predicted body weight as a predictor of mortality in patients with acute respiratory distress syndrome. Intensive Care Med 2019;45:856–64. [DOI] [PubMed] [Google Scholar]
- [46].Ming T, Lai A, Lau PM. Can team triage improve patient flow in the emergency department? A systematic review and meta-analysis. Adv Emerg Nurs J 2016;38:233–50. [DOI] [PubMed] [Google Scholar]
- [47].Bilben B, Grandal L, Søvik S. National Early Warning Score (NEWS) as an emergency department predictor of disease severity and 90-day survival in the acutely dyspneic patient - a prospective observational study. Scand J Trauma Resusc Emerg Med 2016;24:80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [48].Jiménez J. Clasificación de pacientes en los servicios de urgencias y emergencias: Hacia un modelo de triaje estructurado de urgencias y emergencias. Emergencias 2003;15:165–74. [Google Scholar]
- [49].Wuerz RC, Travers D, Gilboy N, Eitel DR, Rosenau A, Yazhari R. Implementation and refinement of the emergency severity index. Acad Emerg Med 2001;8:170–6. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.