Abstract
Background
Hospitalised patients are at risk of deterioration and death. Delayed identification and transfer to the intensive care unit (ICU) are known to be associated with increased mortality rates. The Risk-stratification of Emergency Department suspected Sepsis (REDS) score was derived and validated in emergency department patients with suspected sepsis. It is unknown if the REDS score would risk-stratify undifferentiated hospitalised patients who deteriorate.
Objectives
To validate the REDS score in hospitalised patients who deteriorate.
Methods
This retrospective cohort single-centre study involved hospitalised adult patients who deteriorated and were transferred to the ICU between 1 April 2022 and 31 March 2023. The first admission to the ICU was studied. The National Early Warning Score2 (NEWS2), REDS, Sequential Organ Failure Assessment (SOFA) and change-in-SOFA (ΔSOFA) scores were calculated at the time of referral to the Critical Care Outreach Team (CCOT). The primary outcome measure was in-hospital all-cause mortality. The area under the receiver operator characteristic (AUROC) curves for the scores were compared. Test characteristics at the cut-off points individually and in combination were noted.
Results
Of the 289 patients studied, 91 died. The REDS score had the largest AUROC curve at 0.70 (95% CI 0.65 to 0.75), greater than the NEWS2 score at 0.62 (95% CI 0.56 to 0.68), p=0.03, and similar to the SOFA score 0.67 (95% CI 0.61 to 0.72), p=0.3. The cut-off points for the NEWS2, REDS, SOFA and ΔSOFA scores were >9, >3, >6 and >4, respectively. The sensitivity and specificity for a ΔSOFA≥2 was 91.2% (95% CI 83.4 to 96.1) and 15.7% (95% CI 10.9 to 21.5), respectively. REDS≥4 or NEWS2≥7 had a sensitivity of 87.9% (95% CI 79.4 to 93.8) and specificity of 29.3% (95% CI 23.1 to 36.2).
Conclusion
The prognostic performance of the REDS score was similar to the SOFA score, but greater than the NEWS2 score. The REDS score could be used in addition to the established NEWS2 score to risk-stratify hospitalised patients for mortality.
Keywords: Critical care; Decision making; Decision support, clinical; Hospital medicine; Trigger tools
WHAT IS ALREADY KNOWN ON THIS TOPIC
Hospitalised patients often deteriorate on the wards. The National Early Warning Score (NEWS2) score is used to escalate care. However, the NEWS2 score may not pick up all sick patients who are at risk of death.
WHAT THIS STUDY ADDS
In this study, we have shown that the Risk-stratification of Emergency Department suspected Sepsis (REDS) score of ≥4 can be used, in addition to a NEWS2 score of ≥7, to identify patients at high risk of death. As such, we could identify patients with a sensitivity equivalent to a two-point increase in the Sequential Organ Failure Assessment (SOFA) score but with greater specificity.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
The use of the REDS score to identify high-risk patients who deteriorate, should be studied in a multicentre study.
Introduction
Hospitalised patients are often moved to the intensive care unit (ICU) when their clinical condition deteriorates. Timely recognition and treatment are key to improving outcomes as they will enable prompt assessment and escalation of care.1 Delays in recognising the clinical deterioration may result in worsening the patient’s condition and may result in harm.2 Every hospital should have a system in place to identify deteriorating patients.3 4 Such systems should include a hospital-wide scoring system that will serve as an objective measure of deterioration and should be used in conjunction with the patient’s clinical assessment.
The National Early Warning Score (NEWS) was developed through consensus in 2012.5 In 2017, the Royal College of Physicians updated the score (NEWS2) and recommends its use to monitor adult patients for deterioration.6 Increasing thresholds of the NEWS2 score are used to escalate to higher levels of care with increasing urgency;7 the escalation of a patient with a NEWS2 score of ≥5 to the Critical Care Outreach Team (CCOT) is advised, and a NEWS2 score of ≥7 would warrant continuous monitoring and mandates assessment by a clinician with critical care competencies. The use of the NEWS2 score has also been endorsed by the National Institute for Health and Care Excellence.8
A recent systematic review of early warning scores by Gerry et al found that many early warning scores have methodological weaknesses and may not perform as expected.9 Furthermore, a study by Blackwell et al10 found that multiple predictive models for clinical deterioration perform better than a single-predictive model for clinical events leading to ICU transfer.
The Sequential Organ Failure Assessment (SOFA) score11 is used to risk-stratify patients in the ICU setting. The score ranges from 0 to 24. An increase of two points from baseline is associated with a mortality rate of over 10%, and is used to define sepsis in the context of an infection.12 Previous studies have shown that a cut-off point of 7–7.5 on admission to ICU was associated with mortality.13 14
The Risk-stratification of Emergency Department suspected Sepsis (REDS) score was derived, validated and implemented in the emergency department (ED).15 16 The score is composed of eight variables; age≥65 years, respiratory rate (RR)≥22 breaths/min, systolic blood pressure (SBP)≤100 mm Hg, altered mental state, serum albumin≤27 g/L and an International Normalised Ratio (INR)≥1.3, each attracting a score of 1 point when present. An initial lactate level of ≤2 mmol/L scores 0, a level of 2.1–3.9 mmol/L scores 1 point and a lactate≥4 mmol/L scores 3 points. The presence of refractory hypotension (RH), a mean arterial pressure<65 mm Hg after a fluid bolus of 30 mL/kg or a clinician determined need for vasopressors, with a postfluid lactate of ≤2 mmol/L scores 2 points and 3 points if the postfluid lactate>2 mmol/L scores. The REDS score ranges from 0 to 12 points, and a score of ≥3 is deemed high-risk. Those with a REDS score of ≥5 are referred to the CCOT. None of these variables in the REDS score are specific to sepsis and can occur in deteriorating patients. The performance of the REDS score to risk stratify undifferentiated in-hospital patients for mortality in those who deteriorate prior to transfer to the ICU is not known.
The primary aim of this study is to validate the REDS score in the population of hospitalised patients admitted to the ICU following deterioration, at the point of referral to the CCOT, regardless of the underlying condition or the cause of deterioration. The primary outcome measure is in-hospital mortality. The performance of the REDS score will also be compared with that of the SOFA score11 and the NEWS2 score.6
Methodology
Setting and study population
This study was carried out in a large inner-city tertiary referral hospital with 995 beds on a single site.17 Patients who deteriorate from an in-patient setting are transferred to the ICU through CCOT. This study was carried out in adult patients transferred from an in-hospital setting to the ICU between 1 April 2022 and 31 March 2023. Patients admitted to the ICU directly from the ED and patients transferred from an ICU were excluded. Only the first admission to the ICU was studied.
The list of patients admitted to the ICU were obtained from CCOT. Medically qualified researchers were trained in medical record abstraction from the electronic patient records. The extracted data were entered directly into a preformatted password-protected Excel spreadsheet. The demographic data, the date and time of admission to the hospital, the specialty under which they were initially admitted, the point of referral to CCOT and admission to the ICU were noted. The vital signs (RR, SBP, diastolic blood pressure, peripheral oxygen saturation (SaO2), fraction of inspired oxygen (FiO2), temperature, Glasgow Coma Scale (GCS), the presence of altered mental state) were noted at three time points; on admission to the hospital, at the point of referral to the CCOT team and on admission to the ICU. Results of blood tests taken at the afore mentioned three time points were also noted; namely, haemoglobin (Hb), white cell count (WCC), neutrophil count, platelet count, INR, the use of oral anticoagulants (warfarin and direct oral anticoagulants), urea, creatinine, bilirubin, c-reactive protein, serum albumin and point of care lactate. Baseline GCS, platelets, bilirubin and creatinine were collected to calculate the baseline SOFA score. The use of vasopressors at the time of deterioration, if the primary diagnosis for admission to ICU was infection/sepsis or non-infection/sepsis, the date of discharge from the ICU, and hospital were also noted. A second researcher rechecked all the implausible vital signs against the source data and corrected them as necessary. All blood results were validated against the source data, including the point-of-care lactate measurements and any errors were corrected. Similarly, the calculation of the NEWS2, REDS and SOFA scores was checked and corrected where necessary. Missing variables at the point of referral to CCOT were substituted with the measurements recorded at admission to the ICU, allowing complete case analysis. Missing variables on admission were assumed to be normal and coded as such, when calculating the admission score. The following comorbidities were collected: a history of dementia, active malignancy, care home residency or a minimum three times a day care package which would reflect a dependency for activities of daily living and align with a clinical frailty scale of 6 or more,18 use of long-term oxygen therapy and previous in-hospital or community do-not-attempt-cardio-pulmonary-resuscitation (DNACPR) order.
Arterial blood gases were not always available to calculate the SOFA score on admission to the hospital and at the point of referral to CCOT. For consistency, SaO2/FiO2 was used to calculate the respiratory component of the SOFA score.19 Patients on or requiring vasopressors were given a score of 3 for the cardiovascular parameter in the SOFA score. For those on oral anticoagulants, a score of 0 points was given for INR when calculating the REDS score. The data were anonymised following validation, before analysis.
The NEWS, REDS and SOFA scores for survivors and non-survivors at the point of admission and referral to CCOT were calculated and compared, as was the baseline SOFA score. These analyses were repeated after dividing the study population into those with and without infection/sepsis. The differences in vital signs, blood results and comorbidities were also calculated for survivors and non-survivors.
The change in SOFA (ΔSOFA) score was calculated by subtracting the baseline SOFA score from the SOFA score at the point of CCOT referral. Receiver operator characteristic (ROC) curves were constructed for the NEWS, REDS, SOFA and ΔSOFA scores at the point of referral to CCOT, the cut-off points were identified and the areas under the ROC (AUROC) curves were compared. Mortality rates associated with the different scores were calculated. Test characteristics were studied for the identified cut-off points, and prespecified cut-off points (NEWS2 score≥5 and ≥7 and a ΔSOFA score≥2) were calculated. Patients and the public were not involved in the design or execution of this retrospective study.
Sample size
The minimum sample size for a dichotomous outcome is 10 outcomes per variable.20 The REDS score, with eight variables, had the largest number of variables. Thus, the study population should consist of a minimum of 80 deaths.
Statistical analysis
MedCalc Statistical Software V.22.021 (MedCalc Software, Ostend, Belgium) was used for statistical analysis. Continuous data were checked for normality using the Kolmogorov-Smirnov test. Normally distributed data were presented as mean and standard deviation (SD). When normality was rejected, the data were presented as medians together with their interquartile range (IQR). Normally distributed data were analysed using unpaired t-tests and non-normally distributed data were compared using Mann-Whitney U tests. Paired normally distributed data were analysed using paired-t tests, and paired non-normally distributed data were analysed using Wilcoxon signed-rank tests. Categorical data were compared using the χ2 test. The DeLong21 method was used to compare the AUROC curves. Statistical significance was accepted when p<0.05.
Results
Of the 360 patients on the list of patients admitted to the ICU from in-patient areas, 289 were first admissions; of these 91 died in hospital, a mortality rate of 31.5%. Of the 71 cases excluded from the analysis, one was a transfer between ICUs, and 70 were not the first admission to the ICU on that episode of hospital admission. The initial specialty under which the patients were admitted were as follows: General medicine 92 (31.8%), Cardiology 45 (15.6%), Trauma and Orthopaedics 22 (7.6%), Haematology 20 (6.9%), Neurosurgery 18 (6.2%), Stroke 15 (5.2%), Renal 13 (4.5%), Cardiothoracic surgery 12 (4.2%), General surgery 11 (3.8%), Vascular surgery 8 (2.8%), Urology 7 (2.4%), Infectious diseases 7 (2.4%), Neurology 6 (2.1%), Oncology 4 (1.4%), Plastic surgery 3 (1%), Gastroenterology 2 (0.7%), Obstetrics and Gynaecology 2 (0.7%) and 1 (0.3%) each in Ear Nose and Throat surgery and Maxillofacial Surgery. The blood results that were missing on admission to the hospital and assumed to be normal were as follows: lactate 52 (18%), INR 21 (7.3%), albumin 12 (4.2%), platelets 2 (0.7%), bilirubin 11 (3.8%) and creatinine 2 (0.7%). No variables were missing for calculating the scores at the point of referral to CCOT as any missing variables were substituted with measurements taken at admission to the ICU, allowing complete case analysis. Of the 289 patients, 98 were transferred to ICU with a primary diagnosis of infection/sepsis, of whom 37 died, a mortality rate of 37.8%. Of 98 patients with infection/sepsis, 94 had an increase in SOFA score of ≥2, from baseline.
The characteristics of the study population at the point of referral to CCOT can be found in table 1. Age and some vital signs were significantly abnormal in those who died, as were some laboratory results and the point-of-care lactate. The NEWS2, REDS and SOFA scores were all significantly higher in non-survivors compared with survivors in the population as a whole. When broken down into those with and without sepsis, only the REDS score was significantly higher in those who died with sepsis. All three scores were significantly higher in those who died without sepsis. The mode of admission to hospital, whether it was through the ED or directly to the ward or a transfer from another hospital, did not differ between survivors and non-survivors.
Table 1. Characteristics of the study population at the point of referral to CCOT.
| All (n=289);median (IQR);mean (±SD) | Survivors (n=198);median (IQR);mean (±SD) | Non-survivors (n=91);median (IQR); mean (±SD) | P value | |
| Age (years) | 65 (52–75) | 64 (50–73) | 68 (57–78) | 0.01* |
| Sex (% male) | 189 (65.4%) | 126 (63.6%) | 63 (69.2%) | 0.42 |
| Malignancy | 35 (12.1%) | 21 (10.6%) | 14 (15.4%) | 0.25 |
| Vital signs | ||||
| Respiratory rate (breaths/min) | 21 (18–25) | 20 (18–24) | 23 (18–27) | 0.04* |
| Oxygen saturation (percentage) | 95% (92–97) | 95% (92–98) | 95% (91–97) | 0.36 |
| Fraction of inspired oxygen (FiO2) | 0.28 (0.21–0.60) | 0.28 (0.21–0.60) | 0.28 (0.21–0.46) | 0.89 |
| Heart rate (beats/min) | 96 (74–112) | 99 (75–115) | 91 (72–109) | 0.16 |
| Systolic blood pressure (mm Hg) | 116 (±40) | 119 (±38) | 108 (±45) | 0.03* |
| Temperature (degrees centigrade) | 36.8 (36.5–37) | 36.7 (36.5–37) | 36.8 (36.5–37) | 0.99 |
| Glasgow Coma Scale (GCS) | 15 (13–15) | 15 (13–15) | 14 (11–15) | 0.02* |
| Altered mental state | 122 (42.2%) | 74 (37.4%) | 48 (52.7%) | 0.01* |
| Hypotension requiring vasopressors | 74 (25.6%) | 40 (20.2%) | 34 (37.4%) | 0.002* |
| Blood results | ||||
| Haemoglobin (g/L) | 109 (±27) | 110 (±25.5) | 106 (±28.7) | 0.34 |
| WCC (x109/L) | 10.5 (7.2–14.7) | 9.9 (7.2–13.7) | 12.0 (7.6–17.4) | 0.02* |
| Neutrophil count (x109/L) | 8.1 (5.3–12) | 7.5 (5.3–10.9) | 9.1 (5.5–14.5) | 0.05 |
| Platelets (x109/L) | 200 (154–284) | 203.5 (159–291) | 200 (124–270) | 0.05 |
| International Normalised Ratio (INR) | 1.2 (1.1–1.4) | 1.2 (1.1–1.3) | 1.3 (1.2–1.6) | 0.33 |
| Oral anticoagulants (%) | 43 (14.9%) | 23 (11.6%) | 20 (22%) | 0.02* |
| INR (not on anticoagulants) | 1.2 (1.1–1.3) | 1.1 (1–1.3) | 1.3 (1.1–1.5) | <0.0001* |
| Urea (mmol/L) | 8.5 (5–15.5) | 7.3 (4.6–12.6) | 11.7 (6.1–19.5) | 0.0003* |
| Creatinine (micromol/L) | 101 (70–228) | 94 (65–195) | 131 (79–253) | 0.03* |
| Bilirubin (micromol/L) | 9 (6–16) | 9 (6–15) | 10 (6–19.5) | 0.07 |
| C-reactive protein (CRP) (mg/L) | 66 (17–149) | 66.5 (14.3–149.5) | 65 (24–149) | 0.68 |
| Albumin (g/L) | 26 (22–31) | 27 (22.25–32) | 25 (22–29.5) | 0.03* |
| Lactate (mmol/L) | 1.7 (1–3.2) | 1.4 (0.9–2.4) | 2.5 (1.4–6.4) | <0.0001* |
| Comorbidities | ||||
| Dementia | 12 (4.2%) | 8 (4%) | 4 (4.4%) | 1.0 |
| Malignancy | 35 (12.1%) | 21 (10.6%) | 14 (15.4%) | 0.25 |
| Care home residency or minimum three times a day care-package | 24 (8.3%) | 15 (7.6%) | 9 (9.9%) | 0.5 |
| Long-term oxygen therapy | 3 (3.1%) | 2 (1%) | 1 (1.1%) | 1.0 |
| Previous DNACPR order | 2 (0.7%) | 0 | 2 (2.2%) | 0.1 |
| Scores | ||||
| NEWS2 | 7 (5–9) | 7 (5–9) | 8 (6–11) | 0.001* |
| Infection/sepsis (n=98) | 8 (7–10) | 8 (7–9) | 8.5 (7–11) | 0.15 |
| Non-infection/sepsis (n=191) | 7 (4–9) | 6 (4–8) | 8 (5–11) | 0.007* |
| REDS | 4 (2–6) | 3 (2–5) | 5 (3–8) | <0.0001* |
| Infection/sepsis | 4 (3–7) | 4 (2–5) | 5.5 (3–8) | 0.026* |
| Non-infection/sepsis | 3 (2–5) | 3 (2–4) | 5 (3.25–8) | <0.0001* |
| SOFA | 5 (3–7) | 4 (3–7) | 7 (4–8.5) | <0.0001* |
| Sepsis | 6 (4–8) | 5 (3–7) | 7 (4–8) | 0.10 |
| Non-infection/sepsis | 4 (3–7) | 4 (3–6) | 7 (4–10) | <0.0001* |
| Type of admission | ||||
| Emergency | 259 (89.6%) | 172 (86.9%) | 87 (95.6%) | 0.02* |
| Elective | 30 (10.4%) | 26 (13.1%) | 4 (4.4%) | 0.02* |
| Duration | ||||
| Arrival to ICU admission (hours) | 72.8 (35.5–217.9) | 68.3 (37–214.7) | 81.6 (32.4–219.9) | 0.94 |
| Arrival to referral to CCOT (hours) | 64.9 (23–206.2) | 59.3 (23.5–194.9) | 75.2 (19.9–212.5) | 0.97 |
| Referral to CCOT to ICU admission (hours) | 5.6 (3–12.3) | 5.5 (3–13.9) | 5.8 (2.9–10.3) | 0.75 |
| ICU LOS (days)−1st admission | 5 (3–11) | 5 (3–10) | 6 (2–13) | 0.82 |
| Hospital LOS (days) | 24 (11.5–52) | 30 (16–58) | 16 (7–29.5) | <0.0001* |
Statistical significance.; ; ; ; ; of stay
CCOTCritical Care Outreach TeamDNACPRdo-not-attempt-cardio-pulmonary-resuscitationEDemergency departmentICUintensive care unitLOSlength of stayNEWS2National Early Warning Score 2SOFASequential Organ Failure AssessmentWCCwhite cell count
Figure 1 shows that the AUROC curve was largest for the REDS score, 0.70 (95% CI 0.65 to 0.75) and significantly larger than that of the NEWS2 score of 0.62 (95% CI 0.56 to 0.68), p=0.03. The AUROC of the SOFA score was 0.67 (95% CI 0.61 to 0.72), but the difference compared with that of the REDS score did not reach statistical significance (p=0.31). In this study, the cut-off points for the NEWS2, REDS and SOFA scores were >9 (95% CI>7 to >13), >3 (95% CI>2 to >6) and >6 (95% CI>2 to >7), respectively. The AUROC curve for the ΔSOFA score was 0.65 (95% CI 0.59 to 0.70) with a cut-off point of >4 (95% CI>2 to >6). There was no difference between the AUROC curve of the SOFA and the ΔSOFA scores, p=0.23.
Figure 1. Receiver operator characteristic curves for the NEWS2 score, REDS score and the SOFA score at the point of referral to CCOT, for in-hospital mortality. CCOT, Critical Care Outreach Team; NEWS2, National Early Warning Score 2; REDS, Risk-stratification of Emergency Department suspected Sepsis; SOFA, Sequential Organ Failure Assessment.
Wilcoxon signed-rank test of the paired scores showed that both survivors and non-survivors deteriorated between admission and the point of referral to CCOT, in all three scoring systems (p<0.0001). Figure 2 illustrates the distribution of the scores for survivors and non-survivors in the three scoring systems on admission and at the point of referral to CCOT. Non-survivors had significantly higher scores at the point of referral to CCOT in all three scoring systems. However, only the REDS and SOFA scores were significantly higher in the non-survivors compared with the survivors on admission. There was no difference in the NEWS2 score between survivors and non-survivors at the point of admission to the hospital. The median SOFA score on admission of 2 (IQR 0–4) was significantly higher than the baseline SOFA score of 0 (IQR 0–1), p<0.0001. The median baseline SOFA score for survivors was 0 (IQR 0–1), and for non-survivors was 0 (IQR 0–2), p=0.21. Of note, on admission, 22 of the 198 survivors had REDS scores between 4 and 9 on arrival, which are highlighted as outliers. Of these 15 had a score of 4, three had a score of 5, one scored 6, one scored 7 and two had a score of 9. Of the 22 patients, 10 had improved their REDS score by the time they were referred to CCOT, four had no change in their score, four increased their score by 1 point and four had an increase in their score of between 3 and 7 points at the time of referral to CCOT. No further analysis was undertaken of this group as the numbers were small.
Figure 2. Comparison of the different scores in survivors and non-survivors on admission and at the point of referral to the Critical Care Outreach Team (CCOT). (i) Comparison of the NEWS2 score in survivors and non-survivors at the point of admission to the hospital. (ii) Comparison of the NEWS2 score in survivors and non-survivors at the point of referral to the CCOT team. (iii) Comparison of the REDS score in survivors and non-survivors at the point of admission to the hospital. (iv) Comparison of the REDS score in survivors and non-survivors at the point of referral to the CCOT team. (v) Comparison of the SOFA score in survivors and non-survivors at the point of admission to the hospital. (vi) Comparison of the SOFA score in survivors and non-survivors at the point of referral to the CCOT team. CCOT, Critical Care Outreach Team; NEWS2, National Early Warning Score 2; REDS, Risk-stratification of Emergency Department suspected Sepsis; SOFA, Sequential Organ Failure Assessment.
Table 2 illustrates the mortality rates associated with the different score of the different scoring systems. The test characteristics of the NEWS, REDS, SOFA and ΔSOFA scores at the identified and prespecified cut-off points are illustrated in table 3 together with the different combinations of the scores at these cut-off points.
Table 2. Mortality rates associated with different scores in the different scoring systems calculated at the point of referral to CCOT.
| Score | NEWS2 | REDS | SOFA | ΔSOFA |
| 0 | 12.5% (1/8) | 9.1% (1/11) | 8.3% (1/12) | 17.4% (4/23) |
| 1 | 33.3% (2/6) | 11.4% (4/35) | 0% (0/9) | 25% (4/16) |
| 2 | 25% (1/4) | 18.8% (9/48) | 14.3% (3/21) | 17.6% (6/34) |
| 3 | 21.7% (5/23) | 25.6% (11/43) | 30% (12/40) | 26.3% (10/38) |
| 0–3 | 22% (9/41) | 18.2% (25/137) | 19.5% (16/82) | 21.6% (24/111) |
| 4 | 34.8% (8/23) | 35.8% (19/53) | 24.5% (12/49) | 26.8% (15/56) |
| 5 | 20% (4/20) | 29.2% (7/24) | 28.6% (10/35) | 33.3% (12/36) |
| 6 | 16.7% (5/30) | 23.1% (3/13) | 26.1% (6/23) | 39.1% (9/23) |
| 4–6 | 23.3% (17/73) | 32.2% (29/90) | 26.2% (28/107) | 31.3% (36/115) |
| ≥7 | 37.1% (65/175) | 59.7% (37/62) | 47% (47/100) | 49.2% (31/63) |
CCOTCritical Care Outreach TeamNEWS2National Early Warning Score 2REDSRisk-stratification of Emergency Department suspected SepsisSOFASequential Organ Failure Assessment
Table 3. Test characteristics at the preferred cut-off point for each score and combination of scores.
| Score | Cut-off score | Sensitivity (95% CI) | Specificity (95% CI) | PPV (95% CI) | NPV (95% CI) |
| NEWS2 | ≥10 | 36.3% (26.4 to 47.0) | 82.8% (76.8 to 87.8) | 49.3% (39.2 to 59.4) | 78.9% (70.5 to 77.0) |
| NEWS2 | ≥7* | 71.4% (61 to 80.4) | 44.4% (37.4 to 51.7) | 37.1% (26.2 to 37.2) | 77.2% (70.2 to 82.9) |
| NEWS2 | ≥5* | 81.3% (71.8 to 88.7) | 23.7% (18.0 to 30.3) | 32.9% (26.2 to 37.2) | 73.4% (62.7 to 82.0) |
| REDS | ≥4 | 72.5% (62.2 to 81.4) | 56.6% (49.4 to 63.6) | 43.4% (38.5 to 48.5) | 81.8% (75.8 to 86.5) |
| SOFA | ≥7 | 51.7% (40.9 to 62.3) | 73.2% (66.5 to 79.3) | 47.0% (39.6 to 54.6) | 76.7% (72.4 to 80.6) |
| ΔSOFA | ≥5 | 57.1% (46.3 to 67.5) | 64.7% (57.6 to 71.3) | 42.6% (36.4 to 49.1) | 76.7% (71.7 to 81.0) |
| ΔSOFA | ≥2* | 91.2% (83.4 to 96.1) | 15.7% (10.9 to 21.5) | 33.2% (31.3 to 35.2) | 79.5% (65.0 to 89.0) |
| NEWS2≥7 or REDS≥4 | 87.9% (79.4 to 93.8) | 29.3% (23.1 to 36.2) | 36.4% (33.7 to 39.1) | 84.1% (74.4 to 90.5) | |
| NEWS2≥5 or REDS≥4 | 91.2% (83.4 to 96.1) | 18.2% (13.1 to 24.3) | 33.9% (31.9 to 36) | 81.8% (68.6 to 90.3) | |
| NEWS2≥7 or SOFA≥7 | 78.0% (68.1 to 86.0) | 36.4% (29.7 to 43.5) | 36.0% (32.6 to 39.6) | 78.3% (70.1 to 84.7) | |
| NEWS2≥5 or SOFA≥7 | 83.5% (74.3 to 90.5) | 20.7% (15.3 to 27.0) | 32.6% (30.1 to 35.2) | 73.2% (61.5 to 82.4) | |
| NEWS2≥7 or ΔSOFA≥5 | 80.2% (70.6 to 87.8) | 36.7% (29.7 to 43.5) | 36.6% (33.4 to 38.9) | 80.0% (71.8 to 86.3) | |
| NEWS2≥5 or ΔSOFA≥5 | 85.7% (76.8 to 92.2) | 20.2% (14.8 to 26.5) | 33.1% (30.7 to 35.5) | 75.5% (63.4 to 84.5) | |
pPre-specified cut-off score; Interval.
NEWS2National Early Warning Score 2NPVnegaitive predictive valuePPVpositive predictive valueREDSRisk-stratification of Emergency Department suspected SepsisSOFASequential Organ Failure Assessment
Discussion
In this study of hospitalised patients who deteriorated and were transferred to the ICU, we found that the REDS, SOFA and NEWS2 scores were predictive for all-cause in-hospital mortality. The REDS score had the largest AUROC curve at 0.70 (95% CI 0.65 to 0.75), significantly greater than the NEWS2 score at 0.62 (95% CI 0.56 to 0.68), p=0.03, but similar to that of the SOFA score 0.67 (95% CI 0.61 to 0.72), p=0.3. The cut-off points identified for the REDS, SOFA and NEWS2 scores were >3, >6 and >9, respectively. Combining the use of a NEWS2 score of ≥7 or a REDS score of ≥4 had a sensitivity for mortality which was similar to that of a ΔSOFA score of ≥2 but with a significantly greater specificity.
Analysis of the demographic, physiological parameters and blood results between survivors and non-survivors presented in table 1 reveals the expected differences in the parameters that form the REDS and SOFA scores. Figure 2 illustrates the difference in the three scores between survivors and non-survivors at the point of admission and the point of referral to CCOT. Both the REDS and SOFA scores differ significantly between survivors and non-survivors at both time points. However, no difference in NEWS2 score is noted at the point of admission, although there was a difference at the point of referral to CCOT. This observation suggests that significant and prognostic organ dysfunction and changes to blood markers may precede changes to the NEWS2 score. This view is supported by the fact that there was no statistical difference in the baseline SOFA scores between survivors and non-survivors. The REDS score has been externally validated in two small studies in ED patients with suspected sepsis.22 23 Previous comparisons between the REDS, SOFA and NEWS2 scores in patients with suspected sepsis in the ED revealed findings similar to our study where the REDS score had a similar AUROC curve to the SOFA scores but significantly greater than that of the NEWS2 score,16 24 25 despite the current study population being of unselected disease profile. This is the first study validating the REDS score in a population not restricted to those suspected of having sepsis.
We found that a cut-off point of ≥7 in the SOFA score at the point of admission to the ICU was associated with in-hospital mortality. This finding is similar to that of Fuchs et al13 and Do et al,14 who noted the cut of points at admission to the ICU to be 7 and 7.5, respectively. However, at cut-off point of ≥7, we found that the sensitivity is only around 50%. Analysis of the ΔSOFA score revealed that an increase of ≥5 compared with baseline was associated with mortality but again the sensitivity was low at <60%. However, an increase of ≥2 points in the ΔSOFA score from baseline had the highest sensitivity of >90% but also the lowest specificity. While the calculation of the ΔSOFA score would take into account of the patient’s pre-existing organ dysfunction, it complicates the use of the SOFA score by making it more burdensome by necessitating the calculation of the baseline SOFA score in addition to the current score.
Both survivors and non-survivors deteriorated between admission and the point of referral to CCOT. The deterioration was associated with higher scores in non-survivors, suggesting that the referral’s timing is important. Lower scores were associated with lower mortality rates, but referring every patient with a low score to CCOT is impractical. Adding a REDS score of ≥4 to a NEWS2 score of ≥7 as a referral criterion to CCOT would significantly improve the sensitivity for mortality. We anticipate clinicians using the REDS score as an additional objective tool to refer patients to CCOT.
Introducing a new scoring system across the hospital among many staff can be challenging. The NEWS2 score is a nationally understood common language but it has its critics. Vardy et al26 argue for using clinical judgement with NEWS2, and the Clinical Frailty Scale,18 to support better recognition of clinical deterioration, especially in the elderly. The REDS score uses data from multiple sources (physiological, blood results and point-of-care blood results) and training staff to assimilate data from disparate sources can be challenging. A possible solution could be developing an IT system to fully collate the data required for the REDS score, as it would remove some of the burden from the clinician. There would also be potential benefits for the REDS, SOFA and NEWS2 combined to be the basis for artificial intelligence to further learn from our deteriorating patients and develop more patient-specific and earlier warning systems.
The aim is not to remove the bedside clinician from the chain of response but to provide more tools to protect patients without further task burden. How the REDS score would fit within CCOT referral criteria is also key when considering the practical application. Alam et al27 commented that many studies evaluating the early warning score (EWS) recognise that how the score triggers a CCOT review is also integral to the outcome.
Lonsdale et al28 highlighted the requests put to clinicians to predict outcomes in relation to balancing ICU admission decision-making. This raises the question—would the REDS score be useful to CCOT when making admission decisions as another tool to aid in this complex process? These decisions are often time-critical, and a variety of opinions can exist. Having tools to assist with decision-making, not set criteria or rules, supports clinicians challenged with nuanced differences in patients and the particular hospital resources within and out of critical care. Lonsdale et al28 also noted that mortality might not be the most important outcome to measure. Again, the argument for using the REDS score is that using it along with the NEWS2 gives the CCOT more information to guide them. Further work could consider whether this added data change decisions, as any changes to EWS need to be clinically and cost effective.
Limitations
This study has several limitations. First, it is an observational cohort study. Such studies are at risk of bias from an unaccounted confounder. Second, a NEWS2 score of ≥7 mandates (although a referral of a NEWS≥5 is advised) a referral to the CCOT team. This may explain the high sensitivity for mortality with NEWS2≥7. Third, the study’s retrospective nature means that not all data were available at the referral point. This was managed by using the blood results at admission to the ICU, allowing complete case analysis. Fourth, selection bias may exist as we only studied patients admitted to the ICU. Patients referred but not transferred to ICU were not studied. Fifth, diagnosing RH may be challenging on the wards. We have used the initiation of vasopressors as a criterion for confirming the presence of RH on the wards. Sixth, we limited our outcome to in-hospital mortality. Half of the survivors were discharged within 30 days of admission. It is not known if any died shortly after discharge within 30 days of admission.
'Finally, REDS scould also stand for 'Risk-stratification of Emergency DeteriorationS.
Conclusion
The performance of the REDS score to prognosticate in-hospital all-cause mortality in undifferentiated hospitalised patients who deteriorated and were transferred to the ICU was similar to that of the SOFA score but greater than the NEWS2 score. The REDS score can be used in addition to the well-established NEWS2 score to escalate care to the CCOT.
Footnotes
Funding: The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Provenance and peer review: Not commissioned; externally peer reviewed.
Patient consent for publication: Not applicable.
Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.
Ethics approval: This service evaluation of routinely collected anonymised data did not include an intervention nor change the normal care process. In addition, it is a retrospective study from a single centre and, as such, lacks generalisability. In accordance with the guidance of the National Health Service (NHS), Health Research Authority (HRA) and Medical Research Council (MRC), such studies do not require formal ethics approval. As per the National Health Research Authority guidance around the General Data Protection Regulation and the Data Protection Act of 2018, patient consent is not required for anonymised data. Due to the retrospective analysis of routinely collected anonymised data, patient consent was not required. This study was registered with the Clinical Effectiveness and Audit office of St George’s University Hospitals NHS FT, under the registration code AUDI003413.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
References
- 1.Escobar GJ, Liu VX, Schuler A, et al. Automated Identification of Adults at Risk for In-Hospital Clinical Deterioration. N Engl J Med . 2020;383:1951–60. doi: 10.1056/NEJMsa2001090. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Harris S, Singer M, Rowan K, et al. Delay to admission to critical care and mortality among deteriorating ward patients in UK hospitals: a multicentre, prospective, observational cohort study. Lancet. 2015;385 Suppl 1:S40. doi: 10.1016/S0140-6736(15)60355-5. [DOI] [PubMed] [Google Scholar]
- 3.Honarmand K, Wax RS, Penoyer D, et al. Executive Summary: Society of Critical Care Medicine Guidelines on Recognizing and Responding to Clinical Deterioration Outside the ICU. Crit Care Med. 2024;52:307–13. doi: 10.1097/CCM.0000000000006071. [DOI] [PubMed] [Google Scholar]
- 4.Vincent J-L, Einav S, Pearse R, et al. Improving detection of patient deterioration in the general hospital ward environment. Eur J Anaesthesiol. 2018;35:325–33. doi: 10.1097/EJA.0000000000000798. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Jones M. NEWSDIG: The National Early Warning Score Development and Implementation Group. Clin Med (Northfield) 2012;12:501–3. doi: 10.7861/clinmedicine.12-6-501. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.RCP National early warning score (news) 2 |
- 7.Welch J, Dean J, Hartin J. Using NEWS2: an essential component of reliable clinical assessment. Clin Med (Lond) 2022;22:509–13. doi: 10.7861/clinmed.2022-0435. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.The technology . NICE; 2024. National early warning score systems that alert to deteriorating adult patients in hospital | advice |. [Google Scholar]
- 9.Gerry S, Bonnici T, Birks J, et al. Early warning scores for detecting deterioration in adult hospital patients: systematic review and critical appraisal of methodology. BMJ. 2020;369:m1501. doi: 10.1136/bmj.m1501. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Blackwell JN, Keim-Malpass J, Clark MT, et al. Early Detection of In-Patient Deterioration: One Prediction Model Does Not Fit All. Crit Care Explor. 2020;2:e0116. doi: 10.1097/CCE.0000000000000116. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Vincent J-L, Moreno R, Takala J, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. Intensive Care Med. 1996;22:707–10. doi: 10.1007/BF01709751. [DOI] [PubMed] [Google Scholar]
- 12.Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) JAMA. 2016;315:801. doi: 10.1001/jama.2016.0287. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Fuchs PA, Czech IJ, Krzych ŁJ. Mortality Prediction Using SOFA Score in Critically Ill Surgical and Non-Surgical Patients: Which Parameter Is the Most Valuable? Medicina (Kaunas) 2020;56:273. doi: 10.3390/medicina56060273. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Do SN, Dao CX, Nguyen TA, et al. Sequential Organ Failure Assessment (SOFA) Score for predicting mortality in patients with sepsis in Vietnamese intensive care units: a multicentre, cross-sectional study. BMJ Open. 2023;13:e064870. doi: 10.1136/bmjopen-2022-064870. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Sivayoham N, Blake LA, Tharimoopantavida SE, et al. The REDS score: a new scoring system to risk-stratify emergency department suspected sepsis: a derivation and validation study. BMJ Open. 2019;9:e030922. doi: 10.1136/bmjopen-2019-030922. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Sivayoham N, Hussain AN, Shabbo L, et al. An observational cohort study of the performance of the REDS score compared to the SIRS criteria, NEWS2, CURB65, SOFA, MEDS and PIRO scores to risk-stratify emergency department suspected sepsis. Ann Med. 2021;53:1863–74. doi: 10.1080/07853890.2021.1992495. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.stgeorges.nhs.uk; STGEORGE-annual-report-and-accounts-2021-digi-1.pdf. [Google Scholar]
- 18.Rockwood K, Song X, MacKnight C, et al. A global clinical measure of fitness and frailty in elderly people. CMAJ. 2005;173:489–95. doi: 10.1503/cmaj.050051. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Grissom CK, Brown SM, Kuttler KG, et al. A modified sequential organ failure (MSOFA) score for critical triage. Disaster Med Public Health Prep. 2010;4 doi: 10.1001/dmp.2010.40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Peduzzi P, Concato J, Kemper E, et al. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49:1373–9. doi: 10.1016/s0895-4356(96)00236-3. [DOI] [PubMed] [Google Scholar]
- 21.DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837–45. [PubMed] [Google Scholar]
- 22.Wang L, Sun Z. Predictive value of emergency suspected sepsis risk stratification score for prognosis of patients with sepsis. Clin Med Chin. 2021;37:264–9. [Google Scholar]
- 23.Benchalaoharatana C, Somboonsit P, Durongbhandu T. Prediction of mortality by the REDS score in sepsis patients at emergency department. Chonburi Hosp J. 2023;48:129–36. [Google Scholar]
- 24.Sivayoham N, Hussain AN, Sheerin T, et al. Prognostic performance of the REDS score, SOFA score, NEWS2 score, and the red-flag, NICE high-risk, and SIRS criteria to predict survival at 180 days, in emergency department patients admitted with suspected sepsis - An observational cohort study. Front Med (Lausanne) 2023;10:985444. doi: 10.3389/fmed.2023.985444. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Sheerin T, Dwivedi P, Hussain A, et al. Performance of the CURB65, NEWS2, qSOFA, SOFA, REDS, ISARIC 4C, PRIEST and the Novel COVID-19 Severity Scores, Used to Risk-Stratify Emergency Department Patients with COVID-19, on Mortality—An Observational Cohort Study. COVID . 2023;3:555–66. doi: 10.3390/covid3040040. [DOI] [Google Scholar]
- 26.Vardy E, Lasserson D, Barker RO, et al. NEWS2 and the older person. Clin Med (Northfield) 2022;22:522–4. doi: 10.7861/clinmed.2022-0426. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Alam N, Hobbelink EL, van Tienhoven AJ, et al. The impact of the use of the Early Warning Score (EWS) on patient outcomes: a systematic review. Resuscitation. 2014;85:587–94. doi: 10.1016/j.resuscitation.2014.01.013. [DOI] [PubMed] [Google Scholar]
- 28.Lonsdale DO, Tong L, Farrah H, et al. The clinical frailty scale - does it predict outcome of the very-old in UK ICUs? J Intensive Care Soc. 2023;24:154–61. doi: 10.1177/17511437211050789. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.hra-decisiontools.org.uk; [1-Aug-2024]. Do i need nhs ethics approval.http://www.hra-decisiontools.org.uk/ethics/ Available. Accessed. [Google Scholar]
- 30.Health Research Authority Decision Tools. [1-Aug-2024]. http://www.hra-decisiontools.org.uk/research/docs/DefiningResearchTable_Oct2017-1.pdf Available. Accessed.
- 31.hra.nhs.uk; [1-Aug-2024]. Definitions - health research authority.https://www.hra.nhs.uk/planning-and-improving-research/policies-standards-legislation/dataprotection-and-information-governance/gdpr-guidance/definitions Available. Accessed. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
All data relevant to the study are included in the article or uploaded as supplementary information.