Failure of vital sign normalization is more strongly associated than single measures with mortality and outcomes

Abstract

Background

Modified Early Warning Systems (MEWS) scores offer proxies for morbidity and mortality that are easily acquired, but there are limited data on what changing MEWS scores within the ED indicate. We examined the correlation of changing MEWS scores during resuscitation in the ED and in-hospital morbidity and mortality.

Methods

We conducted a retrospective analysis on medical ED patients with simplified MEWS scores (without urine output or mental status) admitted to a single academic tertiary care center over one year. Triage-to-Last delta MEWS score and Triage-to-Max delta MEWS scores were calculated and correlated to in-hospital mortality, ICU admission, length of stay (LOS) and diagnosis of sepsis.

Results

Our analysis included 8,322 ED patients with an ICU admission rate of 17% and a mortality rate of 2%. Every point of worsened MEWS after triage was more strongly associated with all-cause mortality (OR 2.41, 95% CI 1.96 – 2.97) than triage MEWS alone (OR 1.33, 95% CI 1.23–1.44; p<0.001). Likewise, each point of worsened MEWS was associated with increased odds of ICU admission (Triage-to-Last: OR 2.12, 95% CI 1.92 – 2.33 and Triage-to-Max: OR 1.52, 95% CI 1.45–1.60, respectively). Among patients with suspected infection, similar associations are found.

Conclusions

Dynamic vital signs in the emergency department, as categorized by delta MEWS, and failure to normalize abnormalities, were associated with increased mortality, ICU admission, LOS, and the diagnosis of sepsis. Our results suggest that MEWS scores that do not normalize, from triage onward, are more strongly associated with outcome than any single score.

1. INTRODUCTION

Sepsis contributes to significant inpatient mortality and is the most expensive cause of hospitalization in the US, at more than $20 billion dollars and contributing 1.6 million inpatient hospitalizations annually.^1–6 Patients who are not identified until after hospital admission substantiate an even larger proportion of this burden.⁷ These delays in identification are often due to occult presentation of sepsis. Among patients with septic shock requiring vasopressors, many are actually normotensive and have non-specific complaints at emergency department (ED) triage.⁸ Early identification of these patients therefore is an area of active research and inquiry.

Modified Early Warning Systems (MEWS) scores as single measures have been used to identify clinically deteriorating patients and those with increased probability of developing sepsis.⁹ MEWS vary between institutions but typically are composite scores of vital signs and mental status. MEWS facilitate recognition of occult deterioration through objective quantification of the magnitude of vital sign perturbations.¹⁰ In the inpatient setting and at ED triage, the magnitude of static vital sign abnormalities, quantified by MEWS scores, have been associated with hospital admission, disposition location and mortality,^11–14 but there is a paucity of research on what changing vital signs and changing MEWS in the ED mean prior to clinical deterioration.¹⁵

One study demonstrated that pre-hospital hemodynamic variability was associated clinical deterioration.¹⁶ In a second study, ED blood pressure variability improved outcome prediction compared to single measurements, but in the same study, repeated measures of other vital sign interestingly failed to improve prognostic ability over single measures.¹⁷ Clinical trials of heart rate variability are ongoing in a variety of disease processes and show promise,¹⁸ but beyond this, it remains unknown if changing MEWS scores within the ED correlate with outcome, or improve outcome prediction compared to single/static measures.

Herein, we investigated the relationship between changing MEWS scores in the ED and the patient outcomes of sepsis, intensive care unit (ICU) admission, inpatient mortality and hospital length of stay (LOS). In this study, our objective was to determine the relationships between dynamic MEWS scores while in the ED and outcomes among medical patients admitted to the hospital.

2. METHODS

Our analysis is reported according to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Guidelines.¹⁹

2.1. Data Sharing

To facilitate research reproducibility, replicability, accuracy and transparency, the datasets generated and/or analyzed during the current study, and the associated analytic code, will be made available indefinitely [DOI 10.17605/OSF.IO/ESJ9K], following publication, to anyone who wishes to access the data for analysis, on the Open Science Foundation²⁰ (OSF) repository at [https://osf.io/esj9k/]. Data were de-identified in accordance with Section 164.514 of the Health Insurance Portability and Accountability Act (HIPAA).

2.2. Data Source and Study Population

Data were obtained from the electronic data warehouse at a single tertiary academic medical center in a manner which has been previously described.²¹ Patients were identified if they were seen in the emergency department from November 2017 to November 2018 and were subsequently admitted to the hospital during the index visit. All data included were collected as part of routine clinical care.²² Comprehensive data capture of clinical and outcome variables is ensured through centralized documentation in the electronic health record [EPIC Systems®, Verona, WI], ongoing routine quality audits in the emergency department and hospital billing charges. Data was extracted from the data warehouse by a trained data scientist blinded to the goals of the analysis. Data was complete for all patients.

We included all patients ≥18 years of age at the time of presentation to the ED. In order to focus this preliminary analysis on patients with medical diagnoses, including infectious etiologies of their hemodynamic alterations (such as sepsis), rather than on clearly traumatic, burn or orthopedic etiologies, we excluded patients admitted to the trauma, orthopedic or burn units.

2.2.1. Implementation of automated MEWS tracking and threshold determination

Prior to initiation of our research study, in 2015 the institution implemented a hospital wide quality improvement (QI) project wherein a variation of MEWS (excluding urine output and mental status) were automatically calculated by the electronic medical record (EMR) for all patients and displayed on the electronic patient board to facilitate provider recognition of deteriorating patients.²² For ease of automated EMR calculation, our institution chose this variation of MEWS that was composed entirely of objective vital signs collected in real time as opposed to the standard MEWS or other scores that include subjective assessments, laboratory data, or urine output. For the purposes of this manuscript, MEWS refers to this variation without urine output and mental status.

Institutional MEWS are defined in FIGURE 1. MEWS (min: 0, max: 12) quantify the degree of vital sign abnormalities within a patient and institutional estimates of inpatient mortality; scores comprise of temperature, respiratory rate, heart rate and systolic blood pressure. Vital sign ranges for scores were based on published literature and institutional expert and multidisciplinary agreement as part of the QI project.²³ After retrospective analysis of the MEWS vital sign ranges applied to 10,746 inpatient visits, MEWS score characteristics were determined that then allowed estimation of the number of alerts per unit per week, the mortality associated with each MEWS score, and the approximate sensitivity and specificity for identifying sepsis according to ICD-9 and ICD-10 diagnosis codes (Supplemental Figure 1, Supplemental Figure 2). This data was then presented to stakeholders throughout the institution and thresholds were mutually decided upon that when surpassed would trigger the EMR to send pager-based alerts to providers in order to facilitate recognition and response to septic and decompensating patients.

Figure 1:

Utah Modified Early Warning System

2.3. Study Variables and Outcomes

Our primary outcome is mortality during hospital admission. Secondary outcomes include need for ICU admission, diagnosis of sepsis, and hospital length of stay (LOS; in 24 hour increments). Our primary predictors include Triage-to-Last delta MEWS, Triage-to-Max delta MEWS, and as single measures: triage MEWS, max MEWS, and last MEWS. Covariates included age, sex, severity of illness (Charlson Comorbidity Index [CCI]), influenza diagnosed [yes/no] during hospital stay, sepsis diagnosed [yes/no] during hospital stay, triage presentation date/time, ED entry date/time, admission unit, ED MEWS scores with date/time (triage, maximum, last), hospital unit admission date/time, hospital discharge date/time, disposition location (home, home health care, skilled nursing facility, long term acute care, left against medical advice, hospice, death). The diagnosis of sepsis was determined after two separate occurrences of the clinical diagnosis of sepsis within an attending physician’s note.

We additionally did a subgroup analysis for patients in whom there was suspected infection. We defined these patients as those who received antibiotics while in the ED and repeated the analysis among this group with the same covariates and outcome measures.

2.3.1. Delta MEWS calculations

Triage-to-Last delta MEWS is calculated by subtracting the patient’s last MEWS before leaving the ED from their triage MEWS (first – last). Triage-to-Last delta MEWS ranged from −8 (patient trended from MEWS of 0 [normal] to MEWS of 8 [abnormal], i.e., clinically worsened) to 8 (patient trended from MEWS of 8 [abnormal] to MEWS of 0 [normal], i.e., clinically improved). A Triage-to-Max delta MEWS is calculated by subtracting the patient’s max MEWS while in the ED from their triage MEWS (first – max). Triage-to-Max delta MEWS ranged from −8 (patient trended from MEWS of 0 [normal] to MEWS of 8 [abnormal], i.e., clinically worsened) to 0 (patient first MEWS of 8 [abnormal] was also their max MEWS, i.e., clinically unchanged, or improved). Scores were then scaled for analysis and incremental changes and direction were compared.

We constructed a theorized causal pathway with directed acyclic graphs using Dagitty,²⁴ which is published on the OSF. Our analysis was exploratory (i.e., postdiction),²⁵ and the results should be considered hypothesis-generating, requiring reproduction with other data.

2.4. Statistical Analysis

Our primary goal is to analyze the association of changing vital signs, as categorized by the MEWS score, with outcomes, among medical patients. We first describe baseline clinical and demographic characteristics of all patients. Descriptive statistics, including counts and percent for binary variables, means [standard deviation; SD] for continuous variables, and medians [interquartile range; IQR] for highly skewed continuous variables, were used to assess these characteristics. Categorical characteristics were compared using chi-square test or Fisher’s exact test. Continuous characteristics were compared using independent samples t-test or Wilcoxon-Mann-Whitney test. Primary and secondary analyses were assessed using multivariate regression (logistic for binary [mortality, ICU admission], linear for continuous [hospital LOS]) after adjusting for the covariates age, sex, CCI, time in the ED. For models with a primary predictor of delta MEWS, we additionally adjusted for triage MEWS score. Coefficients, 95% CI’s and p-values were reported from all models. Statistical analyses were conducted in STATA 15.1 (College Park, TX), significance was assessed at the 0.05 level and all tests were two-tailed. This study was approved by the Institutional Review Board (IRB) under #00096120 on January 13^th, 2017.

3. RESULTS

We identified 8,322 ED patients meeting inclusion/exclusion criteria. Patient characteristics are outlined in Table 1, but briefly, patients were 56 years old [SD, 17.8] and 49% female. Twenty-four percent were diagnosed with sepsis during their admission, the median CCI was 3 (IQR 1,7), 17% were admitted to the ICU, and the in-hospital mortality rate was 2%. One thousand three hundred and sixteen patients received antibiotics while in the ED and were included in the subgroup analysis for patients in whom infection was suspected.

Table 1.

Baseline Characteristics

	n (%)
Gender
Male	4,276 (51.4)
Female	4,046 (48.6)
Age, mean (SD)	56.4 (17.8)
Flu	151 (1.8)
Died	167 (2.0)
Sepsis	2,035 (24.4)
Admitted to ICU	1,443 (17.3)
Length of stay, days, median (IQR)	2.8 (1.7–4.7)
Charlson Comorbidity Index, median (IQR)	3 (1–7)
Triage MEWS Score, median (IQR)	1(0–2)
Max MEWS Score, median (IQR)	2 (1–3)
Last MEWS Score, median (IQR)	0 (0–1)
Time Triage to Max MEWS, mean (SD)	84.5 (3.4)
Time Triage to Last MEWS, mean (SD)	335.4 (5.5)

3.1. Mortality

In adjusted analysis, each point of worsened MEWS before hospital admission (Triage-to-Last delta MEWS), adjusted to triage MEWS, was associated with an increased risk of all-cause mortality (OR 2.41, 95% CI 1.96 – 2.97) (Table 2, Figure 2). This was additionally true for any worsening MEWS after triage (Triage-to-Max delta MEWS; OR 1.43, 95% CI 1.29–1.63). Consistent with previous studies, triage MEWS alone was associated with mortality (OR 1.33, 95% CI 1.23–1.44), as was last MEWS alone (OR 1.62, 95% CI 1.49–1.76). In a sensitivity analysis, removing ED LOS as a covariate resulted in a OR for Triage to Last delta MEWS of 2.50 (95%CI 2.03–3.07).

Table 2.

Adjusted Probabilities of Death, ICU Admission, Sepsis and Hospital Length of Stay

	Death	ICU Admission	Sepsis	LOS among Survivors
	OR (95% CI) p-value	OR (95% CI) p-value	OR (95% CI) p-value	β coeff (95% CI) p-value
Triage MEWS**	1.33 (1.23–1.44) p<0.000	1.37 (1.33–1.42) p<0.000	1.71 (1.65–1.77) p<0.000	0.34 (0.27–0.41) p<0.000
Max MEWS**	1.41 (1.31–1.51) p<0.000	1.47 (1.42–1.52) p<0.000	1.80 (1.74–1.87) p<0.000	0.35 (0.29–0.41) p<0.000
Last MEWS**	1.62 (1.49–1.76) p<0.000	1.60 (1.54–1.67) p<0.000	1.75 (1.68–1.83) p<0.000	0.50 (0.42–0.58) p<0.000
Triage-to-Last delta MEWS*	2.41 (1.96–2.97) p<0.000	2.12 (1.92–2.33) p<0.000	1.99 (1.81–2.18) p<0.000	0.78 (0.60–0.97) p<0.000
Triage-to-Max delta MEWS*	1.45 (1.29–1.63) p<0.000	1.52 (1.45–1.60) p<0.000	1.68 (1.61–1.78) p<0.000	0.29 (0.20–0.39) p<0.000

ICU: Intensive Care Unit

LOS: Length of Stay

OR: odds ratio

CI: Confidence Interval

CCI: Charlson Comorbidity Index

^*Adjusted for age, sex, CCI, triage MEWS score

^**Adjusted for age, sex, CCI

Figure 2:

Probability of Death

3.2. ICU admission

Each point of worsened MEWS before hospital admission, compared to triage (Triage-to-Last delta MEWS), was associated with increased odds of ICU admission (2.12, 95% CI 1.92–2.33)(Table 2, Figure 3). Likewise, each point of worsening MEWS score after triage (Triage-to-Max delta MEWS) was associated with increased odds of ICU admission (OR 1.52, 95% CI 1.45 – 1.60).

Figure 3:

Probability of ICU Admission

3.3. LOS

Among survivors (n=8,155), for every point improvement in triage MEWS score before hospital admission (Triage-to-Last delta MEWS), there was an 18 hour decrease in hospital LOS (β coeff 0.78, 95% CI 0.60 to 0.97; Supplemental Figure 3, Table 2, expressed as increase in LOS with increasing scores [(β coeff x 24 hour increment]). As expected, every point increase in MEWS score after triage (Triage-to-Max delta MEWS) was likewise associated with a 0.29 increment (~7 hours) increase in hospital length of stay (β coeff 0.29; 95% CI 0.20 – 0.39). This was comparable to the LOS increases associated with increases in static scores alone. Last MEWS score carried the second largest associated increase in likelihood of LOS (β coeff 0.50; 95% CI 0.42 – 0.58).

3.4. Development of Sepsis

Every point of worsened MEWS before hospital admission was strongly associated with the development of sepsis (OR 1.99, 95% CI: 1.81–2.18) (Table 2, Figure 4).

Figure 4:

Probability of Development of Sepsis

3.5. Suspected Infection

In a sensitivity analysis among the patients who received antibiotics while in the ED, the associations between the various MEWS scores and outcomes were unchanged (Supplemental Table 3).

3.6. Comparison of Models

3.6.1. Mortality

When comparing how well the different models and their scores perform at predicting inpatient mortality (Supplemental Table 2), as assessed by the area under the receiver operating characteristic (AUROC), Triage-to-Last delta MEWS score was superior to Triage MEWS alone (AUROC 0.802 vs 0.767; Figure 5), to Max MEWS score alone (AUROC 0.802 vs 0.786) and to Triage-to-Max delta MEWS (AUROC 0.802 vs 0.786) but not superior to Last MEWS alone (AUROC 0.802 vs 0.801).

Figure 5:

Single vs delta MEWS Scores for the prediction of all-cause mortality—Triage vs Triage-to-Last

Triage-to-Max delta MEWS was superior to triage MEWS alone (AUROC 0.786 vs 0.767), but comparable to Max MEWS (AUROC 0.786 vs 0.786) and Last MEWS (AUROC 0.786 vs 0.801) alone for the outcome of mortality.

3.6.2. ICU Admission

For ICU admission, overall, Triage-to-Last delta MEWS and Triage-to-Max delta MEWS were numerically similar to static Max MEWS scores, but were superior to Triage and Last MEWS scores alone (AUROC [0.731 vs 0.704] and [0.734 vs 0.725]) and (AUROC [0.731 vs 0.704] and [0.734 vs 0.725]) (Supplemental Table 2).

3.6.3. Sepsis

For the diagnosis of sepsis, Triage-to-Last and Triage-to-Max delta MEWS scores were superior to static scores for all comparisons (Supplemental Table 2).

3.6.4. Suspected Infection

In a sensitivity analysis among the patients who received antibiotics while in the ED, similar comparisons between the various MEWS were observed (Supplemental Table 3).

4. DISCUSSION

In the emergency department setting it can be difficult to prognosticate a patient’s clinical trajectory both during and after their stay in the ED. While emergency physicians are trained to identify and correct significant vital sign abnormalities such as profound hypotension or persistent tachycardia, subtle vital sign changes often go unnoticed. Quantification of vital sign abnormalities through clinical decision-making tools such as Quick Sepsis-related Organ Failure Assessment (qSOFA) and MEWS facilitates recognition of these abnormalities.¹⁰ There is a large body of data demonstrating that the magnitude of these vital sign abnormalities at triage and at hospital admission, quantified by these clinical decision tools or early warning scores, is also associated with outcome.^12,26 Vital sign abnormalities collected in the first minutes after ED admission can identify patients at risk of an unfavorable outcome.²⁷ Furthermore, electronic track-and-trigger systems utilizing early warning scores have reduced triage-to-diagnosis and diagnosis-to-antibiotic times in patients with sepsis,²⁸ which has itself been associated with improved mortality.²⁹ In sum, these tools offer guidance in identifying those who are currently septic or at higher risk for inpatient morbidity and mortality. Unfortunately, all are also single static data points that do not incorporate the patient’s trend in their current clinical course.

Repeated measures of vital signs enable assessment of dynamic hemodynamic changes. These assessments are appealing as they should correlate with efficacy of resuscitation and outcome. Though there are limited studies examining this, clinical trials are ongoing. Among studies of repeated vital signs or scores, a recent large retrospective analysis from 12 hospitals found that repeated measurements of qSOFA improved predictive validity for mortality among hospitalized septic patients compared with a single measurement.³⁰ Our study is distinct in that MEWS is automatically calculable and does not require the subjective assessment of mental status inherent in qSOFA. Among comparable studies analyzing objective measurements, the most comparable was a study of febrile or suspected septic emergency department patients (n=359); it found that only mean blood pressure variability, but not heart rate (HR), respiratory rate (RR), or temperature variability, was superior to static measurements for outcome prediction.¹⁷ A pre-hospital study (n=2,586) among non-trauma patients demonstrated that changes in some vital signs (GCS, RR, SBP, pulse pressure, Shock Index) between prehospital and in-hospital was associated with an increased risk of in-hospital mortality.¹⁶ As it has been demonstrated that commonly used early warning systems have been found to be more accurate than the qSOFA score for predicting death and ICU transfer, we believe the objective, vital sign based MEWS offers ideal characteristics for automated serial assessments in the emergency department.^31,32

By utilizing repeated MEWS scores (delta MEWS), we were able to capture variation in vital sign abnormalities and explore the association with outcome. Within our study of 8,322 emergency department patients, we found significantly greater predictive value in the degree of change between serially collected MEWS scores compared to static scores at triage for most outcomes, with an AUROC of Triage-to-Last MEWS for mortality of 0.802. In comparison, the diagnostic performance of positive qSOFA score for predicting 28-day mortality was low in critically ill septic patients, particularly during the early period after ED presentation. Positive qSOFA for predicting 28-day mortality increased from AUROC of 0.58 at arrival to 0.61 within 6 hours.³³ Our results should be interpreted cautiously. We did not compare our model to other scores, nor to their covariates. Our results should be interpreted to suggest that failure to normalize MEWS scores, from triage onward, was strongly associated with outcome, and was superior to many static scores during emergency department stay. Each point worsening of MEWS scores—either from initial triage to max (Triage-to-Max delta MEWS) or before hospital admission (Triage-to-Last delta MEWS)—was associated with increased mortality, ICU admission, and diagnosis of sepsis.

In our analysis, delta scores were more strongly associated in all comparisons to single MEWS values alone for all outcomes. For the prediction of mortality, ICU admission, and diagnosis of sepsis, models utilizing delta MEWS scores were superior to many but not all other models. Specifically, delta scores were variously comparable to Max MEWS score and Last MEWs scores alone. Our findings are important and novel in that they suggest that dynamic changes in hemodynamic status during ED stay, as quantified by MEWS scores, are more strongly associated with these outcomes (survival, ICU admission, sepsis) than the static condition upon initial presentation. Our findings also suggest that the final condition upon leaving the ED (Last MEWS) is, in this analysis, as predictive of outcome as changes during the ED stay.

4.1. Future Directions

MEWS are a simple and easy to use bedside tool. Implementation of MEWS to guide clinical decisions has been inconsistently associated with improved patient outcomes.³⁶ This sentiment was echoed in another recent review that found a lack of high-quality comparative studies.³⁷ To overcome this gap, prospective multicenter studies are needed that test the durability of these findings in other settings and assess the effect of automated MEWS-based decisions on patient outcomes. Accordingly, we encourage researchers to validate our methodology and findings within other settings.

4.2. Limitations

This study was conducted at a single tertiary-care institution that sees a significant number of medically complex patients and is a referral hospital for a very large geographic area (Utah, Idaho, Wyoming, Montana, Northern Nevada). For these reasons the findings in this study may not be applicable to every ED. This study was limited by not having follow-up information for patients discharged from the ED. If these patients died, or returned to another hospital, we did not capture that. It is known that the rate of death after ED discharge is generally low, but this remains probabilistic in our analysis. While we demonstrated the models utilizing delta MEWS had a significantly larger association with the outcomes than single measures (2.4 vs 1.6), the mortality model’s predictive ability was only marginally improved (AUROC improved 0.03); this is unsurprising considering the AUROC is a measure of model prediction ability, and the models are otherwise identical except for the primary predictor. A final limitation of the study was the number of covariates. As this was an exploratory study, we analyzed hypothesized covariates based on our DAG, though there is likely uncontrolled confounding present at some level.

5. CONCLUSIONS

In this study of 8,322 emergency department medical patients admitted to the hospital, we found that dynamic vital signs while in the emergency department, as categorized by MEWS, and specifically failure to normalize abnormalities, were associated with increased mortality, probability of ICU admission, length of stay, and the diagnosis of sepsis. Changes in MEWS during ED stay were superior to static scores at triage for the predicting mortality, ICU admission and sepsis. The final MEWS score in the ED was strongly associated with outcome, comparable to delta MEWS scores for prediction of death and ICU admission, and suggesting that the condition in which patients leave the ED is as important as changes while in the ED.

Supplementary Material

1

Supplementary Figure 1: Historical Mortality at each MEWS score from institutional analysis

2

Supplementary Figure 2: Example Sensitivity and Specificity of institutional MEWS Characteristics on Hospital Unit AIM-A

3

Supplemental Figure 3: Probability of Length of Stay

4

Supplemental Table 1: Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0) Checklist

5

Supplemental Table 2: Area Under the Receiver Operating Curve Values for Models

6

Supplemental Table 3: Sensitivity Analysis examining outcomes within patients who received antibiotics in the ED.

Abbreviation List

AUROC

area under the receiver operating characteristic

CCI

Charlson Comorbidity Index

CI

Confidence interval

ED

Emergency department

EMR

Electronic medical record

GCS

Glasgow Coma Scale

HIPAA

Health Insurance Portability and Accountability Act

ICD

International Classification of Diseases

ICU

Intensive care unit

IQR

Interquartile range

IRB

Institutional review board

LOS

Length of stay

MEWS

Modified Early Warning Systems

QI

Quality improvement

qSOFA

Quick Sepsis Related Organ Failure Assessment

RR

Respiratory rate

SBP

Systolic blood pressure

SD

Standard deviation

STROBE

Strengthening the Reporting of Observational Studies in Epidemiology