Improving patient outcomes following vital sign monitoring protocol failure: A retrospective cohort study

Abstract

Background and Aims

Vital sign monitoring needs to be timely and correct to recognize deteriorating patients early and trigger the relevant clinical response. The aim of this study is to retrospectively evaluate compliance specifically toward the regional vital sign monitoring protocol the so called early warning score protocol (EWS‐protocol) 72 h before a medical emergency team response (MET‐response) and thereby illuminate whether poor compliance translates into a worse patient outcome.

Methods

It was investigated all eligible patients that underwent MET responses during the calendar year 2019. The inclusion criteria encompassed somatic patients above 18 years of age admitted to the hospital and detailed evaluations of the medical records of the included patients were conducted.

Results

Four hundred and twenty‐nine MET‐responses were included in the final analysis. EWS‐protocol failure was observed for more than half the patients within all the time frames assessed. Thirty‐day mortality was significantly higher for patients subject to EWS protocol failure in the timeframes 24−16, 16−8, 8−0 h before MET response. Adjusting for admission length, age, and gender, patients subject to EWS‐protocol failure had an odds ratio (OR) of 1.9, 2.0, 2.1, 2.3 for mortality in the time frames 72−48, 24−16, 16−8, and 8−0 h before the MET‐response, respectively. The adjusted OR for ICU‐admission was 1.7, and 1.6 for patients subject to EWS‐protocol failure in the time frames 16−8 and 8−0 h before MET‐response, respectively.

Conclusion

According to all the data analysis in this article, there is evidence that compliance toward the NEWS‐protocol is poor. EWS‐protocol failure is associated with a significant higher mortality and ICU‐admission rate.

1. INTRODUCTION

Vital sign monitoring is important to detect and prevent physiological deterioration in ward patients. Early warning score (EWS) is a standardized vital sign monitoring protocol and data suggest that EWS prevents adverse events and physiological deterioration why it is recommended as a patient safety modality ¹ The two most common systems for identifying deteriorating patients are the single‐parameter track and trigger system (SPTTS) and the aggregated weighted track and trigger system (AWTTS). ² , ³ The SPTTS triggers a response when a patient is subject to a specific physiological abnormality. ⁴ The AWTTS is designed to allocate points according to the severity of the deviation when vital signs deviate from predefined thresholds. The points are aggregated into a total score that triggers a response. Fifty‐six different AWTTS have been identified. ⁵ The AWTTS called ViEWS is regarded as the best system to predict hospital mortality as it shows the highest area under the receiver operating characteristics with a value of 0.888 for predicting mortality, which outperforms other AWTTS systems. ⁶ The AWTTS used in the capital region of Denmark is the National Early Warning Score (NEWS) also known as EWS (see Appendix 1).

RRS (rapid response system) addresses the discrepancy between patient needs and the resources available in the general wards. ⁷ But general wards are characterized by having a large number of patients with deviating vital signs and nurses are not aware of all these deviations. ⁸ It has also been established that the mortality of patients admitted to the ICU from general wards has significantly higher mortality than those admitted from emergency departments, operating rooms, and recovery rooms. ⁹ Several studies have found that vital signs are not monitored as dictated by the protocol. ¹⁰ A large observational study of 168,000 EWS recordings indicates that there seems to be a preference toward vital sign values that do not generate points as there is an accumulation of vital sign recordings just below the value that triggers points indicating a bias ¹¹ as vital signs generating points may trigger interventions regarded as unnecessary or inconvenient by the healthcare professional. This represents a patient safety issue in hospitals as we know that deviating vital signs precedes serious adverse events. ¹² , ¹³ , ¹⁴ The afferent limb—the part that monitors, detects deterioration in patients, and triggers a response ¹⁵ is arguably the most important component of the RRS, but it relies on timely and correct recordings of vital signs to be effective.

We hypothesized that compliance toward the NEWS‐protocol was poor and leading up to a serious adverse event patients subject to this poor compliance would have a worse outcome. To our knowledge no study has evaluated compliance toward the NEWS‐protocol and whether this translates into a worse outcome for patients. The objective of this study was to investigate the compliance toward the NEWS‐protocol and patient outcomes 72 h leading up to medical emergency team (MET) activation and by that illuminate whether poor compliance results in a worse outcome for patients.

2. METHODS

2.1. Setting

This retrospective cohort study was conducted at Copenhagen University Hospital, North Zealand (NZH) in the Capital Region of Denmark. Patients subject to a MET‐event have been registered in a quality assurance database at NZH since 2007. The quality assurance database contained social security numbers and the exact time and date for MET‐activation for all patients in a secured document. With the social security numbers, patients were looked up in the Electronic Medical Journal system (Sundhedsplatformen by EpicCare®). The data was extracted into an excel‐sheet and each patient was anonymized upon extraction.

2.2. Ethical concerns

This study was done according to the WMA Declaration of Helsinki. As this study was part of a quality assurance audit, the need for patient consent was considered and waived by the hospital administration according to Danish national laws (reference no. 21000282).

2.3. Inclusion and exclusion criteria

All patients subject to a MET‐event from January 1, 2019 to December 31, 2019 were investigated. Patients above 18 years of age who were admitted to NZH and subject to a MET‐event were included; with the exclusion of patients who did not fulfill the inclusion criteria or if the following was the case:

• 1.Documentation from the MET‐event was missing or EWS‐data was missing.
• 2.Unable to read social security number.
• 3.Patients from the obstetric or psychiatric department
• 4.MET‐events canceled upon arrival.

Patients from obstetric and psychiatric departments were not included due to a different monitoring protocol. Patients subject to a MET‐event before admission and nonhospitalized people for obvious reasons have not been monitored leading up to the MET‐event.

2.4. Data extraction

Included patients' electronic medical journals were reviewed in detail. The following information was extracted: Gender, age, length of stay (LOS), surviving to discharge (STDC), ICU‐/high dependency unit (HDU) admission, 30‐day mortality, calling criteria failure, and EWS monitoring failure (EMF). We defined EMF from the NEWS‐algorithm used at NZH (see Appendix 1).

Incorrect monitoring was defined from the regional NEWS‐algorithm used at NZH also known as EWS (see Appendix 1). Allowed time delays were put in the definition to adjust for potential time gaps between monitoring of vital parameters and documentation to eliminate a potential documentation bias. The following was considered an EMF:

• 1.Abscence of monitoring patient every 12 h with EWS‐score of 0−1 with an acceptable margin of +2 h.
• 2.Abscence of monitoring every 6 h with EWS‐score 2 with an acceptable margin of +2 h.
• 3.Abscence of monitoring every 4 h with EWS 3−5 with an acceptable margin of +1 h.
• 4.Abscence of monitoring every 4 h with EWS 6 with an acceptable margin of +30 min.
• 5.Abscence of monitoring every 1 h with EWS 7−8 with an acceptable margin of +15 min.
• 6.Abscence of monitoring every 30 min with EWS > 9 with an acceptable margin of +5 min.

EWS recordings were reviewed 72 h before MET‐call in five different time frames 72−48, 48−24, 24−16, 16−8, 8−0 h. Each timeframe was assessed separately for incorrect monitoring. The assessment was registered as a binary response (yes or no).

ICU‐ or HDU‐admissions were defined as admission to these units within 24 h after the MET‐event. Thirty‐day mortality was defined as the patient dying within 30 days after the MET‐event, and calling criteria failure was defined as not calling the MET‐team with EWS > 9 for more than 1 h.

Age and LOS were recorded as numerical variables. Gender, 30‐day mortality, ICU‐/HDU‐admission, STDC, calling criteria failure, and EWS‐monitoring failure were recorded as binary outcomes with 1 or 2 corresponding to yes and no and 1 corresponding to male and 2 corresponding to female in the gender variable. Partially recordings were defined as containing a minimum of four of the seven vital signs in the EWS monitoring algorithm. If the time frame contained a partial recording but the patient was monitored within the time limits, the number 3 would be allocated. If a recording contained three or fewer vital signs the recording was considered incomplete.

The primary outcome was 30‐day mortality, and the secondary outcomes were STDC and ICU‐/HDU‐admission. The primary outcome was correlated with each timeframe using a Rstudio® statistical software (Version 1.4.1103). 2 × 2 tables were constructed with the “publish” package to calculate the OR. Pearson's χ ² test a two‐sided statistical model was used to test for differences in OR. A multiple linear regression model was used to adjust for any confounding variables.

3. RESULTS

Five hundred and twenty‐one MET‐calls were registered in the period from January 1, 2019 to December 31, 2019. Ninety‐two patients were excluded (Figure 1). Hence, 429 MET‐calls were included in the final study.

Figure 1.

Inclusion and exclusing flow diagram. MET, medical emergency team.

The majority of patients subject to MET‐events were male (57%). The mean age was 73 years, and the mean admission length was 14.8 days. Fourty percent of patients would within 24 h after the MET‐event be admitted to a higher level of care (ICU or HDU). Fifteen percent of patients would be subject to multiple MET‐events within their admission. Thirty‐three percent of patients fulfilling MET‐calling criteria would experience delayed activation of MET‐calls of more than an hour. The 30‐day mortality was 47% (203 of 429). Twenty‐six patients of the 203 who died within 30 days after the MET‐event were alive for discharge (Table 1).

Table 1.

Patient characteristics.

Characteristics	Number of patients	Mean
	429
Age (years)		73
Gender, n (%)
Female	183	43%
Male	246	57%
Length of stay (LOS) (days)		14.8
ICU‐admission, n (%)	173	40%
Survirving to discharge, n (%)	252	59%
30‐day mortality, n (%)	203	47%
Multiple MET‐events within same admission	66	15%
Fulfilling MET‐calling criteria but not calling	55	13%

Abbreviation: MET, medical emergency team.

EMF defined as the absence of monitoring vital signs within the time frames recommended by the NEWS‐algorithm including acceptable time margins was observed for more than half of patients within all the time frames (Table 2).

Table 2.

Patients subject to EWS monitoring failure.

Time before MET‐event (h)	Number of patients	% patients subject to EWS monitoring failure [95% CI]
72−48	204	59% [0.52−0.66]
48−24	264	65% [0.59−0.70]
24−16	304	51% [0.45−0.56]
16−8	336	53% [0.47−0.58]
8−0	429	54% [0.49−0.58]

Note: First column indicates timeframe, second column indicates total number of patients observed in the given timeframe, third column indicates percentage of patients subject to EWS monitoring failure and 95% CI within the given timeframe.

Abbreviation: MET, medical emergency team.

Patients subject to EMF in the time frames 24−16, 16−8, and 8−0 h before MET‐event had a significantly higher mortality compared to the group where the EWS monitoring protocol had been followed correctly (Figure 2).

Figure 2.

Mortality comparison diagram. EWS, early warning score.

Calculating an OR it was found patients subject to EMF 24−16, 16−8, and 8−0 h before a MET‐event had a significantly higher 30‐day mortality. There were no significant 30‐day mortality when looking at EMF in the time frames 72−48 and 48−24 h before a MET event. There was also a significantly increased OR for ICU‐admission in the timeframes 16−8 and 8−0 h before a MET event for patients subject to EMF (Table 3)

Table 3.

EWS monitoring failure outcomes.

Time before MET‐event (h)	Crude odds ratio 30‐day mortality [95% CI]	Adjusted odds ratio 30‐day mortality [95% CI]	p Value adjusted OR
72−48	1.6 [0.9−2.8]	1.9 [1.0−3.1]	p  = 0.05*
48−24	1.3 [0.8−2.2]	1.2 [0.7−2.1]	p = 0.51
24−16	1.9 [1.3−3.1]	2.0 [1.2−3.4]	p  = 0.009*
16−8	1.9 [1.2−2.9]	2.1 [1.3−3.5]	p  = 0.002*
8−0	2.3 [1.6−3.4]	2.3 [1.5−3.6]	p  < 0.0001*

Time before MET‐event (h)	Crude odds ratio ICU‐admission [95% CI]	Adjusted odds ratio ICU‐admission [95% CI]	p Value adjusted OR
72−48	0.9 [0.5−1.7]	0.9 [0.5−1.7]	p = 0.87
48−24	0.7 [0.4−1.2]	0.8 [0.4−1.3]	p = 0.35
24−16	1.2 [0.7−1.9]	1.4 [0.9−2.3]	p = 0.15
16−8	1.6 [1.0−2.6]	1.7 [1.1−2.7]	p  = 0.02*
8−0	1.4 [0.9−2.2]	1.6 [1.1−2.4]	p  = 0.02*

Note: First column indicates timeframe, second column indicates crude odds ratio before adjusting for confounders in the given timeframe, third column indicates odds ratio after adjusting for confounders in the given timeframe, fourth column indicates p‐values for the adjusted odds ratio.

Abbreviation: MET, medical emergency team.

Running a multiple logistic regression model adjusting for age, gender, and LOS yielded significant results for 30‐day‐mortality in 72−48, 24−16, 16−8, and 8−0 h, with an OR of respectively 1.9, 2.0, 2.1, and 2.3. For ICU‐admission the adjusted OR for patients subject to EMF 16−8 and 8−0 h were 1.7 and 1.6, respectively after adjusting (Table 3). LOS had a negative association with mortality and age had a positive association when running the linear regression model.

4. DISCUSSION

It was found that patients were subject to EMF in more than half of the cases (Table 2) and that not monitoring patients within the time frames suggested by the NEWS‐algorithm results in significantly higher mortality (Table 3 and Figure 2). The data shows a trend suggesting increasing mortality with decreasing time to MET‐event. We also found that a third of patients meeting the calling criteria for MET had delayed calls for MET (Table 1). Our findings represent a serious liability for patient safety and quality of care.

current data suggest that compliance toward the NEWS protocol is poor, as shown in a large study of 300,000 vital sign recordings. Vital signs were recorded more often in the daytime than nighttime and on weekdays than weekends, ¹⁰ but the consequences of poor compliance has not been established before and it is possible that the true effect of RRSs is being underestimated due to the scale of failure of the afferent limb that we have demonstrated.

A significantly higher ICU‐admission rate was found in patients subject to EMF. This could be explained by vital sign deviations being detected late; thereby catching deterioration at a late stage where clinical action no longer can prevent physiological decompensation. ICU‐patients require a lot of resources and represent a patient group with high mortality. ¹⁶ From a cost‐benefit perspective, it could therefore be worthwhile to implement measures to improve EWS monitoring compliance.

There was a negative correlation between LOS and 30‐day mortality suggesting that increased LOS decreases mortality, opposite of expected. ¹⁷ As this is a subpopulation of deteriorating patients, patients who survive will naturally also have an increased LOS as the recovery period extends the period of hospital admission.

Data was missing for 225 patients in the 72−48 h, 164 patients in the 48−24 h, 125 in the 24−16 h, and 93 in the 16−8 h, and no missing data in the 0−8 h before MET‐event (Table 2). Missing data was due to EWS‐recordings not being available if patients' admission length was shorter than 72 h before being subject to a MET‐event.

A limitation of this study is the endpoint—mortality, as a lot of these patients cannot be saved even with the right treatment. Mortality, cardiac arrest, and ICU‐admission are traditionally used when investigating the early warning system and no better endpoints have been identified yet. ¹⁸ “Do not resuscitate” (DNR)‐orders were not registered in this study as we assumed patients subject to a MET‐event were clinically evaluated as able to rescue. Patients with DNR‐orders are potentially more prone to not being monitored in time. The potential time gap between vital sign recording and documentation represents a potential documentation bias. At NZH vital signs recordings are documented and registered in the Electronic Medical Journal using a portable device (Rover®), which nurses carry with them when monitoring patients. This combined with the allowed time margins in the definition of EMF minimizes the risk of documentation bias. We accepted partial recordings with at least four out of seven vital signs as a complete recording when assessing for EMF. This is unlikely to have changed the conclusion as only 20 out of 1531 time frames assessed contained partial recordings (1.3%).

Continuous monitoring could be part of the solution to poor compliance. In a controlled clinical trial automated continuous monitoring in a general ward reduced the LOS in the hospital and total LOS in the ICU. ¹⁹ But continuous monitoring is a resource‐heavy system both economically and staff‐wise, as expertise needs to be available at all times. Continuous monitoring does not solve the problem with escalation of care and might exacerbate the problem as alarm fatigue is a common problem with continuous monitoring systems. ²⁰ , ²¹

5. CONCLUSION

According to the data analysis in this article there is evidence that many patients before MET‐event do not have timely vital sign recordings, with more than half of patients not having recorded vital signs according to NEWS‐protocol. Our data suggest that absence of monitoring vital signs within the timeframes recommended by the NEWS‐protocol is associated with a significant higher mortality and ICU‐admission rate and therefore represents a potential liability for patient safety and quality of care. Based on these findings we recommend educating medical staff on the importance of following protocol, have transparent and clear protocols, improve the communication between staff to relay information about the deterioration of patients and install measures to improve attitudes toward the protocol.

Further studies of whether specific EWS‐values are more prone to EMF and if a bias toward certain patient groups is influencing monitoring routine should be conducted. These factors could help identify the cause of why EMF is happening to such an extent.

AUTHOR CONTRIBUTIONS

Hans Eric Sebastian Seitz‐Rasmussen: Conceptualization; data curation; formal analysis; investigation; methodology; writing—original draft; writing—review and editing. Morten Føns‐Sønderskov: Conceptualization; methodology; project administration; supervision; writing—review and editing. Anne Marie Kodal: Data curation; writing—review and editing. Morten Bestle: Project administration; supervision; writing—review and editing. All authors have read and approved the final version of the manuscript.

CONFLICT OF INTEREST STATEMENT

The authors declare no conflict of interest.

TRANSPARENCY STATEMENT

The lead author Hans Eric Sebastian Seitz‐Rasmussen affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

APPENDIX 1. NEWS SCORING AND ESCALATIONS PROTOCOL AT NOH.

VP/score	3	2	1	0	1	2	3
Respiratory rate	≤8		9−11	12−20		21‐24	≥25
SpO2	≤91	92−93	94−95	≥96
O2 supplemented				%		+
Pulse rate	≤40		41−50	51−90	91−110	111−130	≥131
Systolic blood pressure	≤90	91−100	101−110	111−219			≥220
Level of consciousness				A			V, P, U
Temperature	≤35		35.1−36.0	36.1−38.0	38.1−39.0	≥39.1

Abbreviations: A, alert; P, response on pain; U, unresponsive; V, response on verbal command.

	Maximal observational interval	Action protocol
0−1	12 h	Continue NEWS every 12 h, observational frequency may be increased
2	6 h	Nursing staff optimizes ABCDE. Observational frequency may be increased
3−5	4 h	Nurse optimizes ABCDE Nurse evaluate need for orientation of on‐duty physician or summoning on‐duty physician If on‐duty physician is oriented he/she must document a treatment plan
6	4 h	Nurse optimizes ABCDE On‐duty physician must be called The on‐duty physician must institute and document a treatment plan
7−8	1 h	Nurse optimizes ABCDE Nurse immediately summons on‐duty physician The on‐duty physician must see the patient emergently, institutes and documents a treatment plan The treatment plan must be conferred with the attending/on‐call physician
≥9	30 min	Nurse optimizes ABCDE Nurse immediately summons on‐duty physician The on‐duty physician must see the patient immediately (within 15 min, confers with attending/on‐call physician or activates MET, institutes and documents a treatment plan
EWS score of 9 or above AND no response in EWS despite medical treatment instituted by a physician		MET must be activated—local emergency number: 2222 Only exception being in situations of end of life care
If the patient has a new or undocumented single parameter score of 3 the on‐duty physician must immediately see the patient and MET may be activated.

Seitz‐Rasmussen HES, Føns‐Sønderskov M, Kodal A‐M, Bestle MH. Improving patient outcomes following vital sign monitoring protocol failure: a retrospective cohort study. Health Sci Rep. 2024;7:e1754. 10.1002/hsr2.1754

DATA AVAILABILITY STATEMENT

The authors had full access to all of the data in this study and takes complete responsibility for the integrity of the data and the accuracy of the data analysis. Deidentified data and statistical analysis data that underlie the results reported is available upon request.