A RANDOMIZED TRIAL OF REAL-TIME AUTOMATED CLINICAL DETERIORATION ALERTS SENT TO A RAPID RESPONSE TEAM

Abstract

Background

Episodes of patient deterioration on hospital units are expected to increasingly contribute to morbidity and healthcare costs.

Objective

To determine if real-time alerts sent to the rapid response team (RRT) improved patient care.

Design

Randomized, controlled trial.

Setting

Eight medicine units (Barnes-Jewish Hospital).

Patients

571 patients.

Intervention

Real-time alerts generated by a validated deterioration algorithm were sent real-time to the RRT (intervention) or hidden (control).

Measurements

ICU transfer, hospital mortality, hospital duration.

Results

ICU transfer (17.8% versus 18.2%; odds ratio, 0.972; 95% CI, 0.635–1.490) and hospital mortality (7.3% versus 7.7%; odds ratio, 0.947; 95% CI, 0.509–1.764) were similar for the intervention and control groups. The number of patients requiring transfer to a nursing home or long-term acute care hospital was similar for patients in the intervention and control groups (26.9% versus 26.3%; odds ratio, 1.032; 95% CI, 0.712–1.495). Hospital duration [8.4 ± 9.5 days versus 9.4 ± 11.1 days; P = 0.038] was statistically shorter for the intervention group. The number of RRT calls initiated by the primary care team was similar for the intervention and control groups (19.9% versus 16.5%; odds ratio, 1.260; 95% CI, 0.823–1.931).

Conclusions

Real-time alerts sent to the RRT did not reduce ICU transfers, hospital mortality, or the need for subsequent long term care, however hospital length of stay was modestly reduced.

INTRODUCTION

Patients deemed suitable for care on a general hospital unit are not expected to deteriorate; however, triage systems are not perfect, and some patients on general nursing units do develop critical illness during their hospitalization. Fortunately, there is mounting evidence that deteriorating patients exhibit measurable pathologic changes that could possibly be used to identify them prior to significant adverse outcomes, such as cardiac arrest (1–3). Given the evidence that unplanned intensive care unit (ICU) transfers of patients on general units result in worse outcomes than more controlled ICU admissions (1,4–6), it is logical to assume that earlier identification of a deteriorating patient could provide a window of opportunity to prevent adverse outcomes.

The most commonly proposed systematic solution to the problem of identifying and stabilizing deteriorating patients on general hospital units includes some combination of an early warning system (EWS) to detect the deterioration and a rapid response team (RRT) to deal with it (7–10). We previously demonstrated that a relatively simple hospital-specific method for generating EWS alerts derived from the EMR database is capable of predicting clinical deterioration and the need for ICU transfer, as well as hospital mortality, in non-ICU patients admitted to general inpatient medicine units (11–14). However, our data also showed that simply providing the EWS alerts to these nursing units did not result in any demonstrable improvement in patient outcomes (14). Therefore, we set out to determine whether linking real-time EWS alerts to an intervention, notification of the RRT for patient evaluation, could improve the outcomes of patients cared for on general inpatient units.

METHODS

Study Location

The study was conducted on eight adult inpatient medicine units of Barnes-Jewish Hospital, a 1250-bed academic medical center in St. Louis, MO (January 15, 2013 to May 9, 2013). Patient care on the inpatient medicine units is delivered by either attending hospitalist physicians or dedicated house staff physicians under the supervision of an attending physician. Continuous electronic vital sign monitoring is not provided on these units. The study was approved by the Washington University School of Medicine Human Studies Committee and informed consent was waived. This was a non-blinded study (ClinicalTrials.gov Identifier: NCT01741480).

Patients and Procedures

Patients admitted to the eight medicine units received usual care during the study except as noted below. Manually obtained vital signs, laboratory data, and pharmacy data inputted real-time into the EMR were continuously assessed. The EWS searched for the 36 input variables previously described (11,14) from the EMR for all patients admitted to the eight medicine units 24 hours per day and 7 days a week. Values for every continuous parameter were scaled so that all measurements lay in the interval [0, 1] and were normalized by the minimum and maximum of the parameter as previously described (14). To capture the temporal effects in our data, we retained a sliding window of all the collected data points within the last 24 hours. We then subdivided these data into a series of 6 sequential buckets of 4 hours each. We excluded the 2 hrs of data prior to ICU transfer in building the model (so the data were −26 hrs to −2 hrs prior to ICU transfer for ICU transfer patients, and the first 24 hrs of admission for everyone else). Eligible patients were selected for study entry when they triggered an alert for clinical deterioration as determined by the EWS (11,14).

The EWS alert was implemented in an internally developed, Java-based clinical decision support rules engine, which identified when new data relevant to the model were available in a real-time central data repository. In a clinical application it is important to capture unusual changes in vital-sign data over time. Such changes may precede clinical deterioration by hours, providing a chance to intervene if detected early enough. In addition, not all readings in time-series data should be treated equally; the value of some kinds of data may change depending on their age. For example, a patient’s condition may be better reflected by a blood-oxygenation reading collected 1 hour ago than a reading collected 12 hours ago. This is the rationale for our use of a sliding window of all collected data points within the last 24 hours performed in a real-time basis to determine the alert status of the patient (11,14).

We applied various threshold cut-points to convert the EWS alert predictions into binary values and compared the results against the actual ICU transfer outcome (14). A threshold of 0.9760 for specificity was chosen to achieve a sensitivity of approximately 40%. These operating characteristics were chosen in turn to generate a manageable number of alerts per hospital nursing unit per day (estimated at 1–2 per nursing unit per day). At this cut-point the C-statistic was 0.8834, with an overall accuracy of 0.9292. In other words, our EWS alert system is calibrated so that for every 1000 patient discharges per year from these eight hospital units, there would be 75 patients generating an alert of which 30 patients would be expected to have the study outcome (i.e., clinical deterioration requiring ICU transfer).

Once patients on the study units were identified as “at risk” for clinical deterioration by the EWS they were assigned by a computerized random number generator to the intervention group or the control group. The control group was managed according to the usual care provided on the medicine units. The EWS alerts generated for the control patients were electronically stored, but these alerts were not sent to the RRT nurse, instead they were hidden from all clinical staff. The intervention group had their EWS alerts sent real-time to the nursing member of the hospital’s RRT. The RRT is composed of a registered nurse, a second- or third-year internal medicine resident, and a respiratory therapist. The RRT was introduced in 2009 for the study units involved in this investigation. For 2009, 2010, and 2011 the RRT nurse was pulled from the staff of one of the hospital’s ICUs in a rotating manner to respond to calls to the RRT as they occurred. Starting in 2012, the RRT nurse was established as a dedicated position without other clinical responsibilities. The RRT nurse carries a hospital issued mobile phone to which the automated alert messages were sent real-time and was instructed to respond to all EWS alerts within 20 minutes of their receipt.

The RRT nurse would initially evaluate the alerted intervention patients using the Modified Early Warning Score (MEWS) (15,16) and make further clinical and triage decisions based on those criteria and discussions with the RRT physician or the patient’s treating physicians. The RRT focused their interventions using an internally developed tool called the “Four D’s” (Discuss goals of care, Drugs needing to be administered, Diagnostics needing to be performed, and Damage control with the use of oxygen, intravenous fluids, ventilation, and blood products). Patients evaluated by the RRT could have their current level of care maintained, have the frequency of vital sign monitoring increased, be transferred to an ICU, or have a “Code Blue” called for emergent resuscitation. The RRT reviewed goals of care for all patients to determine the appropriateness of interventions, especially for patients near the end of life who did not desire intensive care interventions. Nursing staff on the hospital units could also make calls to the RRT for patient evaluation at any time based on their clinical assessments performed during routine nursing rounds.

The primary efficacy outcome was the need for ICU transfer. Secondary outcome measures were hospital mortality and hospital length of stay. Pertinent demographic, laboratory, and clinical data were gathered prospectively including: age, gender, race, underlying comorbidities, and severity of illness assessed by the Charlson comorbidity score and Elixhauser comorbidities (17,18).

Statistical Analysis

We required a sample size of 514 patients (257 per group) to achieve 80% power at a 5% significance level, based on the superiority design, a baseline event rate for ICU transfer of 20.0%, and an absolute reduction of 8.0% (PS Power and Sample Size Calculations, Version 3.0). Continuous variables were reported as means with standard deviations or medians with 25^th and 75^th percentiles according to their distribution. The Student’s t-test was used when comparing normally distributed data and the Mann-Whitney U test was employed to analyze non-normally distributed data (e.g., hospital length of stay). Categorical data was expressed as frequency distributions and the Chi-squared test was used to determine if differences existed between groups. A P value less than 0.05 was regarded as statistically significant. An interim analysis was planned for the data safety monitoring board to evaluate patient safety after fifty percent of the patients were recruited. The primary analysis was by intention to treat. Analyses were performed using SPSS, version 11.0 for Windows (SPSS, Inc., Chicago, IL).

Data Safety Monitoring Board

An independent data safety and monitoring board (DSMB) was convened to monitor the study and to review and approve protocol amendments by the steering committee.

RESULTS

Between January 15, 2013 and May 9, 2013, 4731 consecutive patients were admitted to the eight inpatient units and electronically screened as the base population for this investigation. 571 (12.1%) patients triggered an alert and were enrolled into the study (Figure 1). There were 286 patients assigned to the intervention group and 285 assigned to the control group. No patients were lost to follow-up. Demographics, reason for hospital admission, and comorbidities of the two groups were similar (Table 1). The number of patients having a separate RRT call by the primary nursing team on the hospital units within 24 hours of generating an alert was greater for the intervention group but did not reach statistical significance (19.9% versus 16.5%; odds ratio, 1.260; 95%CI, 0.823–1.931). Table 2 provides the new diagnostic and therapeutic interventions initiated within 24 hours after a EWS alert was generated. Patients in the intervention group were significantly more likely to have their primary care team physician notified by a RRT nurse regarding medical condition issues and to have oximetry and telemetry started, whereas control patients were significantly more likely to have new antibiotic orders written within 24 of generating an alert.

Figure 1.

Study flow diagram.

Table 1.

Baseline data.

Variable	Intervention Group (n=286)	Control Group (n=285)	P value
Age, years:	63.7 ± 16.0	63.1 ± 15.4	0.495
Gender, n(%)
Male	132 (46.2)	140 (49.1)	0.503
Female	154 (53.8)	145 (50.9)
Race, n(%)
Caucasian	155 (54.2)	154 (54.0)	0.417
African-American	105 (36.7)	113 (39.6)
Other	26 (9.1)	18 (6.3)
Reason for Hospital Admission
Cardiac	12 (4.2)	15 (5.3)	0.548
Pulmonary	64 (22.4)	72 (25.3)	0.418
Underlying malignancy	6 (2.1)	3 (1.1)	0.504
Renal disease	31 (10.8)	22 (7.7)	0.248
Thromboembolism	4 (1.4)	5 (1.8)	0.752
Infection	55 (19.2)	50 (17.5)	0.603
Neurologic disease	33 (11.5)	22 (7.7)	0.122
Intra-abdominal disease	41 (14.3)	47 (16.5)	0.476
Hematologic condition	4 (1.4)	5 (1.8)	0.752
Endocrine disorder	12 (4.2)	6 (2.1)	0.153
Source of Hospital Admission
Emergency department	201 (70.3)	203 (71.2)	0.200
Direct admission	36 (12.6)	46 (16.1)
Hospital transfer	49 (17.1)	36 (12.6)
Charlson score	6.7 ± 3.6	6.6 ± 3.2	0.879
Elixhauser Comorbidities	7.4 ± 3.5	7.5 ± 3.4	0.839

Table 2.

Diagnostic and therapeutic interventions initiated within 24 hours of generating an alert.

Variable	Intervention Group (n=286)	Control Group (n=285)	P value
Medications, n (%)
antibiotics	92 (32.2)	121 (42.5)	0.011
antiarrhythmics	48 (16.8)	44 (15.4)	0.662
anticoagulants	83 (29.0)	97 (34.0)	0.197
diuretics/antihypertensives	71 (24.8)	55 (19.3)	0.111
bronchodilators	78 (27.3)	73 (25.6)	0.653
anticonvulsives	26 (9.1)	27 (9.5)	0.875
sedatives/narcotics	0 (0.0)	1 (0.4)	0.499
Respiratory support, n (%):
noninvasive ventilation	17 (6.0)	9 (3.1)	0.106
escalated oxygen support	12 (4.2)	7 (2.5)	0.247
Enhanced vital signs, n (%):	50 (17.5)	47 (16.5)	0.752
Maintenance intravenous fluids, n (%):	48 (16.8)	41 (14.4)	0.430
Vasopressors, n (%):	57 (19.9)	61 (21.4)	0.664
Bolus intravenous fluids, n (%):	7 (2.4)	14 (4.9)	0.118
Telemetry, n (%):	198 (69.2)	176 (61.8)	0.052
Oximetry, n (%):	20 (7.0)	6 (2.1)	0.005
New intravenous access, n (%):	26 (9.1)	35 (12.3)	0.217
Primary care team physician called by RRT nurse, n (%):	82 (28.7)	56 (19.6)	0.012

RRT = rapid response team.

Fifty-one patients (17.8%) randomly assigned to the intervention group required ICU transfer compared with 52 of 285 patients (18.2%) in the control group (odds ratio, 0.972; 95%CI, 0.635–1.490; P = 0.898) (Table 3). Twenty-one patients (7.3%) randomly assigned to the intervention group expired during their hospitalization compared with 22 of 285 patients (7.7%) in the control group (odds ratio, 0.947; 95%CI, 0.509–1.764; P = 0.865). Hospital length of stay was 8.4 ± 9.5 days [median; interquartile range: 4.5 days; 2.3–11.4 days] for patients randomized to the intervention group and 9.4 ± 11.1 days [5.3 days; 3.2–11.2 days] for patients randomized to the control group (P = 0.038). The ICU length of stay was 4.8 ± 6.6 days [2.9 days; 1.7–6.5] for patients randomized to the intervention group and 5.8 ± 6.4 days [2.9 days; 1.5–7.4] for patients randomized to the control group (P = 0.812).The number of patients requiring transfer to a nursing home or long-term acute care hospital was similar for patients in the intervention and control groups (26.9% versus 26.3%; odds ratio, 1.032; 95% CI, 0.712–1.495; P = 0.870). Similarly, the number of patients requiring hospital readmission before 30 days and 180 days respectively was similar for the two treatment groups (Table 3). For the combined study population, the EWS alerts were triggered 94 ± 138 hours (median; interquartile range: 27 hours; 7–132 hours) prior to ICU transfer and 250 ± 204 hours (median; interquartile range: 200 hours; 54–347 hours) prior to hospital mortality. The number of RRT calls for the eight medicine units studied progressively increased from the start of the RRT program in 2009 through 2013 (121 in 2009, 194 in 2010, 298 in 2011, 415 in 2012, 415 in 2013; P < 0.001 for the trend).

Table 3.

Outcomes

Outcome	Intervention Group (n=286)	Control Group (n=285)	P value
ICU transfer, n(%)	51 (17.8)	52 (18.2)	0.898
All-cause hospital mortality, n(%)	21 (7.3)	22 (7.7)	0.865
Transfer to nursing home or LTAC, n(%)	77 (26.9)	75 (26.3)	0.870
30-day readmission	53 (18.5)	62 (21.8)	0.337
180-day readmission	124 (43.4)	117 (41.1)	0.577
Hospital length of stay, days*	8.4 ± 9.5 4.5 [2.3,11.4]	9.4 ± 11.1 5.3 [3.2,11.2]	0.038
ICU length of stay, days*	4.8 ± 6.6 2.9 [1.7,6.5]	5.8 ± 6.4 2.9 [1.5,7.4]	0.812

^*Values expressed as mean ± standard deviation, median with interquartile range.

ICU= intensive care unit; LTAC= long-term acute care center.

DISCUSSION

We demonstrated that a real-time EWS alert sent to a RRT nurse was associated with a modest reduction in hospital length of stay, but similar rates of hospital mortality, ICU transfer, and subsequent need for placement in a long-term care setting compared with usual care. We also found the number of RRT calls to have increased progressively from 2009 to the present on the study units examined.

Unplanned ICU transfers occurring as early as within 8 hours of hospitalization are relatively common and associated with increased mortality (6). Bapoje et al evaluated a total of 152 patients over one year who had unplanned ICU transfers (19). The most common reason was worsening of the problem for which the patient was admitted (48%). Other investigators have also attempted to identify predictors for clinical deterioration resulting in unplanned ICU transfer that could be employed in an EWS (20,21). Organizations like the Institute for Healthcare Improvement have called for the development and routine implementation of EWSs in order to direct the activities of RRTs and improve outcomes (22). However, a recent systematic review found that much of the evidence in support of EWSs and emergency response teams is of poor quality lacking prospective randomized trials (23).

Our earlier experience demonstrated that simply providing an alert to nursing units did not result in any demonstrable improvements in the outcomes of high-risk patients identified by our EWS (14). Previous investigations have also had difficulty in demonstrating consistent outcome improvements with the use of EWSs and RRTs (24–32). As a result of mandates from quality improvement organizations, most U.S. hospitals currently employ RRTs for emergent mobilization of resources when a clinically deteriorating patient is identified on a hospital ward (33,34). Linking RRT actions with a validated real-time alert may represent a way of improving the overall effectiveness of such teams for monitoring general hospital units short of having all hospitalized patients in units staffed and monitored to provide higher levels of supervision (e.g., ICUs, step-down units) (9,35).

An alternative approach to preventing patient deterioration is to provide closer overall monitoring. This has been accomplished by employing nursing personnel to increase monitoring, or with the use of automated monitoring equipment. Bellomo et al showed that the deployment of electronic automated vital sign monitors on general hospital units was associated with improved utilization of RRTs, increased patient survival, and decreased time for vital sign measurement and recording (36). Laurens et al found that implementation of medical emergency teams (METs) to respond to predefined MET activation criteria as observed by hospital staff resulted in reduced hospital mortality and reduced need for ICU transfer (37). However, other investigators have observed that imperfect implementation of nursing performed observational monitoring resulted in no demonstrable benefit, illustrating the limitations of this approach (38). Our findings suggest that nursing care of patients on general hospital units may be enhanced with the use of an EWS alert sent to the RRT. This is supported by the observation that communications between the RRT and the primary care teams was greater as was the use of telemetry and oximetry in the intervention arm. Moreover, there appears to have been a learning effect for the nursing staff that occurred on our study units as evidenced by the increased number of RRT calls that occurred between 2009 and 2013. This change in nursing practices on these units certainly made it more difficult for us to observe outcome differences in our current study with the prescribed intervention, reinforcing the notion that evaluating an already established practice is a difficult proposition (39).

Our study has several important limitations. First, the EWS alert was developed and validated at Barnes-Jewish Hospital (11–14). We cannot say whether this alert will perform similarly in another hospital. Second, the EWS alert only contains data from medical patients. Development and validation of EWS alerts for other hospitalized populations, including surgical and pediatric patients, are needed to make such systems more generalizable. Third, the primary clinical outcome employed for this trial was problematic. Transfer to an ICU may not be an optimal outcome variable as it may be desirable to transfer alerted patients to an ICU which can be perceived to represent a “soft landing” for such patients once an alert has been generated. A better measure could be 30-day all cause mortality which would not be subject to clinician biases. Finally, we could not specifically identify explanations for the greater use of antibiotics in the control group despite similar rates of infection for both study arms. Future studies should closely evaluate the ability of EWS alerts to alter specific therapies (e.g., reduce antibiotic utilization).

In summary, we have demonstrated that an EWS alert linked to a RRT likely contributed to a modest reduction in hospital length of stay, but no reductions in hospital mortality and ICU transfer. These findings suggest that inpatient deterioration on general hospital units can be identified and linked to a specific intervention. Continued efforts are needed to identify and implement systems that will not only accurately identify high-risk patients on general hospital units but also intervene to improve their outcomes. We are moving forward with the development of a two-tiered EWS utilizing both EMR data and real-time streamed vital sign data to determine if we can further improve the prediction of clinical deterioration and potentially intervene in a more clinically meaningful manner.

ABBREVIATIONS

DSMB

data safety and monitoring board

EWS

early warning system

ICU

intensive care unit

METs

medical emergency teams

MEWS

Modified Early Warning Score

RRT

rapid response team