MORTALITY AND LENGTH OF STAY TRENDS FOLLOWING IMPLEMENTATION OF A RAPID RESPONSE SYSTEM AND REAL-TIME AUTOMATED CLINICAL DETERIORATION ALERTS

Abstract

We performed a study to determine the potential influence of a rapid response system (RRS) employing real-time clinical deterioration alerts (RTCDAs) on patient outcomes involving eight general medicine units. Introduction of the RRS occurred in 2006 with staged addition of the RTCDAs in 2009. Statistically significant year-to-year decreases in mortality were observed through 2014 (r = −0.794; P = 0.002). Similarly, year-to-year decreases in the number of cardiopulmonary arrests (CPAs) (r = −0.792; P = 0.006) and median lengths of stay (r = −0.841; P = 0.001) were observed. There was a statistically significant year-to-year increase in the number of RRS activations for these units (r = 0.939; P < 0.001) that was inversely correlated with the occurrence of CPAs (r = −0.784; P = 0.007). In this single-institution retrospective study, introduction of a RRS employing RTCDAs was associated with lower hospital mortality, CPAs, and hospital length of stay.

INTRODUCTION

Patients deemed suitable for care on a general hospital unit frequently deteriorate during their hospitalization because of disease progression and imperfect triage systems. Rapid response systems (RRSs) have been applied to identify and treat deteriorating patients on general hospital units (1,2). Given the evidence that unplanned intensive care unit (ICU) patient transfer results in worse outcomes than more controlled ICU admission (3–5), it is logical to assume that earlier identification of deteriorating patients could provide a window of opportunity to prevent adverse outcomes. The most commonly proposed approach to the problem of identifying and stabilizing deteriorating hospitalized patients includes some combination of an early warning system to detect the deterioration and a RRS to deal with it, although the evidence supporting this approach is inconsistent (2,6–11). The study team previously demonstrated that a relatively simple hospital-specific prediction model employing routine laboratory values and vital sign data is capable of predicting clinical deterioration, the need for ICU transfer, and hospital mortality in patients admitted to general medicine units (12–16). Therefore, the teame set out to review the outcomes of patients admitted to the 8 general medicine units of Barnes-Jewish Hospital during the development and implementation of a RRS employing real-time clinical deterioration alerts (RTCDAs).

METHODS

Study Location

This retrospective analysis utilized data derived from 8 adult general medicine units of Barnes-Jewish Hospital, a 1250-bed academic medical center in St Louis, Missouri, over 12 years (January 1, 2003, to December 31, 2014). Patient care on the medicine units is delivered by either attending hospitalist physicians or dedicated house staff physicians under the supervision of an attending physician. Nursing care on the general medicine units is carried out according to hospital guidelines and policies. Patients admitted to the general medicine units come primarily from the emergency department and direct admission from physician offices, although patients also can be transferred from outside hospitals. Common problems cared for on these units include pulmonary disorders, congestive heart failure, hypertension and diabetes management, non-acute gastrointestinal problems, and infections (pneumonia, influenza, urinary tract infections, skin and skin structure infections). The nurse to patient ratio is typically 4:1 but varies based on patient acuity. Daily rounds are carried out between the physician teams and unit nurses in the mornings with discharge rounds usually occurring in the afternoons. Nighttime coverage is provided by either the on-call house staff or hospitalist physician. Patients on comfort care or with a do not resuscitate order were not included in this analysis. The study was approved by the Washington University School of Medicine Human Studies Committee and the requirement for informed consent was waived (HRPO Number 201108173).

Procedures

Development of the RRS began in 2005 and initially targeted the 8 general medicine units. This included formalizing the RRS and providing staff education on how to activate and utilize the RRS. The RRS for the 8 general medicine units was introduced in 2006. Limiting the RRS to the general medicine units was done purposefully in order to develop a working system with the greatest opportunity to influence patient outcomes prior to more general hospital implementation. The hospital plan was to expand the RRS once a finalized and validated method was established. RRS activations between 2006 and 2008 were initiated by the nursing staff on the general medicine units as part of routine nursing practice. Starting in 2009, RRS activations could be initiated by the nursing staff as well as by RTCDAs. The development of the RTCDAs was part of a collaborative program carried out between Washington University and Barnes-Jewish Hospital.(12–16) The RTCDA prediction model was developed and validated using patient-specific data from the general medicine units.(12–15) The RTCDA prediction model was implemented in an internally developed, Java-based clinical decision support rules engine, which identified when new data relevant to the model were available in a real-time central data repository. The rules engine queried the data repository to acquire all data needed to evaluate the model. Patients identified as “at risk” for clinical deterioration had an automated alert sent in real time to the nursing member of the RRS team. RTCDAs were generated 24 hours per day 7 days per week.

Details regarding the RTCDA prediction model development have been described previously.(14,15) Briefly, the algorithm was first implemented in MATLAB (The Mathworks, Inc., Natick, Massachusetts). For the purposes of training, a single 24-hour window of data was used for each patient. For patients admitted to an ICU, this window was 26 hours to 2 hours prior to ICU admission; for all other patients, this window consisted of the first 24 hours of their hospital stay. The data set’s 36 input variables were divided into buckets and min/mean/max features wherever applicable, resulting in 398 variables. Input variables were manually obtained (vital signs, age) and automatically downloaded (laboratory values, medications). The first half of the data set was used to train the model for variable identification (Table 1; also see the online Appendix, available at http://journals.sagepub.com/doi/suppl/10.1177/1062860615613841). The second half of the data set was then used as the validation data set. A predicted outcome was generated for each case in the validation data using the model parameter coefficients derived from the training data. Various threshold cut points were then applied to convert these predictions into binary values, and the results were compared against the outcome parameters of interest (ICU threshold and mortality). A threshold of 0.9760 was chosen to achieve a sensitivity of approximately 40% for the detection. This sensitivity was chosen in turn to generate a manageable number of alerts per hospital nursing unit per day (estimated at 1–2 per nursing unit per day) in order to prevent “alert fatigue” from a model that generated too many false positive alerts.

Table 1.

The 10 Highest-Weighted Variables From the Training Data Set.^a

Variable	Coefficient
Respirations (bucket 6 max)	4.45
Oxygen saturation, pulse oximetry (bucket 6 min)	−4.22
Shock index (bucket 6 max)	4.01
Respirations (bucket 6 mean)	3.41
BP, systolic (bucket 6 min)	−2.96
Coagulation modifiers (bucket 1)	2.70
Pulse (bucket 6 max)	2.55
Respirations (bucket 6 min)	2.51
BP, diastolic (bucket 6 max)	2.48
Oxygen saturation, pulse oximetry (bucket 6 mean)	−2.44

Abbreviation: BP, blood pressure.

^aBucket 6 max/min/mean represents the maximum, minimum, and mean values, respectively, from the most recent 4-hour window from the preceding 24-hour time period.

The RRS team comprises a registered nurse, a second- or third-year internal medicine resident, and a respiratory therapist. Prior to 2011, the RRS nurse was pulled from the staff of one of the hospital’s ICUs in a rotating manner to respond to RRS calls as they occurred. Starting in 2012, the RRS team nurse member was established as a dedicated position without other clinical responsibilities. The RRS nurse carries a hospital-issued mobile phone to which the RTCDAs were sent and was instructed to respond to all alerts within 20 minutes of their receipt. Internal audits conducted by the Center for Clinical Excellence, BJC Healthcare, throughout the study period have demonstrated that RRS nurse response times have been less than 10 minutes for more than 95% of all alerts generated. The RRS nurse would initially evaluate the alerted patient using the Modified Early Warning Score (17,18) and make further clinical and triage decisions based on those criteria and discussions with the RRS physician or the patient’s treating physicians. The RRS/RTCDAs was only employed on the 8 general medicine units through 2014. Cardiopulmonary arrests (CPAs) were defined as the need for cardiopulmonary resuscitation or emergent tracheal intubation.

Statistical Analysis

To evaluate the effectiveness of the RRS/RTCDAs, the yearly hospital mortality, incidence of CPAs, and median hospital length of stay were compared for the study years 2003 to 2014. Data on CPAs were only available starting in 2005. The number and year-to-year change in RRS activations also was assessed and compared to the occurrence of CPAs. Year-to-year changes in hospital mortality, CPAs, and hospital length of stay, as well as the relationship between CPAs and RRS activations was described by the Pearson correlation coefficient. Linear regression was used to assess the relationship between changes in year-to-year outcomes while controlling for comorbidities with the Charlson comorbidity score, patient age, and sex. Analyses were performed using SPSS, version 11.0 for Windows (SPSS, Inc., Chicago, Illinois).

RESULTS

Between January 1, 2003, and December 31, 2014, a total of 163–311 consecutive patients were admitted to the general medicine units (Table 2). There was a statistically significant year-to-year decrease in hospital mortality for patients admitted to the general medicine units (r = −.794; P = .002; Figure 1). Linear regression identified study year to be an independent determinant of hospital mortality (0.108% decrease in hospital mortality per study year increment; 95% confidence interval [CI], 0.091% to 0.125%; P < .001). There also was a statistically significant year-to-year decrease in the number of CPAs for patients admitted to the general medicine units (r = −.792; P = .006; Figure 2).

Table 2.

Study Population Characteristics

	2003	2004	2005	2006	2007	2008	2009	2010	2011	2012	2013	2014
General Medicine Units
Patient Number:	11404	13127	13709	12379	11839	13577	14512	14523	14535	14712	14199	14795
Age, year:	60 [46,76]	60 [46,75]	60 [46,75]	59 [45,75]	58 [45,73]	57 [44,73]	57 [44,71]	56 [43,70]	57 [44,70]	56 [43,70]	57 [44,69]	57 [44,69]
Male, %:	44.7	45.6	45.4	44.5	46.1	45.4	46.8	45.6	46.5	47.2	48.2	48.2
Charlson score:	4 [2,6]	4 [2,6]	4 [2,6]	4 [2,7]	4 [2,7]	4 [2,7]	4 [2,7]	4 [2,7]	5 [2,7]	5 [2,8]	5 [2,8]	5 [2,8]
Income, USD:	30962 [25289, 37963]	30887 [25289, 37963]	30887 [25289, 37793]	30876 [25289, 37555]	30876 [25289, 37555]	30074 [25234, 37555]	29281 [25234, 37555]	28973 [24830, 36744]	30028 [25118, 37555]	30091 [25234, 37503]	30091 [25189, 37555]	29286 [25234, 37793]
Hospital mortality,%:	2.87	2.99	2.63	2.59	2.50	2.36	2.22	2.11	2.26	2.41	2.41	2.22
Cardiopulmonary arrests, n:	n/a	n/a	57	50	69	58	49	43	51	43	37	35
Hospital lengths of stay, days:	3.79 [2.02,6.81]	3.69 [1.99,6.68]	3.54 [1.94,6.36]	3.59 [1.98,6.39]	3.52 [1.95,6.31]	3.23 [1.84,6.01]	3.08 [1.78,5.79]	3.17 [1.82,5.85]	3.34 [1.85,5.99]	3.24 [1.86,5.93]	3.28 [1.84,5.98]	3.10 [1.75,5.82]
Rapid Response System activation:	0	0	0	72	57	112	126	149	309	419	400	370

Cardiopulmonary arrest data only collected beginning in 2005. Values expressed as medians with interquartile range.

Figure 1.

Hospital mortality for January 1, 2003 through December 31, 2014 for the eight general medicine units. Solid line represents the mean value with the dashed lines depicting the upper and lower limits of the 95% confidence intervals. P = 0.002 for the year-to-year decrease in hospital mortality for patients admitted to the general medicine units.

Figure 2.

Cardiopulmonary arrests between January 1, 2005 and December 31, 2014 for the eight general medicine units. P = 0.006 for the year-to-year decrease in cardiopulmonary arrests for patients admitted to the general medicine units. P < 0.001 for the year-to-year increase in rapid response system activations. Cardiopulmonary arrest data was only available starting in 2005.

A statistically significant year-to-year increase was observed in the number of RRS activations for the general medicine units (r = .939; P < .001; Figure 2). A statistically significant inverse correlation was found between the occurrence of CPAs and the number of RRS activations (r = −.784; P = .007; Figure 2). Linear regression identified study year to be an independent determinant of CPAs (3.367 decrease in CPAs per study year increment; 95% CI = 2.213 CPAs to 4.521 CPAs; P = .019). Median hospital length of stay demonstrated a statistically significant year-to-year decrease during the study years for patients admitted to the general medicine units (r = −.841; P = .001; Figure 3). Linear regression analysis identified study year to be an independent determinant of the median hospital length of stay (0.081 day decrease in median hospital length of stay per study year increment; 95% CI = 0.055 days to 0.107 days; P = .015).

Figure 3.

Hospital length of stay between January 1, 2003 and December 31, 2014 for the eight general medicine units. Hospital length of stay showed a statistically significant decrease over this time period for patients admitted to the general medicine units (P = 0.001).

DISCUSSION

This single-institution retrospective study demonstrated that the introduction of an institution-specific RRS employing RTCDAs on general medicine units was associated with lower mortality, decreased occurrence of CPAs, and reduced hospital stay. The study also found year-to-year reductions in CPAs to be significantly correlated with increasing RRS activations.

The study RRS is unique in employing RTCDAs, in addition to nursing assessments, in order to trigger the RRS. Davis et al recently described a successful RRS employing a novel curriculum aimed at hospital unit managers and primary responders (nurses and respiratory therapists).¹⁹ The RRS curriculum was designed to be a comprehensive tool to support inpatient resuscitation management utilizing quality improvement data, staff training, and focused initiatives. Davis et al found that the incidence of non-ICU CPAs decreased significantly whereas the incidence of ICU CPAs remained unchanged. Moreover, hospital mortality significantly decreased with implementation of their RRS and they observed a year-to-year change in the incidence of Code Blue that was inversely related to the change in RRS activations. Chen et al described the experience in New South Wales, Australia, where dedicated efforts toward implementing RRSs and improving the clinical impact of such systems have been in place for more than a decade.²⁰ Among 82 acute care hospitals with more than 9 million admissions, these investigators found that as RRSs were progressively introduced, there was a coincidental reduction in CPAs and hospital mortality. The RRSs described by Davis et al and Chen et al utilized traditional bedside staff approaches for the activation of the RRS. The present study experience suggests that automated real-time alerts generated by informatics systems also can be successfully applied to trigger RRS activations.

Other recent investigations also have suggested that the introduction of RRSs can improve patient outcomes in the hospital setting. Maharaj et al conducted a systematic review of all studies conducted between 1990 and 2013 to assess the impact of RRSs on patient outcomes.²¹ These investigators identified 29 eligible studies involving either adult or pediatric patients. The implementation of RRSs was associated with an overall reduction in hospital mortality in both the adult (relative risk [RR] = 0.87; 95% CI = 0.81–0.95) and pediatric (RR = 0.82; 95% CI = 0.76–0.89) inpatient populations. However, there was substantial heterogeneity in both populations. RRS teams also were associated with a reduction in CPAs in adult (RR = 0.65; 95% CI = 0.61–0.70) and pediatric (RR = 0.64; 95% CI = 0.55–0.74) patients. Meta-regression did not identify the presence of a physician in the RRS to be significantly associated with a mortality reduction. A more recent study has shed light on the timing of RRSs and patient outcomes. Chen et al found that RRS activation more than 15 minutes after detection and documentation of patient instability was independently associated with an increased risk of ICU admission and hospital mortality.²² The use of an RTCDA is a method that can ensure rapid activation of the RRS once an alert for clinical deterioration is identified, thus minimizing any delays that could occur with a nonautomated system. Interestingly, the maximum number of RRS activations, approximately one per day, is lower than that described by others.¹⁹ This is likely a function of the desire to limit false negative alerts. However, the majority of the RRS alerts were for preemptive respiratory and hemodynamic deterioration (Table 1). This may explain the observed decreasing rate of CPAs over time.

Although statistically significant and potentially clinically important outcome improvements were observed with the implementation of the RRS/RTCDAs on the 8 general medicine units of Barnes-Jewish Hospital, several important limitations of this study should be noted. First, the reductions in mortality, CPAs, and hospital stay cannot be definitively attributed to the introduction of the RRS/RTCDAs given the retrospective nature of this study. However, the correlation between increased number of RRS activations and decreased CPAs is plausible and provides a clinical explanation for the study findings. Moreover, the experience is consistent with that observed in other settings demonstrating similar findings, ^19–21 and the study team’s earlier prospective study supports these findings.¹⁶ The retrospective nature of hospital system experiences with RRSs accounts in large part for the reported “poor quality” of evidence supporting their application.⁹ Despite the overall poor quality of evidence, recent systematic reviews and meta-analyses of RRSs have concluded that there is increasing evidence that RRSs can be effective in reducing admissions to the ICU and reducing hospital mortality.^9,21 Unfortunately, data regarding the use of RTCDAs was inadequate to make recommendations on this aspect of patient care. A significant limitation of this study is that the study team did not employ a physiologic severity of illness marker to adjust the outcomes over time, which introduces a potential bias in the findings, especially if less ill patients were admitted to the medicine units over time. This seems unlikely given that the Charlson comorbidity scores increased for the patients admitted to the general medicine units over time. It is also notable that the hospital had no other quality initiatives for these units during this time period that would have influenced the findings.

Another important limitation of this study is that it was carried out at a single center employing a locally developed RRS and RTCDAs. It is unknown whether this RRS will perform similarly in another hospital. Additionally, the RTCDA prediction model was developed and validated using only data from medical patients. Development and validation of similar alerts for other hospitalized populations are needed to make such systems more generalizable. It is also important to note that during this experience the RRS team nurse was established as a permanent position. This may have influenced the study results by allowing the RRS team nurse to be more fully available to the medicine units, potentially allowing more preemptive identification of deteriorating patients. Expansion of the RRS/RTCDAs has already occurred at Barnes-Jewish Hospital to include derivation and validation of RTCDAs for surgical patients. Another limitation of this study is that it did not employ real-time vital sign data for the development of the RTCDAs or for triggering the RRS. As automated collection of such data becomes available, with real-time transfer to informatics systems for non-ICU patients, it is expected that RTCDAs may become more accurate and improve the effectiveness of RRSs. The study team also could not ascertain the relative contribution of the RRS and RTCDAs on the outcome improvements observed in this study. Finally, other variables not examined in this study could have contributed to the outcome changes over time, representing another limitation of this analysis.

In summary, this study demonstrated that a locally derived RRS employing RTCDAs was associated with lower mortality, decreased occurrence of CPAs, and reduced hospital stay. These findings suggest that inpatient deterioration on general hospital units can be reduced and patient outcomes improved by the combined use of such systems. Continued efforts are needed to develop and optimize patient care systems that will not only accurately identify high-risk patients on general hospital units but also intervene to improve their outcomes.