Improving Situation Awareness to Reduce Unrecognized Clinical Deterioration and Serious Safety Events

Abstract

BACKGROUND AND OBJECTIVE:

Failure to recognize and treat clinical deterioration remains a source of serious preventable harm for hospitalized patients. We designed a system to identify, mitigate, and escalate patient risk by using principles of high-reliability organizations. We hypothesized that our novel care system would decrease transfers determined to be unrecognized situation awareness failures events (UNSAFE). These were defined as any transfer from an acute care floor to an ICU where the patient received intubation, inotropes, or ≥3 fluid boluses in first hour after arrival or before transfer.

METHODS:

The setting for our observational time series study was a quaternary care children’s hospital. Before initiating tests of change, 2 investigators reviewed recent serious safety events (SSEs) and floor-to-ICU transfers. Collectively, 5 risk factors were associated with each event: family concerns, high-risk therapies, presence of an elevated early warning score, watcher/clinician gut feeling, and communication concerns. Using the model for improvement, an intervention was developed and tested to reliably and proactively identify patient risk and mitigate that risk through unit-based huddles. A 3-times daily inpatient huddle was added to ensure risks were escalated and addressed. Later, a “robust” and explicit plan for at-risk patients was developed and spread.

RESULTS:

The rate of UNSAFE transfers per 10 000 non-ICU inpatient days was significantly reduced from 4.4 to 2.4 over the study period. The days between inpatient SSEs also increased significantly.

CONCLUSIONS:

A reliable system to identify, mitigate, and escalate risk was associated with a near 50% reduction in UNSAFE transfers and SSEs.

Rapid response teams (RRTs) are designed to identify and respond to events in the hours prearrest.^1–13 Although interventions and contexts have varied substantially, these teams have demonstrated decreased codes outside the ICU and hospital-wide mortality in several studies.^{1–4,7,9,10,12,13} Variation in the effectiveness of RRTs may be due to insufficient processes around monitoring and risk identification.¹⁴ Potentially preventable morbidity and mortality from unrecognized deterioration remain, often due to ineffective clinical monitoring that we believe represents poor situation awareness (SA).¹⁴ SA (ie, “knowing what is going on”) exists at 3 levels and is defined as “the perception of elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future.”^15–17 We believe improved SA drives better recognition of early deterioration and is essential in efforts to reduce “failure to rescue” from codes outside of the ICU, an event associated with a 50% to 67% mortality.^18,19

High-reliability organizations (HROs) (eg, commercial aviation, nuclear power, and wilderness firefighting) deal with constant and catastrophic risk yet maintain exemplary safety records.²⁰ Our institution began a journey to become an HRO with the Agency for Healthcare Research and Quality HRO Learning Network in September 2005.²¹ Learnings from this network fueled improvement work and introduced the concept of SA. Our organization has targeted serious safety event (SSE) reduction as a strategic improvement goal since 2006. Our efforts to reduce SSEs, defined as severe harm or death after variation from expected practice, have resulted in significant and sustained reduction.²² Before SA work, SSEs among inpatients had not decreased. Poor SA was a common etiology. To achieve our aim and facilitate rapid learning, our project team defined a precursor outcome measure that we believed would capture events that both represented SA failures and occurred more commonly than inpatient SSEs in our center.²³ We prospectively defined unrecognized situation awareness failures events (UNSAFE) as the transfer of patients from the acute care floor to the ICU where the patient received tracheal intubation, initiation of vasoactive medications for hemodynamic support, or ≥3 fluid boluses in the first 60 minutes of ICU care or before arrival in the ICU. We believed these events represented potentially delayed transfers that are precursors to codes outside the ICU or SSEs. With focused improvement work beginning in November 2009, we aimed to decrease UNSAFE transfers by 50% by June 30, 2011. We hypothesized that a system of care that proactively identified, mitigated, and escalated risk would improve SA and decrease UNSAFE transfers and SSEs.

Methods

Setting

Cincinnati Children’s Hospital Medical Center is a 523-bed academic, quaternary-care, free-standing children’s hospital. Resident teaching teams care for the majority of hospitalized patients. Less commonly, direct hospitalist and nurse-practitioner models are used. Our RRT (called a medical response team [MRT]) has been in place since 2006 with defined activation criteria.² A modified version of the Monaghan pediatric early warning score (PEWS) was tested and spread across the hospital in 2007.²⁴

Human Subjects Protection

Our study was reviewed by the institutional review board and deemed exempt systems improvement. Individual patient care was discussed among clinicians in the course of identification and mitigation of risk. Medical record review was performed by the lead investigator by using a secure password-protected internal database and our hospital’s electronic health record (EHR).

Event Review

Two investigators (Dr Brady and Ms Goodfriend) reviewed 20 consecutive SSEs and 80 consecutive ICU transfers to identify potential predictors of deterioration. The presence of at least 1 of the following 5 risk factors was found in each case: (1) family concern about patient safety, (2) high-risk therapies including unfamiliar therapies on the unit (eg, insulin use outside of the diabetes unit), (3) elevated PEWS of ≥5, (4) watcher or a patient where a clinician had a “gut feeling” that the patient was at risk for deterioration or “close to the edge,” and (5) communication concern that may impact patient safety.

Intervention

The SA intervention included the following: (1) a formalized process where bedside nurses proactively identified these 5 factors, (2) unit-based huddles where charge nurses and physicians discussed identified factors and developed mitigation plans, (3) initiation of 3-times daily inpatient huddle where individual patient risk was discussed and specific predictions made, (4) development of a continuous learning system to evaluate SA and UNSAFE transfers, and 1 year later (5) development of a “robust” and explicit plan for patients identified as having 1 of the risk factors. Figure 1 provides a model of communication and action pathway for identification of patient risk. Figure 2 is the key driver diagram that illustrated the study team’s belief in hypothesized drivers needed to achieve the aim. Table 1 provides details on individual interventions.

FIGURE 1.

Identify, mitigate, and escalate model illustrates which risk factors were systematically identified and how standardized communication about risk occurred throughout the center.

FIGURE 2.

Key driver diagram illustrates the drivers (at right) that would lead to aim through improved situation awareness and no unrecognized clinical deterioration.

TABLE 1.

Specific Interventions, Settings, and Timing for Each Intervention

Category of Intervention	Specific Interventions	Setting	Timing
Proactive identification of risk	Risk categories prototyped and tested on small scale	1 general pediatric unit	November 2009
	Algorithm developed and tested throughout unit	1 general pediatric unit	December 2009
	Algorithm tested and adapted on different patient populations	4 units including subspecialty and surgical care	January 2010
	Algorithm posted and spread throughout hospital	All acute care units	March 22, 2010
Unit-based huddles	Huddle tested on small on 1 unit	1 general pediatric unit	November 2009
	Adapted to include residents only when risk identified	1 general pediatric unit	December 2009
	Tested and adapted on 4 units	4 units including subspecialty and surgical care	January 2010
	Didactic and case-based education for charge nurses	Conference room	February and March 2010
	Spread throughout hospital	All acute care units	March 22, 2010
Three-times daily inpatient huddle	Safety officer attends and each charge nurse lists any patient risks that were not fully addressed with predicted discharges and admissions	4 test units at 8 am	January 2010
	Safety officer and MPS round on each unit	4 test units at 4 pm	January 2010
	MPS rounds on each test unit	4 test units at 12 am	January 2010
	3-time daily inpatient huddle extends to all units	All acute care units	March 22, 2010
	Explicit predictions for calls of medical response team made	All acute care units	April 2010
	Afternoon rounds moved to huddle in conference room with each charge nurse in attendance	All acute care units	October 2010
	Overnight rounds moved to huddle in conference room	All acute care units	January 2012
Continuous learning system	ACA form prototyped	4 units including subspecialty and surgical care	January 2010
	Weekly report with rates of risk identification and escalation and UNSAFE transfers along with patient-level story about situation awareness	All acute care units	March 2010
	Control plan developed for identifying and acting on special cause with process measures and UNSAFE transfers	Used on all acute care units as needed	July 2010
	Database that combined data from ACA forms and admit/transfer data from EHR developed and tested and went into production	Used by Manager, Patient Services to track patients on all acute care units	September 2010
Robust plan	Robust plan checklist generated and tested by 1 charge nurse	1 unit that specialized in transitional tracheostomy and ventilator care	March 2011
	Checklist adapted and tested with all nurses on unit	Same transitional care unit	April 2011
	Physician event note template created and tested in EHR	Same transitional care unit	June–July 2011
	Identified risk factors placed in EHR in format to be scanned by safety officer and other leaders	1 neurosciences unit	July–August 2011
	Checklist, template, and risk factors in EHR spread	All acute care units	September 2011

Proactive Identification of Risk

In our SA model (Fig 1), the fundamental job of the bedside nurses and interns was to identify any of the 5 risk factors. These clinicians have the most touch time with the patients but are generally the least experienced on the team. Tools were developed to support bedside nurses in identifying risk factors during their routine assessments. Structured yes/no questions (Fig 3) were prototyped, tested, and later spread.

FIGURE 3.

Situation awareness algorithm illustrates the tool used during education and early phases and the specific questions and communication pathways.

Unit-Based Huddles

Unit-based huddles between the charge nurse and bedside nurse, regardless of identified risk, were scheduled every 4 hours. Huddles also were promoted whenever new risk factors were identified. The aim of the huddle was to trigger a bedside evaluation by experienced nurses and physicians of any patient with the identified risk. Huddles were led by a watchstander charge nurse and senior resident when risk was identified. The term “watchstander” was borrowed from the military to highlight that the primary job of the charge nurse and senior resident was to know which patients were at high risk for deterioration. When concerns were identified, the clinical care team discussed the risk at the patient’s bedside and developed a plan to mitigate that risk. This provided a standardized opportunity for more experienced clinicians to coach those less experienced in both patient management and communication/escalation techniques.

Three-Times Daily Inpatient Huddle

Our design later leveraged existing structures to proactively escalate identified and unresolved safety risks. Historically, our organization conducted an 8 am huddle that brought together a charge nurse from each inpatient unit and the manager of patient services (MPS) who oversaw the flow and staffing of inpatients. Before our work, there was no standardized discussion of patient safety. We piloted with 4 units and then spread to all noncritical care units a structured process where the charge nurse from each unit (1) reported on any risk factors present that were not fully addressed and (2) predicted any MRT activations. This process continued to be facilitated by the MPS, and a safety officer of the day (SOD) attended each inpatient huddle. The SOD was an experienced pediatrician who was given the authority by our hospital leadership to assist with mitigating any safety and/or communication concerns on the inpatient units. The MPS and SOD provided coaching on how to address concerns raised by the primary clinical team, including positive reinforcement of key behaviors and role modeling. The details of testing similar huddles at 4:30 pm and midnight are displayed in Table 1.

Continuous Learning System to Evaluate SA

The final component of our planned intervention was to develop a data system to rapidly identify process and outcome failures and direct this information to project leaders and leaders on individual units. To achieve this aim, (1) apparent cause analysis (ACA) forms were completed within 1 hour of each floor to ICU transfer to identify potential UNSAFE transfers and associated process failures, (2) a password-protected database was constructed to integrate information from these forms and the EHR, (3) process and outcome data were distributed each week to unit level clinical and medical directors with a story of patient-level SA, and (4) a control plan was designed with inpatient leaders to identify special cause on tracked process and outcome measures and target further interventions.

Robust Plan

One year after SA work began, the improvement team worked with 1 inpatient unit to develop and test a checklist tool to improve the mitigation/escalation process for patients with identified risk (Table 1). During multidisciplinary discussions, we proposed a “robust plan” bundle which included the following: (1) plan with proposed treatment change, (2) explicit communication with care team, (3) prediction of expected outcome, (4) outcome deadline, and (5) escalation plan (usually the MRT or discussion with SOD/MPS) if outcome was not achieved by a predefined deadline. This tool was tested, adapted, and spread throughout the remaining inpatient units (Fig 4). Subsequent testing integrated risk identification within the EHR and added focused discussion of a robust plan during safety rounds.

FIGURE 4.

Situation awareness robust planning tool.

Study of the Intervention

In our observational time series study, data were collected on process measures of systematic identification, mitigation, and escalation of risk that we believed would improve SA and decrease UNSAFE transfers and SSEs. To evaluate the consistency of huddles and how well the identify, mitigate, and escalate intervention was implemented, data initially were collected from each unit on each nursing shift to measure the reliability that each shift identified all patients at risk and mitigated or escalated that risk. This was captured through a checklist-based form that followed the flow of algorithm of Fig 3 and was completed by each charge nurse. The tool was tested and evaluated with charge nurses from several units during early phases. Before spread throughout the hospital, 116 charge nurses received training on the process and tool through a 1.5-hour learning session. Validity of process data were evaluated through discussions during inpatient huddles by investigators, SOD, and MPS. UNSAFE transfers were identified from the ACA process and validated against review of the EHR for each ICU transfer. SSEs were captured through a safety reporting process as previously described.²²

Analysis

Primary analysis of both process and outcome measures was performed by using statistical process control charts. For the primary outcome of UNSAFE transfers, results were tracked by using both a days-between t-chart and rate chart. Established rules for identifying special cause were employed.^25–27

Results

After testing on 4 inpatient units from January 1, 2010, to March 21, 2010, the unit-level huddles and proactive inpatient huddles began on each of the 14 noncritical care inpatient units on March 22, 2010. The process measure evaluated the consistency of huddles and specifically how frequently patient risk was identified and mitigated or escalated each nursing shift on each unit. The number of units by week where ≥90% of weekly nursing shifts fully identified and mitigated or escalated patient risk were tracked on run charts and revealed both improved and sustained performance for 11 months of tracking (Fig 5). On each participating unit, 90% to 95% of identified risk was mitigated by the primary team with no escalation needed. Each inpatient huddle took less than 30 minutes. Although initially there was substantial variation in the number of patient risks that were escalated, a median of 2 risks for each huddle were escalated the first year. This increased over the study period with a median of 7.5 concerns escalated in May 2012.

FIGURE 5.

Process measure run chart illustrating the number of units by week where ≥90% of weekly nursing shifts fully identified patients at risk (solid line/diamond) and where ≥90% of weekly shifts fully mitigated or escalated that risk (dotted line/circle).

The rate of UNSAFE transfers per 10 000 non-ICU inpatient days is displayed in Fig 6. An initial decrease in UNSAFE transfers occurred, though it did not meet rules for special cause and was not sustained. Analysis of UNSAFE transfers through an ongoing ACA process revealed that in the vast majority of UNSAFE transfers, patient risk had been identified but not fully mitigated on unit or escalated to the MRT or safety team. This led to focused improvement work on development of a robust plan as detailed above. After spread, the rate of UNSAFE transfers improved from a baseline of 4.4 to 2.4 transfers per 10 000 non-ICU inpatient days, meeting criteria for special cause variation with 8 points below the median line (Fig 6). Additionally, a significant change in the days-between inpatient SSEs from 100 days to >400 twice was observed in association with the intervention. Shortly before this work began, the number of MRT activations and PICU transfers per month increased significantly in association with a campaign to eliminate informal PICU consults (Fig 7). MRT-preventable codes outside the ICU were rare before and during the study period (Fig 8).

FIGURE 6.

UNSAFE transfer rate chart. Rate of UNSAFE transfers per 10,000 non-ICU patient days at base location by month (n = non-ICU inpatient days by month).

FIGURE 7.

Rate of MRT activations and transfers to ICU by month.

FIGURE 8.

Rate of MRT preventable codes outside of the ICU by month.

Discussion

Our system to reliably identify, mitigate, and escalate patient risk was associated with a significant decrease in UNSAFE transfer and SSEs among inpatients. By providing a proactive and reliable model to identify and rapidly intervene on newly detected risk and early patient deterioration, we improved SA across our inpatient system. SA is achieved by scanning for important information, recognizing patterns and trends, and making short-term predictions.^15–17 We believe that although SA is still uncommonly discussed in the medical literature, interventions such as rapid response systems and early warning scores (both in broad use) employ SA to identify deteriorating patients.²⁸ We sought to build upon this work by addressing limitations in many rapid response systems’ inherently reactive (versus proactive) nature.¹⁴

We developed the concept of UNSAFE transfers to be used as the primary outcome measure for improvement efforts with the belief that these events represented potential precursor events to serious harm such as codes and because these events occurred sufficiently often in our center to enable rapid testing, learning, and adapting. Our time series design allowed us to learn sequentially, and we observed an initial decrease in the rate of UNSAFE transfers followed by a return to our previous baseline levels. With process measures in place that did not reveal a decrease in reliability of our intervention, we further studied where the process failures occurred regarding UNSAFE transfers. With this data, we learned that although 1 year into the intervention we had a system that identified risk, we did not systematically address this risk. We found that even on our most high risk patients, the language of plans included terms such as “continue to observe.” Without explicit and time-bound plans, clinicians were observing patients until they received aggressive resuscitation that met criteria for UNSAFE transfer.

Our second stage of interventions testing a robust plan and prediction was designed by a multidisciplinary team of leaders and front-line physicians, nurses, and respiratory therapists. The improvement team believed that a shared mental model or team SA would not be achieved without explicit prediction and contingency planning. We believed that this was because level 3 SA (the projection of current event status in the near future) was still often not achieved. This may have been due to the limits of individual clinicians in making near-term projections (eg, this tachycardic patient will be in shock within 4 hours if we do not aggressively hydrate) but was believed to more commonly result from doctors, nurses, and other members of the care team not explicitly sharing their mental model. Our basic theory was borrowed from hypothesis testing in the scientific method and explicit predictions in plan-do-study-act cycles. The goal was to increase accountability and to make disconcerting data (eg, patient did not improve as predicted) clear to each member of the team. The spread of this intervention and its integration into proactive inpatient huddle was associated with a sustained decrease in UNSAFE transfers.

One strength of our work is that we created a system of care built on reliable processes, not individual clinicians. We were able to build these processes into the workflows of busy clinicians and provide via the inpatient huddle a valued activity for charge nurses as they gained insight and assistance with their sickest patients. Our sustained reduction in UNSAFE transfers over the last 12 months is further evidence of the success of building interventions into work flow. Importantly, this intervention did not add additional clinicians to our system of care but instead clarified roles and processes for charge nurses, MPS, residents, and attending physicians. The additional responsibilities of SODs are on the order of 1.5 hours per day. We believe our work builds upon previous interventions to address patient deterioration such as rapid response systems and PEWS. The proactive and standardized nature of our intervention offers an important answer to afferent limb failures of the MRTs.¹⁴ Our intervention supplements the early warning score with other risk domains, most powerfully for us was that of the “watcher” or patient that a clinician has “a gut feeling is close to the edge.” We believe this employs the tacit knowledge of experienced clinicians and hence likely will achieve greater sensitivity than any numerical scoring tool, especially since we combined this concept with objective data.^29,30 We also believe that assessment of risk as relayed by family and as emerges from communication problems has substantial face validity in identifying and predicting deterioration. Our final risk category was that of high risk therapy and borrowed from HRO thinking on the need for special oversight and procedures with new and unfamiliar therapies; for example, we believe the administration of insulin on short-stay surgery unit has a fundamentally different risk profile than doing so on diabetes unit. We therefore target these high risk therapies and address any knowledge gaps in close to real-time. Our intervention is perhaps most similar to that of the Rover team as described by Hueckel et al.³¹ Although both are proactive in assessment of risk, meaningful differences include our intervention’s staffing model and broader scanning for risk.

Because our goal was rapid improvement of a single site, we chose a time series design, which did not allow us to address secular trends or establish causality. We believe this design was appropriate for our innovative intervention that evolved and improved through iterative quality improvement methods. This design exposed the study to potential unmeasured confounding from safety work. We do not feel this was a particularly large risk because time series data reveal the rate of inpatient SSEs had not improved in previous years’ safety work and because there were no other large interventions targeted at this population. Additionally, it is uncertain how our results would generalize to medical centers with different patient populations, staffing models, quality improvement capabilities, and safety cultures. A final limitation is that we did not have a measure of SA to reveal that this improved as an effect of our intervention. Available measures of SA involve simulated events and typically require “pausing” the event to perform assessment.³² Clearly this was not possible or ethical in course of clinical care. A recent proposed measure of SA relied on accuracy of prediction that was fundamental to our work.³³ We did use SA for much of our conceptual model, but we cannot say definitely if improved SA was the mediator between the identify, mitigate, and escalate intervention and our decreased rate of UNSAFE transfers.

Our institution additionally has applied and tested models to improve SA and decrease adverse events in diverse clinical settings including the operating room and resident psychiatry. We also believe improved SA and systems that proactively identify, mitigate, escalate, and predict risk have applicability to other untoward events such as flow and patient/family experience failures. Next steps include tailoring and testing interventions in these settings and in other hospital systems that have different patient populations and systems of care. Although we have begun to incorporate risk factors into the EHR, we believe simulation and human factors methods are needed to evaluate the role of technology and automation in improving SA and facilitating clinician workflows.

Conclusions

A reliable system to identify, mitigate, and escalate risk can be implemented in a children’s hospital and is associated with a reduction in safety events in a context where these events were already uncommon. We believe HRO thinking and SA are potentially transformative concepts for health care systems. Models to identify risk early and reliably intervene are likely generalizable both to different clinical systems and to modify different outcomes such as the patient/family experience and patient flow.

Glossary

ACA

apparent cause analysis

EHR

electronic health record

HRO

high-reliability organization

MPS

manager of patient services

MRT

medical response team

PEWS

pediatric early warning score

RRT

rapid response team

SA

situation awareness

SOD

safety officer of the day

SSE

serious safety event

UNSAFE

unrecognized situation awareness failure event