Protocol for a new method to measure physiologic monitor alarm responsiveness on hospital units

Abstract

Evaluating the clinical impacts of healthcare alarm management systems plays a critical role in assessing newly implemented monitoring technology, exposing latent threats to patient safety, and identifying opportunities for system improvement. We describe a novel, accurate, rapidly implementable, and readily reproducible in-situ simulation approach to measure alarm response times and rates without the challenges and expense of video analysis. An interprofessional team consisting of biomedical engineers, human factors engineers, information technology specialists, nurses, physicians, facilitators from the hospital’s simulation center, clinical informaticians, and hospital administrative leadership worked with three units at a pediatric hospital to design and conduct the simulations. Existing hospital technology was used to transmit a simulated, unambiguously critical alarm that appeared to originate from an actual patient to the nurse’s mobile device, and discreet observers measured responses. Simulation observational data can be used to design and evaluate quality improvement efforts to address alarm responsiveness and to benchmark performance of different alarm communication systems.

Background

The Challenge of Measuring and Understanding Alarm Fatigue

Evaluating the sociotechnical impact and limitations of newly adopted biomedical technology is challenging and complex. The Joint Commission National Patient Safety Goal¹ and a growing body of literature recognize alarm fatigue as a healthcare hazard^2,3 and have drawn attention to the need to evaluate the effectiveness of available physiologic monitoring technology.^4,5

Alarm fatigue is defined broadly as the lack of or delay in response to alarms due to excessive numbers of alarms resulting in sensory overload and desensitization of hospital workers.^1,6,7 In children’s hospitals, patients generate between 42–155 alarms per monitored-day⁸ and only about 0.5% of alarms on pediatric wards are actionable.³ Up to 67% of alarms heard outside a patient room are not investigated⁹ and there is wide variation in response time.³ Although many commercially-available alarm management products promise improved alarm communication and patient safety, hospitals often lack the resources to measure the benefits and unintended adverse consequences of implementing these systems.

Analysis of the benefits (and hazards) of an alarm management system is challenging because alarm response is the result of complex interactions between physiologic monitors, the patient, the acuity of illness, the clinician’s competing tasks, and the dynamic hospital environment. Despite advances in characterizing alarm frequency and decision-making surrounding alarm response, evaluation of clinical response to truly actionable and life-threatening alarms has remained elusive. One method for assessing clinician response to alarms is through the use of video observation.¹⁰ Video observation of clinician alarm response is highly accurate, but is complicated and expensive to implement.¹¹ In addition, given the rarity of life-threatening alarms, video recording thousands of alarms is likely to reveal only a handful of truly critical actionable alarms.¹⁰ Novel approaches are needed to measure responsiveness to rare, life-threatening alarms and identify potential latent hazards in our current alarm system.

In-Situ Simulation for Analysis and Evaluation of Critical Alarms

Simulation has been used to evaluate clinician and system performance, particularly in emergent or infrequent clinical scenarios.^12–14 Kobayashi et al evaluated alarm responsiveness in the emergency department (ED) through simulation of a life-threatening arrhythmia.¹⁵ A simulator was connected to the in-room bedside monitor in an empty ED room, a life-threatening electrocardiogram tracing was generated, and clinical response time was measured by an observer. The simulation ended when any clinical provider responded to the life-threatening event or after 3 minutes. In their pre-intervention arm, only 1 of 20 simulation alarms (5%) garnered clinical response.

We developed an in-situ simulation approach to evaluate critical alarm response in a pediatric, non-intensive care setting, adapting Kobayashi et al’s simulation methodology. To our knowledge this form of in-situ simulation, in which simulated alarms appearing to come from an actual admitted patient are injected into a clinical environment to measure alarm response, has not been previously described. Our aim was to develop a method for efficient, rapid, repeated measurement of unit-level response to life-threatening events that trigger physiologic alarms and require an immediate response. Using response time and response rates to quantify alarm fatigue/responsiveness on a unit level, we sought to evaluate our current alarm management system and establish a baseline from which to measure the impact of future changes to the alarm system. The intent of the intervention was not to evaluate individual nurse performance, and individual-level response time data were not available to or shared with unit leadership to avoid the possibility of punitive actions for slow responses. This article describes reproducible methodology that may be useful to other institutions seeking to evaluate hospital alarm systems in non-intensive care, non-telemetry inpatient units.

Patient Safety Learning Lab

This work was part of a portfolio of quality improvement work undertaken by the Patient Safety Learning Laboratory (PSLL) at Children’s Hospital of Philadelphia. The PSLL is funded by the Agency for Healthcare Research and Quality, and one of its primary aims is to re-engineer the system of monitoring hospitalized children on acute care units, with a focus on reducing non-informative alarms and accelerating nurse responses to critical events. In the PSLL, our transdisciplinary team uses systems engineering approaches to proceed through a 5-step process for each aim: problem analysis, design, development, implementation, and evaluation. Evaluation of the relationship between alarm burden and nurse workload was part of the problem analysis phase of this improvement initiative.

The Committees for the Protection of Human Subjects (the IRB) at Children’s Hospital of Philadelphia determined that the problem analysis, design, and development phases of this project were consistent with quality improvement activities and did not meet criteria for human subjects’ research.

Setting

Hospital Environment:

Children’s Hospital of Philadelphia is an urban, tertiary care children’s hospital with a combination of private and semi-private rooms. Each inpatient bed is connected to a physiologic monitor (General Electric (GE) Dash 3000, 4000, or 5000) and are used for patients on general non-intensive care floors who require continuous physiologic monitoring, which requires a physician order. The monitors are capable of simultaneously displaying data from the following channels: electrocardiogram, respiratory rate, non-invasive blood pressure, pulse oximetry, and carbon dioxide. These monitors can physically be connected to a private hospital network (GE Carescape Network) through a wall jack (RJ45 connection) for every patient bed. The network is monitored via a network gateway (MMG ASCOM GE Gateway) that has the ability to route alarm notifications to handheld mobile devices (ASCOM d62 Phone Device). The network gateway has a user interface (GE Unite Assign) that allows users to interact with the server and assign specific patient physiologic monitors detected on the hospital network and route alarm messages to the handheld mobile devices carried by clinical staff.

Nurses are primarily responsible for responding to physiologic monitor alarms. Nurses on a general care unit typically care for three to five patients. Nurse/patient assignments are completed by unit charge nurses, who attempt to balance patient acuity, workload, and patient flow. Currently, continuous monitoring is not considered when evaluating workload for nursing assignment purposes.

Alarm Notifications:

Alarm “primary notification” occurs when the bedside monitor alarm sounds. Alarm “secondary notification” occurs when a notification appears on the handheld mobile device and on the centralized information center (CIC) monitor at the front desk. Each CIC displays a remote live view of up to 16 bedside monitors. When an alarm event occurs, the CIC also alarms and highlights which bed monitor is alarming. No individual is assigned to monitor the CIC. Each unit also has 1–2 remote view monitors of the main CIC screen in other locations. The remote monitors do not audibly alarm, but they do provide a live feed of the patient monitor visual display.

Currently, when a patient is connected to a physiologic monitor and a physiologic alarm is triggered, those data are transmitted to the hospital network which is monitored at all times by the network gateway. Based on the filters set in place in the network gateway, the gateway routes alarm messages to the ASCOM mobile device of the nurse who is assigned to that particular hospital bed during his/her shift (see Figure 1). Nurse ASCOM phones alarm with a ring tone accompanied by a text message including the room location and the alarm details (e.g.: “Bed 5S061 Warning SPO2 LO 76”, see Figure 2). The nurse can “Accept” the message and respond accordingly. When the alarm condition resolves the message is automatically removed from the phone.

Figure 1:

The network gateway is a server with filtering capabilities that sits on the hospital network and monitors the network at all times. When a patient has a bedside monitor that is connected to the hospital network, the gateway can be programmed to detect the monitor alarms, determine its location within the hospital, filter alarm messages based on alarm parameters, create an alarm message, and forward the message to a clinician’s handheld mobile devices.

Figure 2:

Parallel design of simulation unit monitor with patient’s real monitor: Depiction of a real patient monitor and simulated patient monitor connected in parallel without interference with one another. The yellow flow of data shows how the real patient bedside monitor connects from the bedside to the network, alarms are detected and filtered via the MMG Gateway filters. The hospital’s gateway network has a user interface in which users with access can assign the mobile phone device to receive all alarm messages. The blue parallel flow of data shows a simulator connected to a physiologic monitor named similarly to the true patient monitor with the number “0” in “Bed 5S061” replaced by the letter “O”. Its flow of data is otherwise similarly connected to the hospital network and alarm messages can be routed to the desired mobile device.

Simulation Design

Technical Goals

We sought to transmit a high fidelity, simulated critical alarm that appeared to come from an existing, admitted and actively monitored patient in order to observe and measure response times on 3 units. The critical alarm was sent to the patient’s primary nurse’s mobile phone and displayed on all unit CICs, enabling staff who heard or observed the alarm at the central nurse’s station or on the other unit CIC monitors to respond. To avoid disruption to patient-care or patient-clinician trust, we simulated only “secondary notification” (to mobile phone and CIC display), avoided entering patient rooms, and took steps to ensure there was no delay or interference in patient care. Prior to beginning the in-situ simulations, all unit nurses were informed that alarm simulations would be taking place over the coming months as part of a quality improvement effort to better understand and combat alarm fatigue.

Simulation scenario, endpoint, and outcomes

We simulated a critical hypoxemic event, with a pulse oximeter oxygen saturation (SpO₂) between 60–70%. SpO₂ alarms were chosen because they are the most frequent in pediatric hospitals. Though <2% of SpO₂ alarms are actionable,³ true and sustained low SpO₂ will result in cardiac arrest if unresolved.¹⁶ Simulations ended when either a clinician responded to the alarm or 10 minutes passed without a response. We selected the 10-minute maximum based on data from animal models approximating toddler cardiac arrest which demonstrated that sustained hypoxia progresses to pulseless electrical activity and cardiac arrest in approximately half of patients in under 10 minutes.¹⁶

The primary outcome of each simulation was time to response by any clinical staff to the simulated critical alarm, starting from receipt of critical alarm message on the mobile phone devices (t=0 min). An observer was discreetly positioned in the hallway with a line of sight to the door of the patient’s room to assure no bedside alarms were silenced, confirm that alarm systems were functioning, and verify whether a clinician entered the room during the simulation. The simulation was discontinued with any clinician entry into the patient room even if the clinician entered to perform routine care (e.g., administer medication, check an intravenous line) and was not responding to the simulated critical alarm. This accounts for the fact that in true patient decompensation, the bedside monitor alarm (“primary notification”) would alarm and staff inside the room would presumably respond accordingly. Observers recorded simulation end-time and qualitative field notes including whether anyone interacted with the CIC, whether the CIC was making alarm noises, or whether they observed anyone silencing a CIC or phone alarm (Table 1).

Table 1: Simulation Observations.

Simulation observational data and reflections from the debriefing interview were recorded in a RedCap survey immediately following the simulation.

Session Information

Date simulation performed Unit simulation was performed Start and end time of RN’s shift Number of total patients assigned to RN at time of simulation MRN and bed location of RN’s assignment Acuity index of RN’s overall assignment Acuity index of Individual patients captured in the EHR Total alarms and level of those alarms received in the hour leading up to the simulation alarm

Time Point Metrics

Time of low SpO2 alarm on the monitor Time of notification on the ASCOM phone Time of RN or responding staff member to bedside or t=10 min

Observer Gathered Data

Is the CIC making noise during the simulation? Did you notice anyone interact with the CIC during the simulation alarm? How did they interact with the CIC monitors? Was any family at the bedside? Where is RN carrying the ASCOM? (chest, hip, etc)? What caused the end of the simulation? Is the bedside nurse the responder to the alarm?

Debriefing interview with the bedside RN

What other tasks were you engaged in when the monitor’s alarm went off on your ASCOM? Or how would you describe your activities within the last 10 minutes? What did you think when you first got the alarm? Did you think this was a credible alarm? Why or why not? How did you prioritize the alarm vs your other tasks? What are your thoughts about the tone of the alarm, do you recall what the message said? What audible and vibrate settings was your phone set at?

Inclusion criteria

In collaboration with hospital leadership, inclusion criteria were developed to establish an accurate and realistic representation of threats within the current alarm system. We sought to simulate alarms among patients who are particularly vulnerable in our current alarm system (rather than adopting a random sampling approach). Therefore, we defined inclusion criteria for patient room selection including: patient in the room must be on a continuous monitor at the time of the simulation, should be cared for by a nurse who works regularly on the unit, must not have experienced a code-event or rapid-response in the prior 24 hours, and were not designated by the unit as “watchers” (receiving additional surveillance due to heightened risk for clinical deterioration). These criteria were selected in order to evaluate response among patients at risk of harm from alarm fatigue in our current alarm system (as opposed to patients who already receive heightened surveillance within our unit). While such criteria by themselves may be subject to nurse response bias, the level of desaturation (SpO₂ of less than 70%) simulated was intentionally set at a signal threshold that should have superseded existing bias. We did not repeat simulations on the same day within the same unit, to avoid priming responses.

Number of Simulations

We sought to realistically simulate what should be “never events” within a high reliability organization, to evaluate potential vulnerability within our alarm system. In order to determine the optimal number of simulations to conduct, we discussed with hospital and unit leadership the number of simulations required to demonstrate with sufficient statistical confidence the safety of our alarm system. We elected to conduct 20 simulations because it provided an appropriate level of precision in reporting response rates with a feasible completion time frame of 3 months. For instance in the case of perfect response, in which nurses responded to 20 of 20 of simulations (100%, 95% CI: 83 – 100%). If response was observed in 10 instances (50%) the 95% CI of response would be 27– 73%.

Team

We formed an interprofessional team consisting of biomedical engineers, human factors engineers, information technology (IT) specialists, clinical nurses, physicians, facilitators from the hospital simulation center, clinical informaticians, and hospital administrative leadership to create and implement the simulation. The clinical engineering and information technology (IT) departments facilitated in (1) acquisition of simulation and monitoring technology to replicate critical alarms, (2) reconfiguration of the gateway server to recognize the simulation equipment and (3) routing simulated alarm messages to the appropriate CIC and mobile devices.

The simulation team, human factors engineers, and clinical nursing leadership at the unit and hospital level sought to ensure the integrity and acceptability of in-situ simulation design. Aware that this simulation could expose systems vulnerabilities (for instance, delayed or absent response to life threatening alarms) we sought broad-based input on how to maintain psychological safety, defined as “the degree to which team members feel that their environment is supportive of asking for help, trying new ways of doing things, and learning from mistakes.”¹⁷ To this end, nurses and simulation facilitators developed pre-simulation language and post-simulation debriefing scripts to explicitly frame this simulation as systems evaluation (rather than comparative assessment of individual- or unit-level performance).

Simulation Equipment

Hospital Network and Server changes to accommodate Simulation Unit:

Our engineering and information technology (IT) departments created a virtual simulation unit within the hospital’s network gateway (MMG ASCOM GE Gateway) to deliver simulated alarm notifications directly to nurses’ mobile devices and CICs. The virtual hospital unit (named “Simulation Unit”) in the gateway was created using the gateway user interface and populated with unique hospital bed names that appeared visually similar to existing hospital beds (for instance replacing the numeric “0” with the letter “O”). When a physiologic monitor is programmed with the bed name of one of the Simulation Unit beds and is connected to the private hospital network, it can be detected by the network gateway. The monitor then appears in the gateway user interface and a user, such as a charge nurse, can assign the simulation monitor to an active mobile phone device so that the phone receives simulation monitor alarms. Because this monitor’s bed name is unique and not programmed into any other downstream system, it is not recognized by systems outside of the network gateway, preventing downstream consequences such as abnormal data abstracted into the electronic health record (Figure 2).

Simulation Sessions

On the day of a simulation, unit nursing leadership identified eligible patient rooms and selected one using a random number generator. Unit leaders were notified that a simulation would take place, but not the specific room number selected.

A team including a nursing leader, biomedical engineer, simulation observer, and a simulation center facilitator/debriefer assembled with necessary equipment just outside the unit (Table 2). The biomedical engineer and nursing leader programmed the simulator to mimic the selected patient’s true vital signs and connected the simulator to the simulation bed monitor (GE Dash 3000 Series Monitor). The monitor was connected to the hospital network via the RJ45 connector in the data closet on the unit. The engineer pulled the simulation monitor into view on the central workstation display so the critical alarm would be displayed at the central nursing station and other unit central displays. The nursing leader then logged into GE Unite Assign to assign the simulation monitor to the nurse caring for the selected patient room and to a control ASCOM phone held by a team member. To ensure patient safety during the simulation, all the participating nurse’s true patient monitor alarm messages and calls were routed to the unit charge nurse.

Table 2:

Equipment required and used for simulation

Need	Equipment used:
Physiologic monitor	GE Dash 3000 Series
Full Monitor Simulator	FLUKE ProSim 4
Pulse Ox-Jack	RJ45 Connection
Application Processes: Gateway Server access Patient Call Routing	Applications Used: MMG Ascom Unite Assign User Interface simulation log-in account Responder 5 Nurse Call Application

At the start of the simulation the engineer programmed the simulator to a pulse oximeter value between 61–69%, which when detected by the simulation GE Dash 3000 monitor triggered an alarm. The gateway identified the alarm coming from the simulation monitor and routed the message to the nurse’s mobile device and to the team’s control mobile device, signaling the start of the simulation (t=0).

An observer stood within view of the selected patient room to intercept the clinician responding to an alarm. The simulation ended when a clinician arrived at the patient bed or after 10 minutes. Observations and timing were recorded by the observer into a redcap survey.^18,19 The observer recorded interactions with the CIC and monitor, how the RN interacted with the phone while it was alarming, and how the simulation ended (Table 1). The observer also collected information about the patient and nursing assignment. Following the simulation, the debriefer reviewed goals of the simulation with the bedside nurse and solicited suggestions on improvements to the unit alarm system.

Limitations

This methodology must be considered in light of its limitations. First, we did not design this simulation to shed light on the mechanisms that explain delayed responses to alarms. Rather, we sought an overall indicator of the clinical unit’s performance within the socio-technical environment as a whole in response to critical alarms. Second, we created the simulations only on general medical-surgical units, thus limiting the applicability of this method to ICU or telemetry patients and other types of units with different staffing models and alarm systems. Third, we developed and tested this method in a pediatric hospital. Although we believe that alarm response time is a relevant outcome measure across patient populations, some of the aspects of the simulation might need to be modified for use in adult settings. Lastly, our method gives information about specific alarm notification delivered by one mechanism so cannot be extrapolated to different types of alarms (i.e. primary notification, arrhythmia, etc).

Conclusion

As healthcare systems adapt to the rapidly changing landscape of communications technology, it is important to understand the effects and latent threats of these technologies. We describe a reproducible approach using existing hospital technology to test and quantify alarm response times and rates without interfering with routine patient care. We intend to use simulation observations and outcomes to design and evaluate quality improvement efforts to address alarm fatigue. In the future, such simulations could provide a framework to benchmark performance with different communication and alarm systems.