Electronic Alerts for Triage Protocol Compliance Among Emergency Department Triage Nurses: A Randomized Controlled Trial

Abstract

Background

Alerts embedded in electronic health records are designed to improve processes at the point of care.

Objective

To determine if an electronic health record (EHR) alert—at emergency department (ED) triage—increases the ED triage nurse’s utilization of triage protocols.

Methods

ED triage nurses were randomized to receive either a passive EHR alert or no intervention for patients with the following complaints: minor trauma with extremity injuries or females with abdominal pain. The EHR alert notified the nurse that the patient was eligible for diagnostic testing: radiographs (XR) for patients with injured extremities or urinalysis (UA) for females with abdominal pain.

Results

Twenty-eight nurses triaged 20,410 patients in the six months prior to intervention and 19,157 in the six months post. Prior to intervention, the UA protocol was implemented in 101/624 (16.2%) patients triaged by the intervention group and 116/711 (16.3%) triaged by the control group. After the intervention, the UA protocol was implemented in 146/530 (27.6%) patients triaged by the intervention group and 174/679 (25.6%) triaged by the control group. Prior to intervention, the XR protocol was implemented in 58/774 (7.5%) patients triaged by the intervention group and 45/684 (6.6%) triaged by the control group. After the intervention, the XR protocol was implemented in 78/614 (12.7%) patients triaged by the intervention group and 79/609 (13.0%) triaged by the control group.

Conclusion

The use of a passive EHR alert to promote ED triage protocols demonstrated little benefit. Prior to widespread implementation of EHR alerts for patient care, rigorous studies are required to determine the best alert methods and the impacts of such interventions.

The electronic health record (EHR) is being touted as providing new opportunities to advance healthcare by improving workflow, and minimizing deficiencies and errors in paper charts. Electronic alerts are now being implemented into many EHR systems. These alerts are designed to improve patient care by suggesting best practices, alert health care providers (nurses, pharmacists, and physicians) of potential errors, or provide surveillance to identify patients at risk for certain specified disorders (Cho et al., 2012; Colpaert et al., 2007; Herasevich, Yilmaz, Khan, Hubmayr, & Gajic, 2009). The most appropriate type of alert, however, remains unknown. Several examples of beneficial alerts exist, improving care in patients at risk for contrast induced nephropathy (Cho et al., 2012), decreasing inappropriate cardiac markers (Baron et al., 2012), increasing venous thromboembolism prophylaxis (Lecumberri et al., 2011), and increasing awareness of drug resistant bacteria (Kac et al., 2007). These prior studies, however, utilized “before and after” methodology with limitations inherent to that study design. Thus, it is difficult to assess the true benefits of EHR alerts in those studies versus increased provider awareness occurring with time. Furthermore, these studies were performed outside the unique environment of the emergency department (ED).

Other EHR alert studies, however, suggest these alerts have minimal to no benefit. Alerts to control blood in the intensive care unit, improve gastrointestinal protection in high-risk patients, and increase/improve physician prescribing practices, all had little to no benefit (Coté et al., 2008; Mathias, Didwania, & Baker, 2012; Meyfroidt, Wouters, De Becker, Cottem, & Van den Berghe, 2011; Strom, Schinnar, Bilker, et al., 2010). Issues related to failure of EHR alerts are not well studied, but alert fatigue/desensitization is a recognized problem (Rothman, Leonard, & Vigoda, 2012; van der Sijs, van Gelder, Vulto, Berg, & Aarts, 2010). Close observation of physician handling alerts suggest 30% are handled incorrectly (van der Sijs et al., 2010), and clinicians ignore the majority of medication alerts (Lin et al., 2008; Payne, Nichol, Hoey, & Savarino, 2002; van der Sijs, Aarts, Vulto, & Berg, 2006; Weingart et al., 2003). These findings support the idea that rigorous research is now required prior to widespread implementation of EHR alerts (Chrischilles et al., 2002; Lapane, Waring, Schneider, Dubé, & Quilliam, 2008; Lin et al., 2008; Strom, Schinnar, Aberra, et al., 2010; van der Sijs et al., 2010). Characteristics for optimal decision support alerts for medication prescribing are described, and must be focused and simplistic for the clinician (Horsky, Phansalkar, Desai, Bell, & Middleton, 2013). The limited available evidence, however, indicates that the best design and implementation of EHR alerts is not completely understood—especially in the ED.

ED length of stay is decreased when diagnostic tests are ordered by the triage nurse (Rowe et al., 2011). This practice, however, is not routinely performed. This study’s goal is to determine if a simple EHR alert, at ED triage, increases the triage nurse’s utilization of standard ED triage protocols for patients with extremity injuries after minor trauma and/or female patients with abdominal pain. We hypothesize that the use of information technology during ED triage through real-time EHR reminders to triage nurses will increase protocol implementation.

Methods

Design and Setting

We performed a randomized controlled trial (RCT) to determine if receiving an EHR alert during the patient triage process increased a triage nurse’s application of two standard ED triage protocols. The study was conducted at an urban, level 1 trauma center with an annual ED census of 60,000 patients. The institutional review board (IRB) at the study site approved the protocol.

Sample

Study participants were registered nurses who served in the capacity of external ED triage nurse. The external ED triage nurse performs triage on walk-in patients (i.e., no patients transported by ambulance), and documents patient complaint and a brief screening examination. Nurses who floated from another unit and thus were ineligible to serve as a triage nurse, were excluded. Written informed consent was required from all participating nurses and included consent for collecting individual nurse data from July through the following June.

Intervention

The intervention consisted of an EHR alert provided to the triage nurse during the nurse’s evaluation of a patient. Once the specific triage complaint was entered into the EHR, the computer immediately provided an electronic alert to the computer screen (while the patient was still in the triage room), stating that the patient was eligible for the appropriate triage protocol. The alert appeared as a “best practices” box with yellow highlights (appendix) to bring attention and recommendations to follow a specific triage protocol (order radiographs or order urine pregnancy). (See Figure 1.) The alert was passive, i.e., nurses did not have to click on the alert to see it and were not forced to click on the alert to remove it. The control group did not receive the alert. The alert was activated on January 1. Nurses were not provided any additional information regarding the protocols being studied. Data were collected for a six-month period before implementation (pre-) of the intervention (July to December), and another six-month period after implementation (post-) of the EHR alert (January to June). Data collected included the number of patients each nurse triaged, the number of triaged patients eligible for each protocol, and the number of patients for which the protocol was activated.

FIGURE 1.

Alerts as they appeared in the EHR. A (above): alert for patient with B (below): Alert for female patient with abdominal pain.

Procedure

Day-shift and night-shift nurses were randomized separately. At the time of consent, the nurses were informed that a study evaluating the EHR and triage protocols was being planned, but were not told the specific protocols or explicit details. All triage nurses were trained on all triage protocols. The nurses were consented during the month of November. The intervention (EHR alert) was activated on January 1. This delay allowed at least one month between consenting the last nurse and activating the intervention. This “washout” period was done in an attempt to minimize the Hawthorne effect (improvements in triage protocol utilization simply due to knowledge of an ongoing study). The two protocol conditions considered were: (a) patients with injured extremities after minor trauma; and (b) female patients with abdominal pain. Protocols at the study site allow ED triage nurses to order radiographs of injured extremities (elbow, radius/ulna, wrist, hand, knee, tibia/fibula, ankle, or foot) if the nurse deems it appropriate. The second protocol allowed the triage nurse to order a urinalysis for a female patient with a chief complaint of abdominal pain. These two protocols were chosen as uncontrolled data from this site suggesting both protocols were underused, but when utilized, patient total time in the ED was decreased.

Outcome

The primary outcome measure is the rate of triage protocol utilization defined as the percentage of eligible patients for which the protocol is activated for each condition. For each nurse and eligible condition, we calculated the rates before and after implementation of the EHR alert.

Data Analysis Plan

We used a generalized linear mixed model with a logit link to compare rates of protocol utilization, and the proportion of ED visits for which the protocol was followed between the two treatment groups before and after implementing the intervention. The effect of the intervention (received EHR alert vs. control), time period (pre- and post- intervention), and the interaction between these two effects was evaluated for the rates and percentage of ED visits for which the protocol was activated. A random effect was included to account for the dependence of observations from the same nurse before and after implementation of the EHR alert. Statistical significance of the intervention, time period, and the intervention-time period interaction were evaluated with z-statistics. We summarized changes in outcome measures between time periods with odds ratios (ORs) and 95% confidence intervals. All tests were two-sided and a p-value < .05 was considered statistically significant. Statistical analyses were conducted in R version 11.1 (R Development Core Team, 2012). Investigators were masked to randomization until data analysis was completed.

Results

Twenty-eight nurses (of 32 eligible triage nurses) participated in the study. In the six months prior to implementation of the EHR alert, the 28 triage nurses triaged 20,410 patients, and after implementation of the EHR alert the nurses triaged 19,157 patients. Protocol activation was low prior to implementation of the EHR alert, but increased in both triage nurse cohorts after EHR alert implementation (Table 1). Nurses activated the protocol for patients eligible for urinalysis at nearly twice the rate they ordered radiographs both before and after the intervention. Occurrences of implementing the triage protocols did not differ significantly between groups for either patients eligible for the urinalysis protocol (p = .89) or radiograph protocol (p = .67). The interaction between group and time period also was not a significant factor for either protocol (urinalysis: p = .19; radiograph: p = .82). However, for both groups of eligible patients, protocol activation increased significantly after implementation of the intervention (urinalysis: p < .001; radiograph: p = .001). Considering both groups together, the odds increased 2.03 times (95% CI [1.51, 2.75]) and 1.86 times (95% CI [1.28, 2.70]), respectively, after implementation of the EHR alert. After initiation of the EHR alert, the odds of a nurse ordering a urinalysis in intervention group (OR = 2.02; 95% CI [1.51, 2.71]) was similar to that in the control group (OR = 1.54, 95% CI [1.15, 2.04]). Further, the significant change in urinalysis protocol activation in the control group was likely driven by one nurse who had a large increase (about 30%) of ordering a urinalysis over the two periods and also triaged a large number of eligible patients (Figure 2). With exclusion of this nurse, the control group did not change significantly after implementation of the EHR alert (p = .48). In contrast, for patients eligible for radiographs, the change after implementation of the EHR alert was similar in both the intervention and control groups. The odds of a nurse in the control group activating the radiograph protocol increased by 1.99 times (95% CI [1.34, 2.94]) after EHR alert implementation and 1.87 times (95% CI [1.28, 2.73]) in the intervention group.

TABLE 1.

Protocol Eligibility and Activation

Protocol	Group	Before Intervention				After Intervention
		Triaged	Eligible	Activated	(%)a	Triaged	Eligible	Activated	(%)a
Abdominal painb	Intervention	9,544	624	101	(16.2)	9,063	530	146	(27.5)
	Control	10,866	711	116	(16.3)	10,094	679	174	(25.6)
Extremity injuryc	Intervention	9,544	774	58	(7.5)	9,063	614	78	(12.7)
	Control	10,866	684	45	(6.6)	10,094	609	79	(13.0)

Note. Cell entries are sample sizes unless otherwise noted.

^aActivated/eligible.

^bFemales only.

^cMinor injury only.

FIGURE 2.

Change in protocol activation for patients eligible for (A) urinalysis and (B) radiographs. Note that the scale of the pre-post differences is not the same for the urinanalysis and radiograph situations.

For patients eligible for the urinalysis protocol, the intervention resulted in a small but consistent increase in protocol activation with 12/14 nurses increasing protocol use (Figure 2). Among nurses in the control group, however, the change in protocol activation after EHR alert implementation was inconsistent with only 7/14 nurses increasing protocol activation for patients with abdominal pain. However, for the radiograph protocol, both groups show a similar response before and after implementation of the EHR alert (Figure 2).

Discussion

We performed a randomized controlled trial to determine if an EHR alert increases ED triage nurse protocol utilization. We identified a substantial increase in protocol activation after implementation of the EHR alert. This increase, however, was identified in both the control and intervention groups. The increase in protocol utilization in both groups was likely due to two reasons:

1. All nurses were consented prior to implementing the intervention and were aware that protocol utilization was going to be tracked. In the future, behavioral type studies measuring the impact of EHR alerts should be conducted without provider knowledge of the study to minimize introduction of bias.

2. Potential contamination between randomized cohorts may have occurred. A nurse in the intervention cohort may simply discuss this new alert with the observation cohort, increasing the activation of the alert in the observation cohort. A multicenter study with randomization at the hospital level would resolve this issue.

The institution’s IRB required written informed consent from all participating nurses. Thus, the participants were aware of the study and some of the benefit noted after the intervention is likely secondary to the Hawthorne effect. Future research in this area should be conducted without knowledge of the participants to minimize this bias. Blinding the participants to knowledge of the study would require either waiver of informed consent of the participants or studying natural implantation of alerts (i.e. a before and after study design).

The current EHR alert follows many of the guidelines for optimal alert design (Horsky et al., 2013). This study, however, evaluated a passive alert where the alert did not require nurse action to continue with their triage of the patient. The nurse could freely bypass the alert without intervention. Future research on ED-based EHR alerts should focus on more active alerts, requiring provider response. These studies, however, must be meticulous in order to identify unintended consequences—especially delays in patient care.

Limitations

The number of study participants was small, but represented 88% of the eligible nurse population. Results were likely affected by practice changes in a small number of nurses. A multicenter study with more participants would decrease this risk. Nurses in the intervention group may have informed the nurses in the control group of the alert. We are, however, not aware of this happening. The study evaluated a passive alert. A more active alert (alert requiring nurse action) might result in improved compliance.

Conclusion

Use of a passive EHR alert to promote utilization of existing ED triage protocols by nurses demonstrated little benefit. Prior to widespread implementation of EHR alerts for patient care, rigorous studies are required to determine the best method of alert and impact of such interventions.