Evaluation of a Comprehensive Emergency Department Violence Prevention Program

Healthcare clinicians are at high risk for workplace violence (WPV) from patients and visitors;^1–3 with 25.5% reporting at least one recent incident of victimization.⁴ Researchers found that WPV has negative effects on providers’ stress, ability to provide safe and competent care, job satisfaction, and turnover.^2,5–6 In response, several states enacted legislation making assaults against healthcare workers a felony or requiring employers to implement WPV prevention programs.^7–8 Professional organizations voiced their concerns about the safety of healthcare workers. The Emergency Nurses Association’s 2010 position statement, Violence in the Emergency Setting, states, “Health care organizations have a responsibility to provide a safe and secure environment for their employees and the public” (p. 1).⁹

Few WPV prevention programs for healthcare settings are noted in the literature and those programs are limited in scope and evaluation.^10–12 In response to this program gap, the authors partnered with three emergency departments to plan, implement, and evaluate a multifaceted, comprehensive WPV prevention program. Emergency departments are often cited as the healthcare setting with the highest incidence of WPV.^13–15 The overall goal of our program was to develop a comprehensive approach for creating a safer work environment and make recommendations for future programs. It was hoped that the new information gleaned from this work could be used by nursing leaders and employees in a variety of healthcare settings. The purpose of this article is to describe the process and methods used to implement and evaluate the WPV prevention program.

Methods

Setting and Participants

The program was initiated at three U.S. hospital EDs from different hospital organizations: an urban Level 1 trauma center, an urban hospital, and a suburban hospital. The Level 1 trauma center has a separate psychiatric ED adjacent to the adult-only ED. The urban and suburban EDs serve a general ED population: adult, pediatric, and psychiatric patients. Although the project was aimed at the entire ED, clinicians providing direct patient care (i.e., nurses, physicians, and unlicensed assistive personnel) were specifically targeted for risk reduction and evaluation. The overall project was approved by the Institutional Review Boards of the university and respective hospitals. While patients participated during the development of the WPV program, data from patients were not collected for the project evaluation.

Action Research

The authors used the principles of action research as guidance for program planning, implementation, and evaluation.¹⁶ The goal of action research is for academic researchers to partner with clinicians for opportunities to reflect on clinicians’ practice and implement informed actions to reduce WPV exposures. Methods are rooted in the concerns of clinicians in real-world settings. Characteristics include action research being cyclical, having separate but mutually dependent steps, being participative, generating data, and being a reflective process.

The goal of the project was to develop partnerships among academic researchers and ED clinicians and leaders to implement a WPV prevention program tailored to the unique ED settings while increasing the likelihood of program implementation and sustainability. The partnership was initiated by the academic researchers. Initial entrée was facilitated by the networking of one of the researchers and all of the ED leaders being members of the Greater Cincinnati Emergency Nurses Association. The professional networking that occurred prior to the project initiation allowed for an open discussion between the academic researchers and ED leaders to tackle a problem of mutual interest and importance.

Program Objective

Program planning included both formative and summative evaluation to identify not only whether the program was successful, but provide information regarding factors that facilitated or hindered the project’s success. This information is critical for program revisions and future planning. The program objective was to implement a sustainable comprehensive ED-based WPV prevention program in collaboration with stakeholders.

Program Assessment

Assessment and planning for the project took place over a 21-month period and included several strategies to assess the EDs’ risk factors. A 9-month WPV incidence was trended to determine the extent of WPV as a local problem. The average clinician experienced 0.461 violent events per month which extrapolated to 5.5 physical threats and assaults per person per year; there was no significant difference in rates amongst the partner sites.¹⁷ A review of the research literature was conducted to identify known risks for and causes of ED WPV.^2–3,10 Focus groups were held with ED leaders (physicians, nurse managers, nurse educators), employees, and patients to identify WPV prevention, management, and post-incident strategies already in place as well as additional interventions that may be helpful.¹⁸ Walkthroughs were done to identify environmental risks. Meetings were held with ED-based safety committees to identify further risks and strategies for reducing risks. WPV-relevant policies were reviewed. Multiple meetings were then held with medicine, nursing, security/police, registration, management, social work, and psychology to plan the intervention. Using the assessment data, an intervention was proposed by the academic researchers and discussed with ED leaders and staff.

Program Intervention

Data supported the need for a multi-component intervention: WPV policies and procedures, WPV education, and environmental changes. A best practice policy and procedure was written and then tailored to each emergency department in collaboration with the ED leaders. Revisions of the plan and documents continued until approved by ED leaders, staff, and senior hospital administrators. Written policies and procedures for each partner site included each of the following sections: strategies for risk assessment, maintaining a safe environment, communication of risk, response to violent events, record-keeping, surveillance, and post-incident care. The education component included uniform on-line training that provided didactic content for WPV prevention, management, and post-incident recovery; and an interdisciplinary class for participants to apply new policies, procedures, and on-line training.¹⁹ Environmental changes were site-specific, based on assessments and availability of site funds. All three sites adopted a process to alert non-clinical staff (e.g, environmental services) that a violent person was in the room. In addition, the suburban emergency department installed panic buttons, locked doors, and closed-circuit cameras; subcomponents already in place at the other partner sites.

Program Implementation

The program was specifically aimed at all ED clinicians including physicians, nurses, and unlicensed assistive personnel and implemented over a simultaneous three month period. The new policies and procedures were diffused using the intranet, flyers, posters, staff meetings, name badge cards, and an interdisciplinary class session.

Program Evaluation

The program was evaluated through a formative evaluation of the program implementation and a summative evaluation of program and components. Formative evaluation took place over a nine-month period after implementation. Monthly, the project director used a checklist to assess the fidelity of the intervention during her ED rounds. She documented notes related to her monthly assessments. The data for the nine months were summarized in narrative format (see Table 1).

Table 1.

Summary of Program Fidelity during the Nine-Month Post-Implementation Period.

Program Subcomponent	Program Director’s Comments from the Monthly Fidelity Assessments
1. Rounding in the lobby	Two hospitals did rounding consistently, one ED did not and identified that it was staffing dependent.
2. Screening all patients for early signs of potential escalation	A screening checklist was developed and initially planned to be used on all ED patients as a means to communicate potential risk to other employees. The checklist evolved to being used as needed to communicate early signs of potential escalation of violence. The majority of clinicians found it useful as a reminder to be alert to the early signs of escalation.
3. Concealed weapon assessment question during triage	One ED was consistent; clinicians routinely asked about the presence of weapons as part of the triage assessment. Several weapons had been turned over to security for proper storage at all three EDs. At one ED where there was inconsistency, some clinicians commented that they are uncomfortable asking about concealed weapons. At the other ED which was also inconsistent, some clinicians commented that they did not feel safe asking the question when they were alone with a patient in triage.
4. Flagging patients with a history of violence or exhibiting signs of potential violence	No EDs did this in any meaningful way. Reasons given was the size of the ED, one saying it was too big to communicate the levels, and one saying it was too small and that levels are communicated by word of mouth. The larger ED thought a tracker board visual to display the levels would be helpful.
5. Levels of awareness alerting employees of violent risk level in ED	Two sites initiated a system that worked consistently. Two of the hospitals found flagging to be very important. One ED flagged inconsistently.
6. Violent event response table	A table was developed with clear instructions as to what employees are to do in certain violent situations. The ED clinicians thought the table was helpful and used it in theory. Several ED clinicians did not have the response card on their identification badge as planned.
7. Care After Violent Event	Clinicians were consistently knowledgeable as to the procedures to take if they were a victim of workplace violence. This was particularly true if an injury occurred.
8. Reporting violence and debriefing	Clinicians were consistently knowledgeable as to the procedures for reporting a violent event. Clinicians reported that any debriefing that occurred was done informally with their manager.
9. Post incident reviews	One ED consistently posted reviews of violent incidents. One ED was inconsistent with their reviews while the last ED was not doing them.
10. Quarterly surveillance reports	One of the EDs was doing the reports and sharing with the clinicians. Two of the EDs were not consistently doing these.

Summative evaluation began with a sample of 80 employees completing an on-line program evaluation. Employees rated the program benefit using a 0 (no benefit) to 10 (extremely beneficial) scale and the ease of following the program using a 0 (extremely difficult) to 10 (extremely easy) scale. Employees identified their beliefs regarding the level of commitment to the program by themselves, colleagues, and administration on a 0 (no commitment) to 10 (total commitment) scale. Lastly, participants rated the importance of the WPV prevention subcomponents from 0 (not important) to 10 (extremely important). Means were computed to report the results. Participants were given an opportunity to add comments. Additionally, all employees who completed the educational component, evaluated the education for content, format, instructor effectiveness, and time required to complete the training.

Finally, a program evaluation meeting was held with nurse managers and educators from the three EDs to discuss the program implementation after the post-implementation period was complete. The discussion questions are outlined in Table 2. The discussion was transcribed and summarized.

Table 2.

Summary of program evaluation meeting with emergency department managers and educators (n=6).

1. Overall, how beneficial do you believe the violence prevention program was in reducing violence?  a. All managers and nurse educators verbalized their beliefs that the program was beneficial in reducing violence in their departments.  b. The classroom training was rated particularly high with all managers and stated overall compliance was good.  c. Physician participation was very low in two emergency departments. One manager identified that the environmental changes were the most important and beneficial component.

2. Discuss your ability to adapt and follow the new workplace violence prevention policies.  a. One manager stated that it was initially a challenge with another stating that the program was slow getting adopted.  b. There were several comments about the need for increased administrative and physician support for the changes.  c. One manager commented that the new policies and procedures led to increased dialogue about workplace violence.

3. To what degree did your emergency department comply with the new workplace violence prevention policies and procedures?  a. The managers uniformly stated overall compliance was good with compliance really depending on the particular intervention subcomponent.

4. To what degree did your emergency department comply with each of the specific intervention components and subcomponents?  a. Environmental changes were perceived as effective by all managers and nurse educators.  b. On-line training was generally characterized as being too long with technology issues that were challenging for some clinicians.  c. Classroom training was well liked. Only one emergency department had physician participation and support.  d. Lobby rounding to assess for risks varied depending on the department, component, and individual employee and staffing. For example, while lobby rounding was viewed as important, managers stated that compliance depended on patient capacity and staffing.  e. Screening patients for early signs of potential escalation was being used and found to be helpful, but not as intended with too many patients being flagged.  f. Asking about a concealed weapon during triage varied among the ED clinicians and their comfort level in asking. Weapons were turned over by patients in all three emergency departments.  g. Flagging patients with a history of violence or exhibiting signs of potential violence was consistently used by all emergency departments.  h. Levels of awareness to alert employees of violence risk level in the ED was the most difficult intervention for clinicians to understand and generally was not being used.  i. The violent event response procedure and table was determined to be helpful to aide clinicians identify specific actions. The table added as a name badge card was not found to be helpful or used by the clinicians.  j. Care after a violent event was done consistently if managers became aware of the event. Some reported discussing events at meetings.  k. Reporting a violent event remains a problem with clinicians providing reasons for why they do not report violent events.  l. Post-incident review using a root cause analysis process for more severe violent events was consistently performed by managers.

5. How would you rate the partnership between academia and hospital emergency departments?  a. Evaluated as positive uniformly by all managers though sometimes challenging when priorities at the emergency departments and hospitals conflicted with the project.

Results

Formative Evaluation of the Program Implementation

The academic researchers and ED leaders were able to diffuse the program at the EDs over the three-month implementation period. Degree of success varied among the sites and for the individual program subcomponents. The smaller, suburban ED had the best results for institutionalizing and sustaining the intervention subcomponents over the nine-month post-implementation period. The larger, urban Level 1 trauma center had the least success for institutionalization and sustainment. See Table 1 for a summary of the fidelity assessments during the nine month post-implementation period.

Summative Evaluation of Program and Components

Of the 80 employees asked to complete the program evaluation, 66% (n=53) did so. Employees rated the overall program as moderately beneficial (n=53; mean[X]=5.0). Nurses evaluated the benefit of the program the highest (n= 35; X=5.3) whereas physicians evaluated the program’s benefit the lowest (n=9; X=3.4). The suburban ED employees evaluated the program’s benefit higher (n=12; X=6.6) than the urban ED employees (n=12; X=5.8) and the Level 1 trauma center employees (n=29; X=4.1).

Employees rated the ease of program implementation as moderately easy (n=53, X=5.9). Nurses rated the ease of program implementation the highest (n=35; X=6.0), with physicians next (n=9; X=5.7), and then unlicensed assistive personnel (n=5; X=4.6). Employees at the suburban ED rated ease of program implementation highest with a mean of 6.2 (n= 12), followed by the Level 1 trauma center (n=29; X =5.9) and the urban hospital (n=12; X=5.7).

Employees rated the level of commitment for program adoption and fidelity as higher for themselves (X=8.2), than for other employees (X=7.0) and ED leadership (X=6.8). Unlicensed assistive personnel rated their personal commitment higher (X=8.4) than physicians (X=8.3) and nurses (X=8.1). Employees at the suburban ED rated leaders’ commitment higher (X=7.3) than employees at the urban ED (X=7.0) and Level 1 trauma center (X=6.3).

The importance of program components/subcomponents ranged from a high of 8.6 (very important) for surveillance and monitoring to a low of 6.9 (important) for policies and procedures. In addition to surveillance, components/subcomponents deemed as most important were environmental changes, education, and post incident care.

The employees (n=315) who completed the educational component provided both positive and constructive feedback. Employees reported that overall the on-line training was valuable, but too long and fraught with technical problems. The classroom training was preferred over the on-line training. Employees believed that the classroom training could help them translate the content to their clinical practice. The training did lead to significant increases in WPV knowledge.¹⁹

Overall the managers and nurse educators described the program as very positive and believed the program would be beneficial at reducing WPV (see Table 2). The two components/subcomponents identified as most important were the classroom education, and the environmental assessments and improvements. The two components/subcomponents identified as the least effective were the levels of awareness and conducting a WPV assessment screening at triage for all patients.

Discussion

Nurse managers and educators reported the action research partnership between academic researchers and clinicians was very positive. They shared that because they have many competing priorities, the partnership and assistance was extremely valuable. In particular, they valued the expertise needed to evaluate the program and recommend revisions. In retrospect, the group discussed they may have attempted too many interventions over a three month implementation period. However, these were the subcomponents remaining from an even longer wish list of interventions narrowed due to resource availability and potential for success.¹⁸ Competing priorities also contributed to implementation challenges (e.g., visits from The Joint Commission and Centers for Medicare & Medicaid Services). The action research process allowed ED leaders and employees to reflect systematically on their practice and make informed actions for improvements. The project remained rooted in ED clinician concerns and maintained the characteristics of action research that it be cyclical, require mutually dependent steps, be participative, generate data, and be a reflective process. The experience provides support for partnerships between academic researchers and clinicians as a process to tackle social problems such as WPV.

Overall evaluation of the WPV prevention program was mixed. It was not surprising that the physicians rated the benefit of the program the lowest. Only a few physicians participated in the planning meetings and only a few physicians (n=12) participated in program’s educational component. This was in lieu of the fact that physician leaders participated in project planning. Whereas nurse managers required nurses and unlicensed assistive personnel to complete the educational component, physician leaders in two EDs did not. Although WPV against ED physicians is a recognized problem,³ getting physicians to value and complete WPV training will continue to be challenging due to physicians’ competing time, reimbursement demands, and not being employees of the ED. The lack of physician participation was interesting given Peek-Asa et al.’s discussion that physicians were commonly excluded from training and needed to be included.¹¹ Strategies to increase participation need to use data reflecting a critical need for change, leverage physician and hospital leadership relationships to support change, solicit physician champions to engage physician peers, and communicate the importance of physician contributions.

Employees rated surveillance and monitoring as the most important strategy for reducing WPV against employees. Yet, the monthly fidelity assessments and the program evaluation meeting with the ED managers and educators indicated that this intervention subcomponent was one of the least successfully implemented in all three emergency departments. The only reason given by managers and educators for this was that employees do not report incidents, a consistent problem reported in the literature.^2,11 Involving employees in the surveillance and monitoring of WPV could help adoption. The ED safety committee is an optimal group to diffuse the program, ensure adherence to regular WPV surveillance and monitoring, and champion the need to report incidents. Unfortunately, recordkeeping was rated as the second lowest subcomponent in terms of importance in reducing WPV. This finding may be related to employees not recognizing any benefit of recordkeeping as a form of WPV prevention.

Surveillance is important in WPV programs;¹¹ however, surveillance cannot be effectively leveraged to offer meaningful recommendations for change without valid data to determine problem severity, evaluate hazard control methods, and identify educational needs. Gates et al. found healthcare employees do not report WPV for several reasons, including undervaluing the importance of reporting, not having time to complete reports, and believing that nothing changes in response to reporting.^2,18

Administration has a key role in addressing both the importance of reporting incidents, and ensuring that changes based on reporting are visible to employees. Employees rated the policies and procedures component as the lowest in terms of importance in preventing WPV. Narrative comments from physicians at the Level 1 trauma center where no physicians attended the educational component stated that they did not know anything about WPV-related policies and procedures. Although the policies and procedures were discussed and applied during the classroom-based education component, the physicians’ lack of knowledge reflects a critical gap in the program’s diffusion. It is also possible that employees who rated policies and procedures low were reacting to a potential lack of enforcement and compliance with hospital policies in general. Reasons found to be associated with low compliance include perceived higher level priorities, fragmented organizational structures, and inadequate communication.

One of the program’s major success stories was the environmental changes at the suburban ED. During the assessment phase, the project team identified that the ED was very accessible to the public and at an increased risk for WPV. This ED’s high adherence to the WPV program may be a result of the clinician’s seeing the hospital’s financial commitment to instilling environmental changes. Interestingly, during the planning meetings many employees and managers at the suburban and urban EDs expressed that adding metal detectors would be the best intervention for reducing WPV. In contrast, employees and managers at the Level 1 trauma center with the metal detector expressed that metal detectors were not necessarily deterrents for WPV.¹⁸ This may be related to the metal detector only being used during night shift hours. Whereas the environment plays a crucial role in preventing WPV, it is important employees and managers do not underestimate the importance of a comprehensive approach that includes efforts to reduce risks related to patients, visitors, employees, and the environment.

The classroom education was evaluated more favorably than the on-line education, even though ED leaders and employees stated during the planning phase that on-line training was preferred. The on-line content was necessary because it provided didactic information about WPV, phases of escalation, and strategies to reduce WPV as preparation for the classroom education.¹⁹ The interdisciplinary classroom experience was interactive and included problem-based learning exercises using an interdisciplinary team application process.¹⁹ The on-line program was revised based on the feedback from the participants to reduce the time commitment; in particular, repetition was reduced with a focus on critical content. ED employees have many time demands and the required number of training hours continues to increase. While it is critical that content be essential and presented in the most effective format, it is important that ED leaders not resort to on-line training based simply on flexibility, cost, and ease of use. Hybrid education as used in this project offers an alternative to choosing between on-line or classroom strategies. The EDs will continue to provide the education to new hires and are committed to using the program for their annual training competencies.

Support for the program by ED leadership varied, with the majority of support coming from ED nurse managers. Yet, employees perceived ED leadership as the group least committed to the prevention efforts. Security and police involvement was positive, with several officers and managers involved in the planning and implementation phases. A collegial relationship with security officers is essential for an effective WPV prevention program.²⁰ At one ED, the social workers and patient escort managers were very supportive and involved as they identified that their employees were also at risk. As described above, the physician group was the least engaged in the planning, implementation, and evaluation phases.

Implications

Hospital employees, especially those in the ED, will continue to be at risk for WPV from patients and visitors. To support quality improvements and reduce risks, the use of a comprehensive program developed collaboratively between academic researchers and clinicians should emphasize communication and monitoring (surveillance) linked to education and feedback. Single table discussions need to occur amongst ED leaders, physicians, clinicians, and safety committee members so that a commitment for WPV prevention, management, and recovery can occur. An inter-professional safety committee can be responsible for ensuring the uniform adoption and implementation of the program.

Whereas certain settings have unique environmental and training needs, a facility-wide approach addressing primary, secondary, and tertiary prevention strategies is highly recommended. Diffusion efforts should include strategies that engage all stakeholders, including the hard-to-reach physician group. One strategy to accomplish this may be adopting a WPV program at a regional level so that all entities in the regional hospital association are active participants. WPV occurs against all healthcare workers, and prevention of incidents will continue to be dependent on the involvement of all disciplines. It is critical that all employees know what to do in specific situations and how to communicate risk amongst each other so that all employees remain safe. It takes just one untrained and unprepared employee to place other employees and an organization at risk.