Preparing for the Surge: A Half-Day Emergency Preparedness Training Course for the “Second Front”

Abstract

Purpose

Clinical disaster medicine requires providers working collaboratively to care for multiple patients, yet many clinicians lack competency-based training. A 5-hour emergency preparedness training (EPT) curriculum was created using didactics, small group discussion, and scenario-based learning. The goal was to evaluate the effect of a short course on improving clinical-provider knowledge, confidence and skill.

Methods

Participants were enrolled in a medical university between 2011 and 2014. The course consisted of didactic lectures, small group exercises, and live mass-casualty training scenarios. Core competencies and performance objectives were developed by a task force and assessed via facilitator observation, pre- and posttesting, and a course evaluation.

Results

A total of 708 participants were trained, including 49.9% physicians, 31.9% medical students, 7.2% nurses, and 11% allied health personnel. The average percentage of correct answers increased from 39% to 60% (P <0.01). Following didactics, trainees met 73% and 96% of small group performance objectives. Trainees also met 68.5% and 61.1% of the mass-casualty performance objectives. Average trainee self-assessment of disaster-preparedness skill improved from 36 to 73 points out of 100.

Conclusion

A brief, intensive EPT course can improve the disaster knowledge and comfort level of a diverse group of clinical providers as well as foster disaster-performance skills.

Delivering quality medical care during a disaster requires adequately prepared teams of health-care providers working together to save patient lives. Few providers receive competency-based training to respond to low-probability, high-risk events such as natural disasters, terrorist attacks, or other mass-casualty scenarios. The 2014 American College of Emergency Physicians report on the state of emergency care in the United States gave the country a “C-” in disaster preparedness.¹ Overall, 14 states received an “F” grade, partly because of inadequate education and funding for emergency preparedness training (EPT).¹

Despite recommendations from the Association of American Medical Colleges and the Institute of Medicine (IOM), US medical schools have been slow in developing stand-alone EPT curricula.^2,3 Although Joint Commission regulations require all “emergency services” facilities to perform yearly disaster drills, hospitals have been reluctant to develop comprehensive EPT programs on their own.⁴ Consensus reviews suggest that health-care worker training programs in the United States lack clarity, objectivity, competency-driven goals, scientific rigor, prospective validation, and consistency across medical specialties.^5–8

In a mass-casualty event, there are 2 fronts of personnel. Forward-deployed first responders include emergency medical technicians, police, firefighters, paramedics, and others at the scene of an emergency. The “Second Front,” or first receivers, includes clinicians, hospital workers, mental health providers, public safety officials, community volunteers, trainees, and administrative or other lay personnel. Experience has shown that in the immediate aftermath of a mass-casualty event, more than half of the victims will often bypass the first responders and report directly to hospitals. For example, during the 1995 Tokyo sarin-gas attack, more than 80% of patients self-transported to hospitals where staff and patients were exposed to sarin gas because of deficits in decontamination, lack of personal protective equipment (PPE) and awareness, and inadequate training.⁹

The hospital-coordinated and bystander responses to 2 recent large-scale mass-casualty incidents illustrate the importance of coordinated health-provider responses to save lives. In the recent Paris mass-casualty incident in November, 2015, 130 patients died before reaching the hospital. Of the 302 victims who presented to 16 different hospitals, only 4 (1%) died at the hospital. In addition to a swift and coordinated mobilization of resources, credit for this successful response has been given to the presence of health-care professionals able to perform triage as well as prehospital damage control.¹⁰ In the Boston Marathon bombings of April, 2013, 281 individuals were injured and 3 patients died before reaching a hospital. Of the 127 victims who presented to a Level 1 Trauma center, no patients died thanks, in part, to prehospital interventions and organized risk-stratification of victims.¹¹ In both Paris and Boston, Second Front providers were able to rapidly identify and risk-stratify patients during a moderate patient surge.

Educational research has demonstrated that collaborative, multimodal and problem-based learning environments increase learner satisfaction and may lead to better performance and long-term trainee retention.^12–15 With this in mind, and in response to a great need for EPT among health-care professionals, a competency-based curriculum was developed, which included a mixture of didactics, small group discussion, problem-based learning, and collaborative live scenarios. The goal was to evaluate the effect of a short, performance-based EPT course to improve the ability of a diverse group of disaster-care providers to respond effectively to mass-casualty scenarios.

METHODS

Curriculum Development

The Center for Health Professional Training and Emergency Response (CHPTER) was founded in 2009 with the mission to develop state-of-the-art, performance-based EPT curricula utilizing community resources from clinical, public health, private, governmental, and non-governmental coalitions (www.musc.edu/chpter). In 2010, a CHPTER curriculum task force established 9 learning objectives, 18 competencies, and 24 performance objectives for a 5-hour EPT course. Course elements were designed for a diverse group of learners with various degrees of expertize as well as different learning styles.

Detailed descriptions of the course goals, development of the curriculum task force and curriculum content, evaluation measures, and preliminary data have been published previously. These publications include preliminary cognitive and self-assessment data as well as limited small group and mass-casualty-incident (MCI) skill assessment data.^16–18

Participants

Trainees were enrolled on a first come, first serve basis between November 2011 and August 2014. Advertising for the course included the use of university e-mail, social media, paper fliers, and word-of-mouth. The target audience included any provider who might render patient care during a disaster, including physicians, nurses, medical students, and other interested medical-university learners and employees. The course was voluntary and offered at no charge. The study was approved by a university-based Institutional Review Board.

Course

Training was initiated with brief multimedia presentations, followed by small group exercises to provide trainees with the knowledge and skills they would need to mitigate multipatient scenarios at the conclusion of the course. Didactic presentations included background on emergency preparedness, PPE, teamwork, communication, and disaster triage. During the first small group exercise (SGE1), teams of 4–6 participants worked on large jigsaw puzzles to experience and practice disaster-scene teamwork and communication. The performance objectives for SGE1 included having a prehuddle, establishing a team leader, assigning roles, understanding the roles, discussing the threat, completing the puzzle, and describing the scene. The second small group exercise (SGE2) challenged teams to triage 60 patients simulated by small toys with printed description and vital signs. The performance objectives for SGE2 included establishing a chain of command, verbal communication, relaying key information and communication accuracy, conciseness, and clarity. Performance objectives for the small group exercises were either “met” or “not met,” according to facilitator assessment.

Following small group exercises, teams were assessed on their ability to mitigate 2 different multipatient MCI training scenarios. In the first scenario (“Storm Surge”), teams triaged live patient actors in a dark, chaotic environment. During “Storm Surge”, teams were assessed by trained facilitators using 6 performance objectives based on a 10-point Likert scale in which 1 = Did Not Meet Expectations and 10 = Exceeded Expectations (Table 2).

TABLE 2.

“Storm Surge” and “Influenza-Like Illness” Mass-Casualty Training Scenarios, Facilitator Assessment of Team Performance by Performance Objectives Using 10-Point Likert Scale (1 = Did Not Meet Expectations, 10 = Exceeded Expectations)

	N	Mean	SD	25th Percentile	Median	75th Percentile
Storm surge (Scenario 1)
Recognition of disaster	12	9	1.13	8	9.5	10
External communication	12	6.8	3.04	6	8	8.5
Internal communication	12	7.3	1.83	5.5	8	9
Utilization of resources	12	7.8	1.27	7	8	9
Personal and staff safety	12	7.8	2.37	7.5	8	9
Facility safety and security	12	7	2.95	7	8	9
Influenza-like illness (Scenario 2)
Recognition of disaster	24	7.7	1.92	7	8	9
External communication	24	7.3	2.38	5	8	9
Internal communication	24	7	1.98	5	7	8
Utilization of resources	24	6.9	1.36	6	7	8
Personal and staff safety	24	6.1	2.5	4	5	8
Facility safety and security	24	4.0	3.32	1	3	8

In the second scenario (“Influenza-Like Illness”), teams were confronted with patient surge and a possible terrorist threat from outside the emergency department (ED).

During “Influenza-Like Illness,” teams were assessed by facilitators using 12 performance objectives based on a 10-point Likert scale in which 1 = Did Not Meet Expectations and 10 = Exceeded Expectations (Tables 2 and 3). For example, if providers secured the ED from the outside threat, a performance objective was met. If they failed to secure the ED, they would face a chaotic patient environment and a potentially uncontrolled bioterrorism threat.

TABLE 3.

“Influenza-Like Illness” Mass-Casualty Training Scenario; Number of Teams that met Each Performance Objective as Assessed by Facilitator (Binary Did or Did Not Meet), Out of a Total of 24 Teams

	Number of Teams that Met Objective	Percentage of Teams that Met Objective (of 24 teams)
Established roles	23	95.83
Personal and team safety	22	91.67
Used proper PPE	24	100.00
Established communication link	21	87.50
Communicated needs	15	62.50
Followed team roles	23	95.83
Established chain of command	16	66.67
Reassessed patients and scene	13	54.17
Secured doors of ED	6	25.00
Did not leave ED	7	29.17
Stopped influx of contaminated patients	4	16.67
Prevented egress of patients	2	8.33

Abbreviations: PPE, personal protective equipment; ED, emergency department.

Trained facilitators helped organize both small group and MCI scenarios. When available in sufficient numbers, facilitators – all of whom were volunteers – were asked to submit written assessments of team performance. Although facilitator evaluations were helpful in assessing trainee performance, facilitators were commonly unavailable. Performance data are limited to training with facilitators working in sufficient numbers to evaluate trainees.

Evaluation Measures

Trainees answered a 24-question online test both before and immediately following the EPT course. Of the 24 questions, 18 were identical and form the basis of the pre- and post-comparison; whereas the pretest was required to start the training day, the posttesting was voluntary and not all students completed posttest assessments. In addition, whereas all trainees completed the didactic component of the course, only about half completed the small group and MCI portions of the course, largely due to time constraints of the trainees.

To increase enrollment, trainees were allowed to simply show up on training day. As a result, estimates of which students completed the full course are simply estimates based on the date of online pretest completion.

An early version of the course utilized high-fidelity human simulators in addition to the standard live-actor patients. In an effort to make the course easily reproducible in a variety of training contexts, high-fidelity mannequins were entirely replaced by patient actors after 2012.

Statistical Analyses

Descriptive statistics including means, standard deviations, frequencies, and percentages are reported. Pre- and posttesting were carried out using unpaired t-tests. When the course was first offered, data collected from trainees were not linked between pre- and posttest responses. As the course was improved for content, the data-collection procedures were also improved to allow for paired responses from the trainees. As the entire data set could not be paired, analyses are performed on unpaired records.

RESULTS

A total of 708 individuals enrolled and participated between November 2011 and August 2014. The participants included 353 (49.9%) physicians, 226 (31.9%) medical students, 51 (7.2%) nurses, and <1% each of emergency managers, mental health providers, EMS personnel, law-enforcement officials, and others.

Of the 708 participants who completed the 18-question pretest, 509 completed an identical posttest. The average number of correct answers increased from 39% to 60% (P <0.01). Average trainee self-assessment of overall disaster-preparedness skill improved from 36 to 73 points out of 100 (P <0.01) (“Overall comfort level performing to protect and save human life during a disaster”). Average training self-assessment of overall disaster-preparedness knowledge improved from 33 to 74 points out of 100 (P <0.01) (“Overall knowledge of disaster preparedness and disaster medicine”) (Table 1).

TABLE 1.

Pre- and Post-Cognitive Assessment and Trainee Self-Assessment

	Pre			Post
	N	Mean	SD	N	Mean	SD	P value
Cognitive assessment
Knowledge—# Correct (of 24)	708	9.24	5.24	509	14.37	7.25	<0.01
Knowledge—% correct (of 24)	708	39%	22%	509	60%	30%	<0.01
Knowledge—# correct (of 18 identical questions)	708	6.87	4.04	509	10.87	5.50	<0.01
Knowledge—% correct (of 18 identical questions)	708	38%	22%	509	60%	31%	<0.01
Self-Assessment—Scale 0 (not confident at all) to 100 (very confident)
Define and recognize disaster	525	41.57	25.86	386	76.34	15.63	<0.01
Ability to serve on disaster response team	528	31.17	24.43	382	72.83	17.93	<0.01
Perform scene communications	526	39.25	24.50	379	73.28	16.38	<0.01
Triage patients during disaster	528	35.79	24.91	382	79.66	15.72	<0.01
Perform role in incident command system	524	31.32	25.29	380	71.79	18.32	<0.01
Overall knowledge of emergency preparedness	529	33.34	23.60	382	73.72	15.59	<0.01
Overall skill protecting live during disaster	528	35.70	25.64	381	72.60	18.72	<0.01

A total of 251 participants completed a course assessment. Most felt that the course was relevant to patient-care providers (96%), that the size of the training class was “adequate size to achieve course objectives” (92%), and that they had an “adequate length of time to achieve course objectives” (86%). Overall, 92% highly recommended the training.

Facilitators evaluated 38 teams who completed small group exercise 1 (SGE1) and 37 teams who completed small group exercise 2 (SGE2). During the jigsaw-puzzle communication exercise (SGE1), teams met, on average, 5.08 ± 2.01 of the 7 performance objectives (average of 6 participants in each team). During the triage tabletop exercise (SGE2), teams met, on average, 5.76 ± 0.55 of the 6 performance objectives (average of 6 participants in each team, n =37 total teams). In all, 89% (33/37) of SGE2 teams met all of the objectives and 100% (37/37) of SGE2 teams were able to establish verbal communication with incident command and convey an accurate description of the disaster scene.

Facilitators evaluated 12 teams during MCI scenario “Storm Surge” and 24 teams during “Influenza-Like Illness” (average of 6 participants per team). During “Storm Surge,” 6 performance objectives were assessed using a 10-point Likert scale (1 =Did Not Meet Expectations and 10 =Exceeded Expectations). Overall, teams scored >6 out of 10 on all performance objectives, with the highest average scores for teams’ recognition of a disaster (9.0 ± 1.1), utilization of resources (7.8 ± 1.3), and personal and staff safety (7.8 ± 2.4) (Table 2).

During the MCI scenario “Influenza-Like Illness,” 12 performance objectives were assessed. Teams successfully used proper PPE (24/24), established team roles at the beginning of the scenario and followed team roles throughout (23/24), adhered to personal and team safety (22/24), established a communication link (21/24), established a chain of command (16/24), communicated needs with incident command (15/24), and reassessed patients and the scene (13/24). Teams were least successful at securing the facility and preventing an influx of contaminated patients (Table 3).

DISCUSSION

This study demonstrates that a half-day, easily reproducible EPT course can improve knowledge and foster the performance skills necessary for a diverse group of trainees to respond to a clinical disaster training scenario. Trainees increased their knowledge of emergency preparedness, improved their comfort and confidence with handling clinical disasters, and highly recommended the course. Trainees completing the full course also successfully completed a series of challenging small group and multipatient scenarios.

The course was completed in a medical-university setting without compromising existing course schedules, demonstrating that disaster training can occur in addition to existing university course requirements. An unexpected finding of the curriculum-development process was that the majority of training was completed at little or no cost. Unique, low-cost tools such as puzzles and toys significantly reduced the financial burden of the course. In addition, using trainees as patients during 2 (simultaneous) scenarios eliminated the need for paid actors or costly simulators. Further iterations of the curriculum could benefit from cost per provider, cost per training hour, and in-kind cost assessments associated with utilizing volunteers.

The low average score on the knowledge-based pre-assessment underscores the need for introductory-level EPT among health-professional and medical-student populations. This study contributes to a growing understanding that poorly trained staff members can place patients and providers at significant risk during even a small-scale disaster.

As seen earlier in previous publications, the percentage of correct answers to knowledge questions increased from pre- to posttest (P <0.01) following the course.¹⁸ In terms of the curriculum, trainees were more likely to succeed in mechanical objectives such as triage, donning PPE, and providing emergency communication. Other skills, such as teamwork, general communication skills, ability to comfort patients in distress, and ability to problem solve under pressure were not as easily mastered during a half-day course.

It could be argued that trainee performance during MCI simulation was poor when compared with typical 100% scales. For example, teams on average met 68.5% of objectives when asked to manage a disaster scene with multiple patients in a dark, chaotic environment. However, the MCI scenarios were intentionally designed to be loud and difficult to manage. In the “Influenza-Like Illness” scenario, for example, the vast majority of groups failed to prevent the threat from entering the ED. Though facility safety is addressed in the didactic component of the course, the chaos and timing of this scenario proves to be a significant challenge to trainee groups. As the threat breaches their facility, participants experience a simulated version of what could be a very costly mistake in the event of a real disaster or bioterrorism event. The room becomes noisy and crowded, and a potentially lethal substance is introduced. Experiencing the consequences of the mistake, along with a debriefing following the simulation, allows for a more profound learning experience by allowing for failure. Further long-term assessment of this curriculum will hopefully show that students learn more, perform better, and retain knowledge and skills longer when they confront failure via a challenging testing environment.

Limitations

This study has several additional limitations. First, the study included a pre- and posttest for knowledge and personal assessment of skills, but performance was not measured before the course. Because of this, any success trainees may have demonstrated could have been related to previous experience. It would be ideal to understand how profession, subspecialty, and degree of previous training impact performance during the course, although given the low self-assessment and pretest cognitive scores, the impact of previous disaster experience in this study sample is arguably low.

The study was also limited to 1 site and mostly to medical students and physician participants, therefore generalization of the curriculum to other trainees and lay people may be limited. Because of the challenges of enrolling voluntary trainees and facilitators (during their free time), not all students completed the full course and post-assessments. Therefore, conclusions about which components of the course contribute to student success are limited.

Although reliance on volunteer facilitators reduced the burden of hiring trained researchers, it also greatly limited our sample size to measure performance. The small sample for team assessments limits the generalizability of the curriculum and its expected impact on clinical disaster performance. Finally, further iterations of this study could provide a more robust assessment of cost including cost per provider, cost per training hour and in-kind costs associated with utilizing volunteers.

CONCLUSIONS

This course does not represent comprehensive training for disaster response; however, as shown in this study, a relatively short, intensive EPT course can improve knowledge and comfort level. It can also foster the performance skills necessary for a diverse group of trainees to respond to disaster-medicine training scenarios. Components of the course may be adapted for use in “just-in-time” training for medical students, nurses, hospital teams, or other personnel to foster knowledge, comfort level, and skill.

The AAMC, IOM, and other organizations have been clear in their call for action to improve emergency preparedness at all levels. Although further research is needed, this course is an innovative, effective, potentially affordable, and easily reproducible contribution to the clinical EPT ecosystem.