Competency in Chaos: Lifesaving Performance of Care Providers Utilizing a Competency-Based, Multi-Actor Emergency Preparedness Training Curriculum

Abstract

Objective

Providing comprehensive emergency preparedness training (EPT) to care providers is important to the future success of disaster operations in the US. Few EPT programs possess both competency-driven goals and metrics to measure performance during a multi-patient simulated disaster.

Methods

A 1-day (8-hour) EPT course for care providers was developed to enhance provider knowledge, skill, and comfort necessary to save lives during a simulated disaster. Nine learning objectives, 18 competencies, and 34 performance objectives were developed. During the 2-year demonstration of the curriculum, 24 fourth-year medical students and 17 Veterans Hospital Administration (VHA) providers were recruited and volunteered to take the course (two did not fully complete the research materials). An online pre-test, two post-tests, course assessment, didactic and small group content, and a 6-minute clinical casualty scenario were developed. During the scenario, trainees working in teams were confronted with three human simulators and 10 actor patients simultaneously. Unless appropriate performance objectives were met, the simulators “died” and the team was exposed to “anthrax.” After the scenario, team members participated in a facilitator-led debriefing using digital video and then repeated the scenario.

Results

Trainees (N = 39) included 24 (62%) medical students; seven (18%) physicians; seven (18%) nurses; and one (3%) emergency manager. Forty-seven percent of the VHA providers reported greater than 16 annual hours of disaster training, while 15 (63%) of the medical students reported no annual disaster training. The mean (SD) score for the pre-test was 12.3 (3.8), or 51% correct, and after the training, the mean (SD) score was 18.5 (2.2), or 77% (P <.01). The overall rating for the course was 96 out of 100. Trainee self-assessment of “Overall Skill” increased from 63.3 out of 100 to 83.4 out of 100 and “Overall Knowledge” increased from 49.3 out of 100 to 78.7 out of 100 (P <.01). Of the 34 performance objectives during the disaster scenario, 23 were completed by at least half of the teams during their first attempt. All teams except one (8 of 9) could resuscitate two simulators and all teams (9 of 9) helped prevent anthrax exposure during their second scenario attempt.

Conclusions

The 1-day EPT course for novice and experienced care providers recreated a multi-actor clinical disaster and enhanced provider knowledge, comfort level, and EPT skill. A larger-scale study, or multi-center trial, is needed to further study the impact of this curriculum and its potential to protect provider and patient lives.

Introduction

The lack of emergency preparedness training (EPT) for patient care providers—including clinicians, hospital workers, mental health providers, public safety and law enforcement officials, community volunteers, EMS, HazMat and Fire personnel—poses significant risks to both patients and providers. During the 1995 Tokyo sarin gas attacks, for example, most patients bypassed first responders and reported directly to hospitals, where staff was exposed to sarin due to inadequate personal protective equipment (PPE) and training.¹ Furthermore, during Hurricane Katrina, EPT deficits were cited as significant factors contributing to adverse patient outcomes.^2–7

Providing comprehensive EPT for medical trainees is important to the future success of emergency preparedness operations, yet few medical schools have defined and implemented EPT core competencies for health professionals.^8–24 Recent reviews suggest that health care worker EPT programs lack clarity, objectivity, competency-driven goals, scientific rigor, prospective validation, and consistency across medical specialties.^25–28

A prior study revealed how medical students can value and adopt core EPT elements via a novel addition to existing curriculum.^{19, 29} A significant limitation of the study was the relatively simplistic measurement of EPT performance. A new course from the Center for Health Professional Training and Emergency Response (CHPTER) was developed to create loud and chaotic, multi-actor clinical disaster scenes (combining >10 human actors with human simulators at one time) that would enhance provider knowledge, skills, and comfort necessary to save lives during a simulated disaster. This study presents the performance data of trainees participating in the CHPTER EPT course between 2011 and 2012, including trainee self-assessment, course assessment, precognitive and postcognitive performance, small-group scenario performance, and clinical mass-casualty scenario team performance.

Methods

Curriculum Development—Organization

In 2009, CHPTER was formed as South Carolina’s first collaborative EPT center for patient care providers (www.musc.edu/chpter). Over the course of several months, a CHPTER task force established learning objectives for a 1-day EPT course utilizing existing evaluative frameworks, coursework and data.^30–39 Using a modified Delphi process, dozens of competencies were consolidated into 18, and subsequently assigned to five competency domains (Table 1). The task force then developed performance objectives to match competency objectives, and these guided content development for the didactic, small group and simulation course components. The project was approved by the Medical University of South Carolina Institutional Review Board (IRB).

TABLE 1.

COURSE LEARNING OBJECTIVES, COMPETENCY DOMAINS, AND PERFORMANCE OBJECTIVE CATAGORIES

Learning Objectives Define a healthcare disaster and the components of emergency preparedness as it applies to patient providers.1 Understand ethical implications of a healthcare disaster and its impact on the community.3 Differentiate between National Incident Management System (NIMS), Incident Command System (ICS/HICS) and the operational disaster/emergency preparedness plan for a healthcare facility. 1 Identify functional roles (and appreciate your individual limits) of care providers during a disaster.1,3 Define and demonstrate ability to function within the chain of command during a patient care disaster scene.1,2 Define and respond to vulnerabilities and security risks facing providers, health care workers, and health care facilities during a disaster scenario.1,2 Summarize components of teamwork, communication, and triage that are essential to an effective response during a health care disaster and list specific actions to take and to avoid during a health care disaster.1 Demonstrate effective teamwork, communication and triage to protect care providers and save lives during a disaster scenario.2 Achieve a greater comfort level with knowledge and skill to provide effective patient care during a clinical disaster3

Competency Domains and Performance Objective Categories

Mobilization: Pre-stage Planning and Team Development Define and Recognize a Disaster. [A, F]4 Stop: Establish a Safety Plan. [D, F, G, H] Develop Clinical Disaster Team (i.e., establish leadership, roles and duties). [D, F, G, H] Select and don appropriate personal protective equipment (PPE) and supplies. [F, G, H]

Clinical Disaster Operations and Communications Establish operations command (i.e., assume team roles). [C, D, E, H] Establish communications with health care authority and activate Healthcare Incident Command System. (HICS). [C, D, E, H] Appropriately report scene information and needs. (i.e., maintain situational awareness). [C, D, E, H] Optimize teamwork and coordinate tasks. [C, D, E, H]

Protect and Preserve Human Life and Continuity of Health Care Facility During a Disaster Ensure personal safety. [C, D, E, F, G, H] Ensure safety of patients, families, and staff. [D, E, F, G, H] Accurately assess, reassess and care for patients. [D, E, F, G, H] Ensure continuity of patient care operations. [D, E, F, G, H] Preserve integrity and conservation of the physical plant. [D, E, F, G, H] Perform maneuvers to save simulated patients during a disaster scenario. [F, H]

Demobilization Ensure effective demobilization of health care resources. [A, C, F, G]

Clinical Disaster and Emergency Preparedness Awareness Understand role of provider to support patients and the community. [B, D] Understand ethical implications of patient care during a disaster. [B, H] Self-assess capabilities and limits as a provider during a disaster. [B, D]

¹Discrete Knowledge/Cognitive Objective;

²Performance/Skill Objective;

³Attitudinal/Affective Objective;

⁴References are to learning objectives, above

Small Group Exercise Development

Didactic Module 1 was developed as an introductory lecture to the training course. Didactic modules 2, 3 and 4 were developed to include a small group exercise designed to prepare students for the afternoon disaster scenario. Following a short slide-based lecture, trainees were randomly assigned to teams of four to six and were assessed by CHPTER instructors who served as small group facilitators.

The team building exercise (module 2) consisted of seven performance components and four fictional disaster scenarios printed on 34-piece puzzles. The unsolved puzzles were presented to teams in large envelopes, and teams were instructed to complete at least one disaster scenario. Instructors evaluated the teams based on their ability to complete the puzzle and verbalize appropriate responses to threats presented. The communications exercise (module 3) consisted of three scenarios (bus crash, factory explosion, and chemical leak) presented to team members on preprinted handouts, and teams were instructed to complete at least one scenario. Scenarios were discussed in small groups, guided by a facilitator who measured group effectiveness in communicating clinical disaster information. The triage exercise (module 4) consisted of a tabletop triage exercise using 60 small toys; each toy represented a simulated patient and was imprinted with clinical information. Groups were asked to classify the 60 simulated patients according to Simple Triage and Rapid Treatment (START). Facilitators evaluated trainees based on their ability to work as a team to rapidly assess and accurately triage several patients at once.

Human Simulation and Actor Scenario Development

Over six months, CHPTER worked with a university-based human simulation center to develop a series of fictional clinical disasters that combined up to six patient SimMan high-fidelity and low-fidelity patient simulators (Laerdal, Wappinger Falls, New York, USA) and up to 15 trained live “actors” to simulate a clinical disaster. One of the center’s larger observation rooms resembled a small emergency waiting room, and a door to the outside of the simulation center simulated an emergency department (ED) entrance. Storyboards and a stage map (Figure 1) were developed for the simulated exercise. On the stage map, three zones were identified, with an observer assigned to each zone (O1, O2, O3). The letter A designated a human actor and the letter S designated a human simulator. On the color stage map (available online), blue indicated an observer, while green, red, yellow and black indicated a patient’s START triage classification. If a chair appeared empty, with no patient (no letters, no color), then it was simply an empty chair.

Figure 1.

Simulator and Actor Staging in Human Performance Lab

Blue, observer trainer; green, triage green; yellow, triage yellow; red, triage red; black, triage black A, actor; S, human simulator; 1, zone 1; 2, zone 2; 3, zone 3

Example: 2S1 (red) is a high-fidelity human simulator patient who is triage red in zone 2

Members of the CHPTER task force were trained to operate simulation center equipment and software. Observers were assigned to different geographical areas (zones 1, 2 and 3) of the scene so that they could more easily focus on performance objectives during the chaotic movement of patients and actors during the scenario.

Two months prior to training, 15 actors were recruited —including trained patient actors from the medical university—and provided pre-scripted roles. To enhance reliability and validity of performance objectives expected to be achieved by trainees, actors were taught how to follow specific behaviors when confronted with trainees in a chaotic environment.

Multi-Actor Clinical Disaster Scenario: “Influenza-Like Illness”

The clinical casualty scenario developed by the curriculum task force involved the acute presentation of cruise-line tourists complaining of cough and shortness of breath. During the 6-minute scenario (‘Influenza-Like Illness’) both actors and simulators presented with various levels of respiratory complaints, several of them outside awaiting access to the ED waiting room. Working in teams of four to six, trainees were told they were health care providers from different areas of the hospital called down to mitigate the complex and chaotic scene using skills they had learned during didactic and small group lessons. The responders were not aware that some of patients were suffering from inhalation anthrax. They are also unaware that one of the patients was carrying two bags of powder, presumed to be anthrax.

Of the four human simulators used in this scenario, two were unstable and required active airway maneuvers (i.e., simple jaw thrusts) in order to open their airways. If teams recognized patient acuity and acted prior to four minutes, the simulators’ vital signs normalized. If not, CHPTER observers allowed the physiologic parameters of the simulators to deteriorate irreversibly, resulting in cardiac arrest.

Trainees were also confronted with several actors demanding care who were trained to escalate their behavior during the scenario unless appropriate performance measures were met. Once escorted to the Green Triage (this coincides with the “Team Staging Area and BLS/PPE Equipment” area in Figure 1) waiting area, patients with minor complaints changed costumes and presented to the ED entrance as new patients. One actor (representing the terrorist disguised as a tourist) utilized a distraction caused by other actors to bypass security and enter the ED waiting room. If within four minutes the teams recognized that their resources were outstripped and closed the ED door, the patient surge (and terrorist threat) was averted. If not, the actor entered the ED and held up two bags of powder, exposing all patients and all providers to a simulated lethal dose of anthrax.

An educational intervention was developed for all teams participating in the mass-casualty scenario. Immediately after completing their first attempt at the scenario, team members participated in a facilitator-led debriefing using digital video (and split-screen technology) from four different camera viewpoints. CHPTER instructors reinforced key components of the learning objectives and competencies for the course without prompting trainees with specifics about how to remedy the disaster scene. Per study design, teams repeated the 6-minute scenario after the debriefing. Team membership and scenario content did not change.

Research Metrics

The task force developed an online pre-test and two post-tests for trainees utilizing a pool of questions developed by the task force to meet the learning objectives and competencies of the course. The pre-test consisted of two parts. In the first part, Likert-scale self-assessment questions measured trainees’ sense of personal capability and comfort level to handle a disaster. The second part included 23 discrete multiple-choice questions.

The post-test consisted of three parts. The first part was identical to the first part of the pre-test (Likert-scale self-assessment questions). The second part of the post-test contained 23 discrete multiple-choice questions from the question pool. In some cases, slightly modified questions from the question pool were selected. For example, one post-test question described the same patient triage scenario in the pre-test but assigned different vital signs to the patient so the triage answer was “Red” instead of “Yellow”. The third part of the post-test consisted of Likert-scale and open-ended questions for trainees to evaluate course content and instructors. Trainees were asked to complete the same post-test four to six months following the training.

Performance objectives during the mass-casualty scenario were measured and recorded by three CHPTER observers (zones 1, 2 and 3 observers). Observers recorded whether performance objectives were “met” based on their observations of the team during the scenario. Observers developed a single list of “met” performance objectives for each team and, with the help of digital video playback, were able to complete any part of the checklist they did not have time to finish. Any data missing from an observer’s checklist was considered an “unmet” performance objective.

Descriptive statistics (means, standard deviations, and medians) were used to describe elements of the EPT training, as appropriate. Wilcoxon rank sum tests were used to compare the pre- and post-test values within the cohorts of medical students and VHA trainees. Because many of the pre- and post-tests were completed by the same individuals, and because the test was completed anonymously (as required by the IRB and, unfortunately, making it impossible to conduct paired testing), the p-values obtained from the Wilcoxon rank sum tests are conservative estimates. In other words, if the pre- and post-test scores were linked and a paired analysis performed, the resulting p-values would have been smaller than what was observed in the independent testing done by the Wilcoxon rank sum test.⁴⁰

Results

Participants

On February 28, 2011, 10 fourth-year medical students received approximately nine hours of training at the university simulation center. The students volunteered for the course after receiving email from CHPTER and the College of Medicine. On March 13, 2011, 17 participants from the Veterans Health Administration (VHA) Strategic Health Care Group (EMSHG) were provided the same 9-hour course. The VHA clinicians volunteered for the course after receiving email from CHPTER and the local VHA medical center. In the second year of the course, 14 fourth-year medical students were trained on April 2, 2012. With the exception of a small percentage of actors and course instructors, recruitment methods and course content did not change in the second year of the course. Trainee descriptions are shown in Table 2, along with medical student and VHA demographic data. Trainees (n=39) included 24 (62%) medical students; seven (18%) physicians; seven (18%) nurses; and one (3%) emergency manager. Of the VHA providers, seven (47%) reported greater than 16 annual hours of disaster training while 63% (n=15) of the medical students reported no annual disaster training. One medical student erroneously self-identified as a mental health provider. After confirming that this was an error, the student’s responses were included in the analysis. In addition, two of the VHA clinicians completed the course but did not complete all components of the online assessments, including the demographic data survey. The responses from these clinicians were excluded from the analysis.

Table 2.

Trainee Demographics and Hours of Disaster Training Required and Performed per Year

Variable		VHA Clinicians (n=15)		Medical Students (n=24)		Combined (n=39)
		N	%	N	%	N	%
Male		10	67	12	50	22	56
Nonhispanic White		9	60	20	83	29	74
Age group	25 or under	0	0	4	17	4	10
	26–40	2	13	20	83	22	56
	41–55	9	60	0	0	9	23
	56+	4	27	0	0	4	10
Education	Vocational degree	1	7	0	0	1	3
	Bachelors degree	2	13	16	67	18	46
	Masters degree	4	27	1	4	5	13
	Doctoral degree	0	0	3	13	3	8
	Professional degree	8	53	4	17	12	31
Occupation	Nurse	7	47	0	0	7	18
	Physician	7	47	21	88	28	72
	Emergency Management	1	7	0	0	1	3
	Mental Health	0	0	1	4	1	3
	Other	0	0	2	8	2	5
Years working	Still in training	0	0	22	92	22	56
	3–5 years	1	7	2	8	3	8
	6–10 years	3	20	0	0	3	8
	11–20 years	4	27	0	0	4	10
	>21 years	7	47	0	0	7	18
Training required	0 hours	5	33	17	71	22	56
	1–2	7	47	2	8	9	23
	3–8	2	13	2	8	4	10
	9–16	1	7	0	0	1	3
Training performed	0 hours	0	0	15	63	15	38
	1–2	1	7	3	13	4	10
	3–8	4	27	4	17	8	21
	9–16	3	20	0	0	3	8
	17–25	3	20	0	0	3	8
	26+	4	27	0	0	4	10

Discrete Knowledge: Before and After EPT

Pre- and post-test data for medical students and VHA clinicians are depicted in Table 3 for before and after EPT as well as at a point four to six months after EPT

Table 3.

Trainee Cognitive Results Before and After Training and Self Assessments, and Course Assessment Results

		VHA Clinicians				Medical Students
		Pre Mean (SD)	Post Mean (SD)	p-value	Follow-up Mean(SD)	Pre Mean (SD)	Post Mean (SD)	p-value	Follow-up Mean(SD)
Cognitive	Overall scores (out of 24)	15.1 (3.0)	19.4 (2.3)	<0.01	18.5 (1.6)	10.6 (3.2)	17.8 (2.0)	<0.01	15.3 (2.0)
	Percent correct	63% (12%)	81% (10%)	<0.01	77% (7%)	44% (13%)	74% (8%)	<0.01	64% (8%)
Self Assessment	Mobilization	69.1 (20.9)	87.5 (9.3)	<0.01	82.8 (12.5)	40.6 (23.5)	77.7 (14.0)	<0.01	80.2 (11.7)
	Team	74.7 (21.7)	88.4 (9.0)	0.02	86.2 (12.0)	26.5 (16.5)	74.4 (12.1)	<0.01	81.7 (5.5)
	Communications	69.9 (23.6)	85.2 (10.1)	0.04	76.7 (6.6)	36.2 (17.5)	75.8 (12.8)	<0.01	81.0 (6.1)
	Triage	69.7 (12.7)	89.1 (7.1)	<0.01	88.5 (4.5)	33.3 (19.9)	82.7 (11.9)	<0.01	83.3 (11.9)
	ICS	71.6 (16.9)	87.6 (10.0)	<0.01	87.5 (10.5)	22.5 (19.9)	71.0 (13.8)	<0.01	73.0 (9.7)
	Overall Knowledge	73.5 (16.7)	83.5 (12.8)	0.04	83.0 (9.3)	24.6 (15.2)	71.7 (12.2)	<0.01	76.3 (6.0)
	Overall Skill	77.8 (18.4)	82.8 (17.3)	0.4	88.7 (5.6)	31.7 (15.8)	75.9 (13.5)	<0.01	79.0 (3.6)
		Median	Min	Max		Median	Min	Max
Course Assessment	Recommend course	92.5	65	100		98	78	100
	One day course feasible	82.5	38	100		84	67	100
	Performance	84.5	60	100		95	70	100
	Pre course material	81	19	100		75	50	100
	Lectures	90	42	100		88	71	100
	Small group sessions	94.5	71	100		82	68	100
	Mass casualty	90	74	100		93	70	100
	Presenters	94	67	100		100	62	100
	Facilities	95	58	100		97.5	66	100
	Overall evaluation	94.5	76	100		100	61	100

Self-Assessment of EPT Knowledge and Skill

Both medical students and VHA clinicians were asked to assess their EPT skills and comfort level with performance during a clinical disaster utilizing a 1 to 100 analog scale with higher scores indicating increased comfort levels. Data for medical student skill rating are depicted in Table 3.

EPT Course Assessment

Trainees were asked to rate the course using an analog scale from 1 (did not meet expectations) to 100 (exceeded expectations). Higher scores indicated positive responses for the evaluation of the training components. These data are presented in Table 3.

Small-Group Exercise Performance

Results of the small-group exercise were as follows:

Team Building: Module 2

For the seven performance components in the team building exercise, 89% (8/9) of teams “Identified a Team Leader” and 78% (7/9) “Pre-Huddled, Team Leader Assigned Roles, and Team Members Understood Roles” whereas 69% (6.9) of teams “Recognized and Discussed the Threat.” All teams “Completed at Least One Image Puzzle, Accurately Described Scene” and reported a positive learning experience with the team building exercise.

Communication: Module 3

Eight of the nine teams completed all performance components (Discussion of Chain of Command, Discussion of Relevant Verbal Communication, and Verbalize an Accurate, Concise and Clear Report). Teams’ average rating of the ‘Bus Crash’ scenario during the small group exercise using an analog scale, from 1 (did not meet expectations) to 5 (exceeded expectations) was 4.4 out of 5.

Triage: Module 4

For the first five performance components (Developed Team, Communicated Appropriately with Other Medical Personnel, Constructed Appropriate Scene Layout, Classified Patients Using START Triage System, and Reassessed Patients Already Triaged), 8 of 9 groups completed all. For the last performance component (Classified All Victims within the Time Limit), 3 of 9 groups were not able to finish on time. The average number of patients reported by each triage category relative to accurate triage count was Red (20.3 versus 21), Yellow (17.0 versus 19), Green (12.2 versus 12) and Black (8.1 versus 8).

Clinical Mass-Casualty Scenario Team Performance

During the 6-minute scenario, nine teams of four to five trainees were asked to mitigate a clinical disaster scene including three simulated patients and 11 actors. Ten of the actors were patients and one of the actors served as the ED security guard. The zone 1 observer was responsible for rating team performance for all teams (Table 4).

Table 4.

Team Performance Objectives Completed by Nine Teams During Clinical Mass-Casualty Scenario, by Zone

Zone	Performance Objective Number	Patient	1st Scenario Attempt	2nd Scenario Attempt	Performance Objective
1	1	All	6 (67%)	9 (100%)	Established roles and responsibilities
	2	All	8 (89)	7 (78)	Selected appropriate equipment
	3	All	5 (56)	9 (100)	Discussed safety plan prior to action
	4	All	8 (89)	9 (100)	Selected and donned appropriate PPE
	5	All	8 (89)	8 (89)	Established communication link with EOC
	6	All	4 (44)	7 (78)	Effectively communicated needs on scene to EOC
	7	All	6 (67)	9 (100)	Followed assigned roles
	8	All	7 (78)	6 (67)	Followed chain of command
	9	All	1 (11)	5 (56)	Reassessed patients and scene
	10	All	3 (33)	9 (100)	Secured doors to ER after recognizing disaster
2	1	2A1	8 (89)	9 (100)	Correctly assigned yellow
	2	2A1	8 (89)	8 (89)	Attempted to reassure and calm
	3	2A1	2 (22)	2 (22)	Reassessed (no change)
	4	2S1	4 (44)	8 (89)	Performed airway maneuver
	5	2S1	4 (44)	7 (78)	Assigned red
	6	2S1	0 (0)	3 (33)	Reassessed and changed triage to yellow
	7	2S1	0 (0)	2 (22)	Reassured and calmed after performing airway maneuver
	8	2A2	7 (78)	9 (100)	Assigned green
	9	2A2	8 (89)	8 (89)	Reassured and calmed
	10	2A2	7 (89)	9 (100)	Reassessed (no change) and sent to green triage staging facility (no change)
3	1	3S1	6 (67)	7 (78)	Performed airway maneuver
	2	3S1	7 (78)	9 (100)	Assigned black
	3	3S1	3 (33)	6 (67)	Reassessed (no change)
	4	3S1	3 (33)	7 (78)	Discontinued treatment
	5	3S2	5 (56)	9 (100)	Performed airway maneuver
	6	3S2	5 (56)	8 (89)	Assigned red
	7	3S2	2 (22)	8 (89)	Reassessed
	8	3S2	3 (33)	8 (89)	Reassured and calmed
	9	3A1	6 (67)	8 (89)	Assigned green
	10	3A1	7 (78)	7 (78)	Reassured and calmed
	11	3A1	5 (56)	8 (89)	Reassessed and sent to green triage staging facility
	12	3A2	6 (67)	9 (100)	Assign green
	13	3A2	6 (67)	8 (89)	Reassured and calmed
	14	3A2	7 (78)	8 (89)	Reassessed and sent to green triage staging facility

Abbreviations: ER, emergency room; EOC, emergency operations center; PPE, personal protective equipment

Likert Scale Assessment of Team Performance

There were nine categories of team performance (Recognition of Disaster, External Communication, Internal Communication, Utilization of Resources, Patient Triage and Care, Patient Counseling, Personal and Staff Safety, and Facility Safety and Security), and their assessments are depicted in Table 5. For all categories, teams improved their scores after the debriefing (Table 5).

Table 5.

Clinical Mass-Casualty Scenario Team Assessment

Team Assessment Categories	Teams’ 1st Attempt		Teams’ 2nd Attempt
	Mean	SD	Mean	SD
Recognition of Disaster	5.5	1.9	8.1	1.5
External Communication	6.6	2.4	8.0	2.1
Internal Communication	5.3	1.8	7.9	1.5
Utilization of Resources	6.3	1.3	7.6	1.0
Patient Triage and Care	6.5	1.1	7.8	1.6
Patient Counseling	6.4	2.1	7.2	1.7
Personal and Staff Safety	5.3	2.0	7.4	2.3
Facility Safety and Security	3.4	2.5	6.8	2.4

(1–10 Scale; 1=Did not meet expectations and 10=Exceeded expectations)

Disaster Scenario Team Performance

Of the 34 performance objectives during the 6-minute scenario, 23 were completed by at least half of the teams during their first attempt. Thirty-one were completed by at least half of the teams during their second attempt. Of the 13 patients in this disaster scene, two simulated patient (2S1 and 3S2) were suffering from occluded airways and could be saved if teams intervened clinically with an airway maneuver prior to the 4-minute mark. Four of nine teams in zones 2 and 3 were able to resuscitate 2S1 during their first attempt; 5 of 9 teams were able to resuscitate 3S2 during their first attempt. All but one of the teams resuscitated 2S1 and 3S2 after the debriefing. In zone 1, 3 of 9 were able to secure the ED door and thwart an exposure to anthrax within four minutes during their first attempt and all teams were able to secure the ED within four minutes during their second attempt. Team performance (i.e., teams’ ability to resuscitate 2S1 and 3S2 and prevent anthrax exposure) as a function of declining patient stability and scene safety are presented in Figures 2, 3, and 4.

Figure 2.

Median Time to Meet Performance Objectives in Zone 1, by Team Attempt and Safety of Disaster Team

Figure 3.

Median Time to Meet Performance Objectives in Zone 2, by Team Attempt and Stability of Patient 2S1

Figure 4.

Median Time to Meet Performance Objectives in Zone 3, by Team Attempt and Stability of Patient 3S2

Discussion

Efforts to foster EPT to medical trainees in a simulated environment date back to the 1950s when researchers assessed the use of movies to augment students’ training for military and disaster events.⁴¹ The Institute of Medicine’s 1999 report, “To Err is Human: Building a Safer Health System,” spurred new development of the use of human patient simulators to train medical providers.⁴² In the 1990’s, lessons learned from the aviation industry’s Cockpit Resource Management (CRM) led a group of anesthesiologists to pioneer the Acute Crisis Resource Management (ACRM) training module that uses human patient simulators to help reduce error during emergent clinical events.^43–45 Subsequent research has suggested a potential benefit of human simulators to train care providers for clinical emergencies.^46–56

Early applications of simulation to emergency medicine include its efficacy to provide emergency medicine team training,⁵⁷ a pilot study of simulation to provide Emergency Medicine Crisis Resource Management (EMCRM) training⁵⁸ and the theoretical use of multiple patients during simulation to replicate the chaotic environment in the Emergency Department.⁵⁹ Other descriptions of simulation in emergency and military medicine are available.^{9, 13, 28, 57, 60–67} Recently a comprehensive list of disaster core competencies for acute care providers was published, but it provides little guidance as to how to incorporate simulation into the disaster classroom.⁶⁸

A literature search in the PubMed database was performed. Additional online searches were completed to capture government reports from, for example, AHRQ (US Agency for Healthcare Research and Quality, www.ahrq.gov). The search yielded approximately 350 articles and reports. Several evidence-based studies were found that established the efficacy of human simulation to train care providers for a disaster. ^{12,33,63,69–72} Only one study utilizing a multi-actor clinical disaster scenario to measure the lifesaving performance of medical trainees was found. Wallace quantified the disparity between the times required to resuscitate simulators and actors during a simulated disaster drill. Six simulators and six actors were presented to triage teams in waves of three patients at a time.⁷³ Because the maximum number of patients during an individual triage encounter was limited in the Wallace study, it is not clear whether the tested curricula would have an impact on trainees confronted with several patients at once.

The Center for Health Professional Training and Emergency Response’s 1-day EPT course for care providers recreated a loud and chaotic clinical disaster scene (> 10 human actors with human simulators at one time) and enhanced care provider knowledge, skills, and comfort level necessary to save lives during a simulated disaster. Discrete knowledge of trainees improved significantly across the board after taking the course in 2011 and 2012. Statistically significant improvements were noteworthy for both the medical students and the experienced disaster medical providers from the VHA, suggesting that this course could be suitable to both novice and experienced audiences. Trainee self-assessment of their EPT knowledge and skill also dramatically improved after taking the course. The overall rating for the course was 96 out of 100, which represents one of the highest course evaluations completed at the medical university.

The human performance measured during the three small group exercises (Team Building, Communication and Triage) demonstrated that nearly all of the groups were able to complete performance components accurately and on time. During the disaster scenario in the afternoon session, trainees were faced with a multi-actor simulated disaster. Overall, teams performed well; two out of three performance objectives were met during the first attempt and teams improved their performance after participating in a short debriefing. Teams successfully identified and performed rescue airway maneuvers to resuscitate two simulated patients who otherwise would have expired. They also identified security threats and implemented measures to prevent exposure to anthrax.

While none of the components of the curriculum are individually unique or novel, the curriculum is competency-based with measurable performance outcomes and potentially applicable to a wide range of medical providers. With the exception of Wallace, this study is unique in that it demonstrates medical trainees’ ability to recognize and resuscitate human simulators during a multi-actor simulated clinical disaster. Based on a review of the literature, this is the first published curriculum to use high-fidelity simulation and multi-actor scenarios to measure the lifesaving performance of care providers confronted with >10 patients at once.

Limitations

The power assessment of this study (n=39) is low and the external validity of this curriculum has not been established. A larger-scale study, or preferably a multi-center trial, would allow further analysis of the impact and validity of the curriculum. The use of trained professional actors (who played key patients) as well as trained volunteers (who played patients in the crowd) was utilized to minimize variations in human behavior, improving but not perfecting the study’s reliability.

The study sought to measure the impact of EPT on discrete knowledge both immediately after and four to six months after the training day. Unfortunately, these results were not statistically significant, largely because several trainees were lost to follow-up during the first year of the course. A more robust predesign and postdesign with appropriate follow-up could sufficiently power this comparison. In addition, trainees were not tested prior to the disaster scenario (before the training day), primarily because of time and financial constraints. This limits the study’s ability to ascertain a performance baseline of the trainees.

Future iterations of CHPTER curricula will need to include a cost assessment to demonstrate the feasibility of this curriculum for target audiences lacking human simulation capability. A version of the curriculum that uses low-cost, low-fidelity simulators and actors is currently under production by CHPTER.

Conclusion

This one-day EPT course improved care provider knowledge, skill, and comfort necessary to save lives during a simulated disaster. Trainees were able to resuscitate two unstable simulated patients and prevent anthrax exposure to the hospital during a chaotic and loud disaster scenario. A larger-scale study, or preferably a multi-center trial, is needed to further study the impact of this curriculum and its potential to protect provider and patient lives.

Abbreviations

CHPTER

Center for Health Professional Training and Emergency Response

ED

emergency department

EPT

emergency preparedness training

IRB

institutional review board

START

Simple Triage and Rapid Treatment

VHA

Veterans Health Administration