Impact of Simulation-Based Training on Perceived Provider Confidence in Acute Multidisciplinary Pediatric Trauma Resuscitation

Abstract

Purpose:

Simulation-based training has the potential to improve team-based care. We hypothesized that implementation of an in situ multidisciplinary simulation-based training program would improve provider confidence in team-based management of severely injured pediatric trauma patients.

Methods:

An in situ multidisciplinary pediatric trauma simulation-based training program with structured debriefing was implemented at a free-standing children’s hospital. Trauma providers were anonymously surveyed one month before (pre-), one month after (post-), and two years after implementation.

Results:

Survey response rate was 49% (n=93/190) pre-simulation, 22% (n=42/190) post-simulation, and 79% (n=150/190) at two-year follow-up. These providers reported more anxiety (p=0.01) and less confidence (p=0.02) one-month post-simulation. At two-year follow-up, trained providers reported less anxiety (p=0.02) and greater confidence (p=0.01), compared to untrained providers.

Conclusions:

Implementation of an in situ multidisciplinary pediatric trauma simulation-based training program may initially lead to increased anxiety, but long-term exposure may lead to greater confidence.

Level of Evidence:

II. Prospective cohort.

Introduction

Unintentional injury remains the leading cause of death among children 1 to18 years old.[1] Treatment of pediatric trauma patients at centers with well-coordinated multidisciplinary teams results in improved survival and better outcomes.[2,3] Specialized care at a pediatric trauma center can further improve outcomes, particularly for those with injury severity scores (ISS) > 15.[4,5] Children with severe injuries, however, constitute a minority of patients at level I or II pediatric trauma centers.[6,7] Because mortality is most likely to occur within 24 hours after injury,[8] the initial resuscitation phase is a critical component of these patients’ care. Quality, efficiency, and cohesiveness of the trauma resuscitation team may be life-saving for severely injured children, but individual provider workload during the resuscitation of these patients is high.[9] Establishing cohesive team function may be more difficult in centers with high team member turnover or teams of providers who do not frequently work together treating critically injured children in this setting.[10]

The use of simulation-based team training has been shown to have many benefits in high-stakes multidisciplinary trauma resuscitation. Team performance, communication, and efficiency improve with the use of structured simulation-based training in a variety of settings – including trauma resuscitation.[11–14] An improvement in patient-level outcomes has been shown after institution of simulation-based “mock codes” for pediatric cardiopulmonary arrest.[15] For pediatric trauma resuscitation, simulation-based training can improve specific task and team performance.[16]

Several studies have examined individual provider perceptions of the value of simulation. Simulation-based training in the pediatric emergency department has been shown to improve provider perceptions of patient safety as well as decrease patient safety-related adverse events.[17] Improvements in provider confidence in individual skills following simulation-based training have been reported in the context of adult trauma resuscitation, management of intensive care emergencies, and within nursing education.[14,18,19] None of these studies have quantified the overall effect of in situ simulation on a participant’s confidence as a member of the team, particularly in the setting of high acuity pediatric trauma resuscitation. Few studies have addressed long-term effects after implementation of an in situ multidisciplinary simulation program in any setting.[20,21]

We implemented a multidisciplinary in situ simulation-based team training program at our free-standing children’s hospital in 2014 with the aim of improving the outcomes for severely injured pediatric trauma patients. The purpose of this study was to evaluate the impact of this simulation-based team training program on provider confidence within this environment. We hypothesized that sustained implementation of our program would improve provider confidence as a member of the trauma resuscitation team. Secondarily, we hypothesized that provider anxiety and perceived quality of care would improve after introduction of simulation-based training.

1. Methods

2.1. Participants

This two-year (2014 to 2016) prospective observational cohort study was approved by the Institutional Review Board at Children’s Hospital Los Angeles. Our multidisciplinary simulation program was designed to include all potential providers who typically respond to a level I (highest tier) trauma activation. This cohort included pediatric emergency medicine attending physicians and fellows, pediatric surgery residents and fellows, as well as trauma nurses from the emergency department (ED) and critical care nurses from the pediatric intensive care unit (PICU). Additional disciplines included respiratory care practitioners, clinical pharmacists, radiology technologists, ED technicians, vascular access nurses, and nursing supervisors. Random times were selected for the training, making it possible that a provider could respond twice or more during the study period.

2.2. Simulation-Based Training and Debriefing

Beginning in 2014, we implemented a multidisciplinary in situ simulation-based training program for pediatric trauma resuscitation. Simulation-based team training continued on a regular basis at our institution, with 15 sessions conducted before the two-year follow-up evaluation. Simulations most frequently occurred near the summer to coincide with new hires and to avoid times of peak ED volume (e.g., influenza season). To preserve fidelity, simulation-based training sessions were not pre-announced and were paged out as actual level I trauma activations. These simulations were carried out in situ in the trauma bay, with sessions conducted during daytime and overnight shifts. Predetermined criteria to terminate the simulation included a coincident actual trauma activation or high surge capacity in the ED.

Each simulation-based training session used one of the following high-fidelity pediatric simulators: SimBaby™ (Laerdal Medical, Stavenger, Norway), Gaumard HAL S3004 pediatric simulator (Gaumard® Scientific, Miami, FL), or TraumaChild^® (Simulab Corporation, Seattle, WA) combined with the Laerdal SimBaby™ software and vital sign monitors. SimBaby™ and HAL S3004 are physiologic simulators capable of responding to interventions including intubation, intravenous and intraosseous line placements, chest tube insertions, and cardioversion. In contrast, TraumaChild^® is a procedural simulator that does not respond to interventions but does allow extensive procedures, such as tracheostomy, pericardiocentesis, diagnostic peritoneal lavage, and central line placement. For the purposes of this study, when TraumaChild^® was used, the SimBaby™ physiologic patient monitor and software were utilized manually without SimBaby™ mannequin to add additional realism and consistency to the TraumaChild^® scenario.

We developed three scenarios lasting between 10 and 15 minutes based on actual trauma patients previously treated at our hospital. These scenarios included: 1) two-year-old with pneumothorax and diminishing mental status after a multi-story fall, 2) three year-old with hemorrhagic shock, significant facial trauma, an unstable airway, and multiple mangled extremities after a high-speed motor vehicle collision with ejection, and 3) 18 month-old severe traumatic brain injury (TBI) with impending herniation after a fall from a balcony. Each scenario started with a simulated EMS pre-hospital notification giving the trauma team five minutes to organize and prepare before the patient arrived. Essential elements of EMS pre-hospital history and physical findings were provided at the onset. We specifically designed scenarios that require immediate procedures, team communication, and implementation of Pediatric Advanced Life Support (PALS) and Advanced Trauma Life Support (ATLS) protocols. The facial trauma scenario eventually necessitated mobilization of the blood bank and operating room.

After each simulation-based training session, participants underwent a 40-minute structured debriefing with one of two trained debriefers. Debriefing used the advocacy-inquiry method[22,23] to focus on teamwork and team function through a reflective learning process. We chose this method of debriefing to isolate thought processes and biases or assumptions that led to communication errors or suboptimal performance. The goal was to improve leadership, team communication, planning ahead, and resource management.

2.3. Outcomes Measurement

To evaluate the effect of simulation on providers’ perceptions, trauma providers that may potentially respond to level I trauma activations were asked to complete an anonymous survey one month before implementation of the training program (pre-simulation survey), one month after implementation (post-simulation survey), and two years after the initiation of the simulation-based training program (follow-up survey). The survey was developed by the senior author (ARJ, a pediatric trauma and critical care surgeon) and one of the co-authors (RVB, an epidemiologist trained in qualitative and survey research). It was piloted for content and readability by a multidisciplinary panel of trauma providers (trauma nurses, emergency room nurses, surgeons, and emergency physicians) prior to distribution to the study sample. The final survey (Appendix) included 35 statements that were based on behavioral domains and goals described in the trauma nontechnical skills (T-NOTECHS) scale, a validated measure of trauma teamwork.[24] These statements evaluated perceived limitations of team function (individual skills, teamwork skills, or leadership skills) and the perceived utility of simulation-based training for individual and teamwork skills. Survey items also assessed confidence, leadership, communication, cooperation, assessment and planning, awareness and the ability to deal with stress, and areas for improvement.

All trauma providers were surveyed, including those who did not participate in one of the simulated trauma sessions. The survey was administered in paper format to providers at mandatory division meetings to maximize response rates. Study participants were permitted to take the survey home and return it to a study team member at their convenience. Provider and staff turnover were not tracked. To differentiate trained from untrained provider responses, post-simulation and follow-up surveys queried whether the individual participated in any simulation session. All items were scored on a five-point Likert scale from 1 = “strongly disagree” to 5 = “strongly agree.”

2.4. Statistical Analysis

The role of respondents was compared between pre-simulation, post-simulation, and follow-up periods using the χ² test. Because surveys were anonymous, post-simulation and follow-up surveys could not be compared to pre-simulation surveys in a paired fashion. Pooled pre-simulation responses of all providers were compared to post-simulation and follow-up responses for: 1) trained providers who participated in a simulation, and 2) untrained providers who had not participated in a simulation. Survey responses were also compared between trained and untrained providers at two-year follow-up. The Mann Whitney-U test was used to compare response data because survey responses did not follow a normal distribution. Comparisons were two-tailed, with alpha = 0.05 considered statistically significant. Data analyses were performed using SPSS v 17.0 (Chicago, IL).

2. Results

One hundred ninety total providers comprised the pool of providers who might respond to a trauma activation. Survey respondents included 93 (49%) in the pre-simulation period, 42 (22%) in the one-month post-simulation period, and 150 (79%) at two-year follow-up. Respondents who underwent training included 15 (36%) in the one-month post-simulation period and 85 (57%) at two-year follow-up. In the one-month post-simulation period, no providers had participated in more than one simulation (median [IQR] 1 [1,1] simulation). At two-year follow-up, respondents participated in a median (IQR) of 1 (0, 2) simulations, with 34 (23%) participating in more than one simulation. The breakdown of survey respondents by specialty (Table 1) was different across time periods, with a higher proportion of nurses at two-year follow-up (p < 0.01). No respiratory care practitioners or radiology technicians responded to the post-simulation survey, while these specialties accounted for 16% and 19% of total pre-simulation and follow-up respondents, respectively.

Table 1:

Breakdown of trauma team provider role in survey respondents before and after implementation of in situ simulation-based training for pediatric trauma resuscitation.

Provider Role	Pre-Simulation n=93	One-month Post-Simulation n=42	Two-year Follow- Up n = 150
Pediatric Emergency Medicine Physician	13 (14)	11 (26)	11 (7)
Surgeon	15 (16)	7 (17)	11 (7)
Registered Nurse	48 (51)	18 (43)	98 (65)
Respiratory Care Practitioner	12 (13)	0	15 (10)
Radiology Technologist	3 (3)	0	14 (9)
Unknown	2 (2)	6 (14)	1 (1)

All values reported as n (%). Total providers available to complete a survey at any time point, N = 190.

During pre-simulation surveys, providers indicated that overall team function was most limited by teamwork, and not individual skills (Fig. 1). More than 85% reported simulation-based training would improve both individual and team performance with a higher perceived benefit for team skills (Fig. 2).

Figure 1:

Survey of provider perception of limitations of trauma team function prior to implementation of in situ simulation-based training for pediatric trauma resuscitation.

N = 93. Survey responses were scored on a 5-point Likert scale with 1=strongly disagree and 5=strongly agree. P-value represents comparison of median rating with Mann Whitney U.

Figure 2:

Survey of provider perceptions of the utility of simulation-based training prior to implementation.

N = 93. Survey responses were scored on a 5-point Likert scale with 1=strongly disagree and 5=strongly agree. P-value represents comparison of median rating with Mann Whitney U.

Comparing pre- and post-simulation surveys (Table 2), trained providers at one-month post-simulation reported increased anxiety, less comfort with individual roles, less confidence in individual role, and less confidence in overall team function. Trained providers reported a decrease in prompt communication regarding patients’ needs and decreased quality of patient care. For the providers who did not participate in simulation, none of these differences were found (all p>0.05). At two-year follow-up, trained providers no longer reported changes in anxiety, comfort level, confidence in their role or team function, communication, or quality of care. Conversely, trained providers reported increased comfort with unexpected findings in a trauma patient at the two-year follow-up point when compared to the pre-simulation surveys.

Table 2:

Comparison of selected survey responses from trauma team providers before and after implementation of in situ simulation-based training for pediatric trauma resuscitation.

	Pre-Simulation	Post-Simulation		Two-year Follow-Up
Question	All Providers n=93	Trained Providers n=15		Trained Providers n=85
	Mean (SD)	Mean (SD)	p-value†	Mean (SD)	p-value†
I feel anxiety when participating as a member of the trauma team	2.3 (1.0)	3.0 (1.0)	0.01	2.6 (1.2)	0.05
I am confident playing my role in the trauma team	4.4 (0.7)	3.9 (0.7)	0.02	4.3 (0.7)	0.61
I feel each trauma patient is given the best possible quality of care	4.2 (0.8)	3.6 (1.0)	0.03	4.1 (0.9)	0.53
I feel patient needs are identified and communicated promptly	3.9 (0.8)	3.3 (1.0)	0.01	3.9 (0.8)	0.69
I feel injuries are assessed efficiently and effectively	4.1 (0.8)	3.7 (1.0)	0.11	4.1 (0.8)	0.54
I feel comfortable addressing unexpected findings in a trauma patient	4.0 (0.7)	3.7 (0.8)	0.19	4.2 (0.7)	0.02

Survey responses were scored on a 5-point Likert scale with 1=strongly disagree and 5=strongly agree. Total providers available to complete a survey at any time point, N = 190.

^†Mann-Whitney U. Comparison is to aggregate pre-training survey data for all providers (including providers not participating in simulation-based training)

Comparisons between trained and untrained providers at two-year follow-up (Table 3) revealed many differences. Trained providers reported feeling more prepared, less anxiety, more individual confidence, more understanding of individual role, and more comfortable treating trauma patients in extremis, when compared to untrained providers. Trained providers were less likely to report their individual skills to be a limitation of team function.

Table 3:

Comparison of selected survey responses between trained and untrained trauma team providers at two-year follow-up after implementation of in situ simulation-based training for pediatric trauma resuscitation.

	Two-year Follow-Up
Question	Trained Providers n=85	Untrained Providers n=64	p-value†
I feel anxiety when participating as a member of the trauma team	2.6 (1.2)	3.1 (0.9)	0.02
I am confident playing my role in the trauma team	4.3 (0.7)	3.7 (0.9)	0.01
I feel each trauma patient is given the best possible quality of care	4.1 (0.9)	3.9 (0.8)	0.18
I feel patient needs are identified and communicated promptly	3.9 (0.8)	3.8 (0.8)	0.76
I feel comfortable addressing unexpected findings in a trauma patient	4.2 (0.7)	3.7 (0.8)	0.01
Overall team function is currently most limited by my individual skills	2.5 (1.1)	3.1 (1.1)	0.01

All values reported as mean (SD). Survey responses were scored on a 5-point Likert scale with 1=strongly disagree and 5=strongly agree. Total providers available to complete a survey at any time point, N = 190.

^†Mann-Whitney U comparing trained to untrained providers.

3. Discussion

This study examined the changes in provider confidence over a two-year period after implementation of a multidisciplinary in situ simulation-based training program for pediatric trauma resuscitation. Our findings likely reflect an institutional cultural change in the approach to team-based resuscitation, with an emphasis placed on communication, teamwork, leadership and collaborative learning through simulation. Trained providers felt less prepared, more anxiety, and less comfortable with their role on the team one month after simulation-based training. At two-year follow-up, however, trained providers reported less anxiety, more confidence, and more comfort with their role and management of challenging scenarios, when compared to those who did not receive training. These results suggest that simulation-based training in pediatric trauma resuscitation may initially help identify perceived provider and team deficiencies, leading to improvements over time by directing education to address these perceived deficiencies.

The initial decrease in provider confidence may reflect a relative lack of experience in the high acuity trauma scenarios used for training. As a regional referral center, over 60% of our trauma patients arrive as transfers from another facility, resulting in limited opportunities for our trauma resuscitation team to participate in the acute resuscitation of severely injured patients. The simulated scenarios were likely the first experience with high acuity injuries for many providers, potentially alerting them to the poor communication and teamwork that frequently occur in this chaotic environment. Another source of the initial decrease in provider confidence may have been the post-simulation debriefing technique. The advocacy-inquiry technique used for debriefing probes into the origins of communication errors by clearly identifying errors to the team and encouraging discussion from multiple team-member perspectives.[22,23] Although the focus was not on individual errors, debriefing as a team and openly discussing errors may have increased anxiety and decreased confidence more than a written or individual debriefing.

The increase in confidence between trained and untrained providers at two-year follow-up may be associated with repeated exposure. Similar findings were reported by van Schaik et al, who examined an in situ pediatric simulation program (primarily medical resuscitation) two years after implementation, finding an increase in leadership confidence scores among participating residents.[20] Falcone et al also demonstrated the impact of repeated exposure in team-based pediatric trauma simulation, finding an increase in task completion when comparing early (first 4 months) to late (last 4 months) simulation groups over a 1-year period.[16] With repeated exposure to simulation, anxiety and confidence issues may be reduced as providers have more opportunities to practice and become more comfortable with their respective roles. Unfortunately, the ideal number of pediatric trauma simulations needed to provide the necessary experience, skills and knowledge has yet to be established. It is likely that a sustained simulation program with regularly scheduled simulation training is most impactful.

In situ simulation has been shown to be more realistic and more effective at developing leadership and teamwork compared to training in a simulation center.[21] This realism also may facilitate effective identification of latent safety threats during in situ simulation.[25] We witnessed a similar effect when providers in two pneumothorax simulations reported not knowing where the chest tubes were located. This identification of latent safety threats may have contributed to the initial decrease in confidence and increase in anxiety that then decreased over time as these safety threats were addressed.

This study has several limitations. First, the low response rate at one-month post-simulation limited our internal validity at that time point. It is not clear why the response rate was poor at one month. Low response rate may have been due to providers ignoring the survey at that time if they had not yet participated in a simulation, although we attempted to prevent this effect by encouraging all trauma providers to complete the survey regardless of their simulation participation. The survey responses may also have been biased towards participants who had a more negative experience, leading to a type 1 error at one-month follow-up. The second limitation was the difference in composition of survey respondents across time points. In another study evaluating the effect of simulation-based training in an intensive care unit setting, advanced practitioners reported a significant increase in confidence after simulations, but fellows did not.[18] If a similar role-specific effect happened in our study, it may have biased our two-year follow-up surveys away from the null hypothesis. A third limitation was that surveys were anonymous. We could not compare individual survey scores over time, and the direct individual effect of simulation could not be assessed. The anonymity also limited our ability to track the impact of provider turnover as some providers responding to the pre-simulation survey may have left the institution and some providers responding to later surveys may have been new hires. This unmeasured factor may have led to the perceived confidence difference in the two-year follow-up time point if all new hires had low confidence and had not participated in simulation-based training. Lastly, this study is limited by the lack of objective measures of trauma team performance. Although we showed improvements in provider confidence over time, these findings may have been more meaningful in the context of improvements in team performance or real-life clinical outcomes.

In conclusion, the implementation of an in situ simulation-based training program for multidisciplinary pediatric trauma resuscitation led to contrasting immediate and long-term effects. In the short-term, simulation-based training raised self-awareness and may have functioned as a ‘wake-up call’, making providers more aware of individual and team deficiencies and the consequences of poor teamwork on patient care. At two years, trained providers were more confident and comfortable than untrained providers. This study provides evidence that simulation training can yield long-lasting improvements in provider confidence, likely through a change in institutional culture rather than simple change in individual provider perceptions.