Skip to main content
Annals of the American Thoracic Society logoLink to Annals of the American Thoracic Society
. 2015 Apr;12(4):512–519. doi: 10.1513/AnnalsATS.201501-030OC

A Randomized Trial Comparing Didactics, Demonstration, and Simulation for Teaching Teamwork to Medical Residents

Matthew W Semler 1,*,, Raj D Keriwala 1,*, Jennifer K Clune 2, Todd W Rice 1, Meredith E Pugh 1, Arthur P Wheeler 1, Alison N Miller 1, Arna Banerjee 3, Kyla Terhune 4, Julie A Bastarache 1
PMCID: PMC5466182  PMID: 25730661

Abstract

Rationale: Effective teamwork is fundamental to the management of medical emergencies, and yet the best method to teach teamwork skills to trainees remains unknown.

Objectives: In a cohort of incoming internal medicine interns, we tested the hypothesis that expert demonstration of teamwork principles and participation in high-fidelity simulation would each result in objectively assessed teamwork behavior superior to traditional didactics.

Methods: This was a randomized, controlled, parallel-group trial comparing three teamwork teaching modalities for incoming internal medicine interns. Participants in a single-day orientation at the Vanderbilt University Center for Experiential Learning and Assessment were randomized 1:1:1 to didactic, demonstration-based, or simulation-based instruction and then evaluated in their management of a simulated crisis by five independent, blinded observers using the Teamwork Behavioral Rater score. Clinical performance was assessed using the American Heart Association Advanced Cardiac Life Support algorithm and a novel “Recognize, Respond, Reassess” score.

Measurements and Main Results: Participants randomized to didactics (n = 18), demonstration (n = 17), and simulation (n = 17) were similar at baseline. The primary outcome of average overall Teamwork Behavioral Rater score for those who received demonstration-based training was similar to simulation participation (4.40 ± 1.15 vs. 4.10 ± 0.95, P = 0.917) and significantly higher than didactic instruction (4.40 ± 1.15 vs. 3.10 ± 0.51, P = 0.045). Clinical performance scores were similar between the three groups and correlated only weakly with teamwork behavior (coefficient of determination [Rs2] = 0.267, P < 0.001).

Conclusions: Among incoming internal medicine interns, teamwork training by expert demonstration resulted in similar teamwork behavior to participation in high-fidelity simulation and was more effective than traditional didactics. Clinical performance was largely independent of teamwork behavior and did not differ between training modalities.

Keywords: patient simulation, education, team training, medical care team


Safe and effective patient care during emergencies requires highly trained individuals with distinct skillsets working together as a team. Failures in teamwork contribute disproportionately to errors during acute care (16), and improved teamwork is associated with improved patient safety (7). Nonetheless, the most effective method by which to teach the skills of leadership, communication, and mutual performance monitoring that constitute teamwork remains unknown.

Optimal team functioning during crises is commonly taught under the label of “crisis resource management” (8, 9). Initially developed in the aviation industry (10), crisis resource management has been used to train practitioners across medical specialties (8, 1118) using a range of educational modalities including (1) formal didactics (e.g., lecture) (13, 19, 20), (2) demonstration (e.g., behavior modeling) (13, 20), and (3) experiential learning (e.g., simulation) (21). Increased focus on patient safety (22) and work-hours regulations for residents (23) have made simulation training an attractive adjunct to traditional didactic and clinical learning (22, 2427). Perceived advantages of simulation include linking teamwork skills to the clinical context in which they are needed, addressing specific scenarios known to challenge teamwork, and cross-training (28).

However, a simulation-based curriculum is expensive and time-intensive and requires a specialized environment and staff (29). Demonstration-based methods may share some of the advantages of simulation with lower effort and expense. No study has directly compared didactic, demonstration-based, and simulation-based methods for teaching residents teamwork during emergencies. We designed a randomized controlled trial comparing the effect of didactic, demonstration-based, and simulation-based methods of team training on teamwork behavior outcomes. We hypothesized that demonstration and simulation-based training would both be superior to traditional didactic instruction.

Methods

Study Oversight

We conducted a single-center, observer-blinded randomized controlled trial of three teamwork training methods. This study was approved by the Institutional Review Board with an exemption and waiver of informed consent. Per simulation facility protocol, participants provided written informed consent for videotaping independent of study participation.

Location and Participants

The study was conducted in the Center for Experiential Learning and Assessment facility at Vanderbilt University using two traditional classrooms and three simulation suites. All simulation scenarios were conducted in a simulated intensive care unit room with an experienced nurse confederate, blinded to the study, using the Laerdal SimMan high-fidelity mannequin run by dedicated, trained simulation technicians under course-faculty supervision. As part of an established, single-day, simulation-based orientation, all incoming internal medicine interns were enrolled in this study and participated in one of three randomly assigned teamwork training interventions followed by a standardized simulation scenario during which teamwork behaviors were evaluated (evaluation simulation). Before study participation, all interns had successfully completed a standardized Advanced Cardiac Life Support (ACLS) course including low-fidelity code simulation but were presumed to have no specific background in crisis resource management beyond that provided by typical undergraduate medical education.

Teamwork Training Interventions

At the start of the study day, all participants were oriented to the facility and informed that they would undergo a training exercise intended to develop teamwork skills followed by an evaluation of their teamwork behaviors. Participants were randomized in a 1:1:1 ratio to one of the three teamwork training modalities: (1) didactics: viewing a slide presentation on teamwork skills; (2) demonstration: watching a video of experts modeling teamwork skills in a simulated case; or (3) simulation: participating in simulated patient scenario using a high-fidelity mannequin, without formal instruction before or after the case (Figure 1). Beyond the assigned training intervention, no additional education or feedback regarding teamwork was provided to any of the three groups before evaluation.

Figure 1.

Figure 1.

Recruitment, randomization, assessment, and analysis.

Didactic teamwork training

Participants randomized to the didactic group viewed a 14-minute narrated slideshow outlining teamwork principles (see Video E1 in the online supplement). The presentation covered healthcare teamwork principles generally and specifically emphasized elements included in the Teamwork Behavioral Rater score, such as identification of a leader and role assignment, evaluation of the environment, prioritization and implementation of tasks, effective communication, use of resources, awareness of available information, anticipation, and adaptable planning (30, 31).

Demonstration-based teamwork training

Participants randomized to the demonstration-based group viewed a 12-minute digital video showing four pulmonary and critical care medicine faculty managing a simulated emergency. The critical care faculty modeled the same teamwork elements included in the didactic team training. The video was filmed within the Center for Experiential Learning and Assessment using a different simulation case than was later used to evaluate participants’ teamwork behavior (Video E2).

Simulation-based teamwork training

Participants randomized to the simulation-based group were divided into teams of three or four participants and were allowed to practice their teamwork skills using one of four previously developed, standardized, 12-minute critical care simulation scenarios. All four scenarios differed from the simulation later used to evaluate participants’ teamwork behavior. Those randomized to simulation training were informed that the purpose of the simulation exercise was to learn teamwork behavior but received no formal instruction in teamwork principles before the simulation, no guidance or coaching during the simulation, and no debriefing after the simulation.

Evaluation simulation

Within 3 hours of finishing the assigned teamwork training intervention, all participants completed an evaluation simulation (Figure 1). Participants who had all received the same teamwork training intervention were redistributed into groups of three or four and participated in a standardized simulation scenario (discrete from the scenarios used for training) that was videotaped for scoring. During the simulation, a confederate acting as a bedside nurse provided the initial history and remained available to obtain vital signs, apply oxygen, administer medications, and operate the crash cart. Confederates were unaware of the study, blinded to group assignment, and redistributed between the simulation-based teamwork training and evaluation simulation to minimize participant familiarity with the confederate. The simulation scenario highlighted components of the 2005 American Heart Association (AHA) ACLS treatment algorithms (32). The scenario required participants to adequately manage each key clinical issue before progressing to the next event. The simulation ended at 12 minutes regardless of the management provided or clinical state of the simulated patient.

To ensure uniform exposure to crisis resource management for all interns taking part in the training day, after the evaluation simulation all 52 interns completed a standardized, nonstudy simulation explicitly emphasizing teamwork competencies over task work and using expert faculty to guide hands-on practice and debrief in small groups with video review (33). At the end of the study day, all interns completed a learner satisfaction survey.

Outcome Measures

Five pulmonary and critical care medicine faculty with previous training in simulation and teamwork assessment independently scored each group’s videotaped evaluation simulation. Raters were blinded to study group allocation and scored participants on teamwork behavior and clinical skills using three independent rating tools, one for teamwork behavior and two for clinical performance.

Teamwork behavior

Teamwork behavior was assessed using the Teamwork Behavioral Rater, a previously validated tool for the measurement of healthcare team performance (30, 31). The tool contains 20 individual behaviors, which subcategorize into the domains “Leadership and Team Coordination,” “Verbalizing Situational Information,” and “Mutual Performance Monitoring” (Table E1). Each item is rated on a scale from 1 (worst) to 7 (best). In preparation for this study, investigators developed an objective list of behaviors that would correlate with each score on the scale and calibrated the tool via review of practice simulation scenarios. Adequate agreement between raters could not be achieved on a single item, and this item was excluded from the final scoring system before study initiation (Table E1). The primary outcome for this study was the average overall Teamwork Behavioral Rater score, and secondary outcomes included the average score for each of the three domains, each individual item, and clinical performance.

Clinical performance

Clinical performance was evaluated by two metrics: (1) adherence to the 2005 AHA ACLS algorithms; and (2) a novel “Recognize, Respond, Reassess” scoring system developed for this study. The 2005 AHA ACLS treatment algorithms were operationalized into a 30-item checklist in which each completed action received one point (Table E2). The Recognize, Respond, Reassess scoring system was developed to evaluate the sequential components of providers’ clinical management in a manner independent of the clinical details of the specific simulation scenario. Providers were evaluated on their ability to (1) recognize aberrations in clinical status (e.g., hypoxia), (2) respond appropriately (e.g., administering supplemental oxygen), and (3) reassess to ensure improvement with an intervention (e.g., confirming improved saturation). One point each was awarded for recognition, response, and reassessment. In our evaluation simulation, 13 clinical elements were evaluated on the Recognize, Respond, Reassess score for a maximum possible score of 39 points (Table E3).

Statistical Analysis

Based on a prior study of simulation-based evaluation using the Teamwork Behavioral Rater scoring system (31), planned enrollment of 50 participants divided into 15 groups provided 80% statistical power to detect a difference of 1.5 points in the average teamwork score with a type I error rate of 0.05. Demographics and baseline characteristics of participants were summarized by mean and SD for continuous variables and numbers and percentages for categorical variables. The primary outcome of average overall Teamwork Behavioral Rater score was compared across groups randomized to each of the three teamwork training interventions using the Kruskal-Wallis test and between each of the groups individually using the Mann-Whitney U test. Secondary outcomes of average score in each domain and scores for individual items were compared between the three teamwork training interventions using the Kruskal-Wallis test and between each intervention individually using the Mann-Whitney U test. Given multiple raters and a continuous outcome measure, correlation between the raters for the Teamwork Behavioral Rater, AHA ACLS checklist, and Recognize, Respond, Reassess scoring system was evaluated by Spearman correlation. All analyses were performed using SPSS Statistics v.22 (IBM Corp., Armonk, NY), and a two-sided P value < 0.05 was used to determine significance.

Results

On June 30, 2010, 52 incoming internal medicine interns were enrolled and randomized to receive either didactic (n = 18), demonstration-based (n = 17), or simulation-based (n = 17) teamwork training before completing a simulation in which teamwork behavior was evaluated (evaluation simulation). The three groups had similar baseline demographic and educational characteristics (Table 1).

Table 1.

Baseline characteristics of participants by randomized study group

 
Didactic
Demonstration
Simulation
Baseline Characteristics (n = 18) (n = 17) (n = 17)
Age, yr 27.8 ± 2.8 27.2 ± 2.0 28.0 ± 1.8
Men 7 (38.9) 11 (64.7) 10 (58.8)
Race      
 White 10 (55.6) 14 (82.4) 12 (70.6)
 African American 4 (22.2) 1 (5.9) 3 (17.6)
 Asian 4 (22.2) 2 (11.8) 2 (11.8)
Medical school location      
 Northeast 2 (11.1) 1 (5.9) 4 (23.5)
 Southeast 9 (50.0) 11 (64.7) 9 (52.9)
 Midwest 3 (16.7) 3 (17.6) 1 (5.9)
 West coast 2 (11.1) 1 (5.9) 2 (11.8)
 Canada or Europe 2 (11.1) 1 (5.9) 1 (5.9)
Medical school rank* 43 ± 25 47 ± 33 53 ± 38
Alpha Omega Alpha 6 (33.3) 8 (47.1) 4 (23.5)
USMLE score      
 Step 1 235 ± 19 239 ± 14 237 ± 17
 Step 2 244 ± 18 249 ± 16 248 ± 14
Subspecialty interest      
 Procedural 5 (27.8) 4 (23.5) 4 (23.5)
 Nonprocedural 6 (33.3) 7 (41.2) 4 (23.5)
 Undecided 5 (27.8) 4 (23.5) 5 (29.4)
 Preliminary Internship 2 (11.1) 2 (11.8) 4 (23.5)

Definition of abbreviation: USMLE = U.S. Medical Licensing Examination.

Values are presented as mean ± SD or n (%).

*

U.S. News and World Report 2014 ranking for research.

Self-described subspecialty interest at the time of study conduct. Cardiology, gastroenterology, and pulmonary and critical care medicine classified as “procedural”; all others classified as “nonprocedural.”

Regarding the primary outcome, the average overall Teamwork Behavioral Rater score was 3.10 ± 0.51 in the didactic group, 4.40 ± 1.15 in the demonstration group, and 4.10 ± 0.95 in the simulation group (Table 2, Figure 2A). Overall scores were significantly higher in the demonstration group compared with the didactic group (mean difference, 1.30; 95% confidence interval [CI], 0.13–2.47; P = 0.045) but similar between demonstration and simulation (mean difference, 0.31; 95% CI, −1.23 to 1.84; P = 0.917). Simulation demonstrated a trend toward better scores than didactic training (mean difference, 0.99; 95% CI, 2.00 to −0.02; P = 0.068). The mean score for the demonstration group was numerically higher than the didactic group for all 20 items composing the Teamwork Behavioral Rater and higher than the simulation group for 16 of the 20 (Table 2). Significantly higher scores were seen for the demonstration group compared with the didactic group for individual items in all three domains (Figure 2B). There was no difference between the three groups in clinical performance by either the AHA ACLS treatment algorithm checklist or the novel Recognize, Respond, Reassess score (Table 2), and teamwork behavior only weakly correlated with clinical performance (Figure 3). On the post-study survey, there were no differences in learner satisfaction between didactics, demonstration, and simulation (Table E4).

Table 2.

Teamwork Behavioral Rater and clinical performance scores by training modality

  Didactic Demonstration Simulation P Value
Overall Teamwork Behavioral Rater score 3.10 ± 0.51 4.40 ± 1.15 4.10 ± 0.95 0.074
Leadership and team coordination 2.96 ± 0.56 4.32 ± 1.34 4.13 ± 1.16 0.057
 A leader was clearly established 2.50 ± 0.91 3.92 ± 1.99 3.64 ± 1.68 0.280
 The leader's plan for treatment was communicated to the team 2.63 ± 0.70 4.08 ± 1.63 3.84 ± 1.18 0.204
 Priorities and orders of actions were communicated to the team 2.53 ± 0.69 3.84 ± 1.18 3.52 ± 1.13 0.183
 The team leader showed an appropriate balance between authority and openness to suggestion 3.28 ± 0.96 4.84 ± 1.18 4.74 ± 1.10 0.067
 The team leader was able to maintain an overview of the situation 2.83 ± 0.81 3.80 ± 1.60 4.32 ± 1.29 0.204
 Plans were adapted when the situation changed 3.80 ± 0.40 4.56 ± 1.47 4.72 ± 0.79 0.213
 Each team member had a clear role 2.47 ± 0.56 3.92 ± 1.80 3.68 ± 1.26 0.080
 Instructions and verbal communication were explicit and directed 2.43 ± 0.37 4.32 ± 1.59 3.68 ± 1.43 0.085
 Task implementation was well coordinated 2.83 ± 0.55 4.48 ± 1.70 4.04 ± 1.28 0.147
 Disagreements or conflict impaired team performance 4.30 ± 0.79 5.40 ± 0.62 5.08 ± 1.15 0.132
Mutual performance monitoring 3.78 ± 0.92 5.24 ± 0.90 4.83 ± 0.74 0.065
 The team sourced external assistance when appropriate 3.23 ± 1.59 5.84 ± 0.48 4.68 ± 1.21 0.013
 Team members called attention to potentially hazardous actions or omissions 3.73 ± 1.01 4.74 ± 1.35 4.48 ± 0.86 0.262
 Individual team members reacted appropriately when other team members pointed out their potential errors or mistakes 4.37 ± 0.69 5.13 ± 1.20 5.32 ± 0.74 0.208
Verbalizing situational information 2.99 ± 0.53 4.12 ± 1.06 3.67 ± 1.06 0.133
 Team members repeated back or paraphrased instructions and clarifications 1.73 ± 0.41 3.08 ± 1.15 2.60 ± 1.12 0.064
 When directions were unclear team members asked for repetition and clarification 3.24 ± 0.76 3.90 ± 0.91 3.53 ± 1.05 0.611
 Team members shared situation assessment information 4.03 ± 0.427 5.48 ± 1.12 4.44 ± 0.74 0.043
 Team members asked each other for assistance before or during periods of task overload 3.10 ± 0.51 4.40 ± 1.36 4.24 ± 1.32 0.130
 Team members offered assistance when other team members became task overloaded 3.33 ± 1.08 4.25 ± 0.92 3.75 ± 1.49 0.498
 Team members verbalized important clinical interventions 2.50 ± 0.37 3.60 ± 1.17 3.44 ± 1.04 0.129
Clinical performance        
 AHA ACLS Clinical Checklist score 19.0 ± 2.6 20.5 ± 4.5 19.4 ± 1.9 0.692
 Recognize, Respond, Reassess score 21.7 ± 3.3 23.3 ± 4.9 22.7 ± 2.7 0.883

Definition of abbreviation: AHA ACLS = American Heart Association Advanced Cardiovascular Life Support.

Values are presented as mean ± SD.

Figure 2.

Figure 2.

Teamwork Behavioral Rater (TBR) scores by study group. (A) Average overall TBR score (median and 95% confidence interval) by teamwork training modality. Each dot represents the score awarded by one of the five raters to one of the 16 groups in the evaluation simulation. (B) Mean differences in score favored demonstration (squares) over didactic training and showed a trend in favor of simulation (triangles) over didactic training overall and within the domains of Leadership and Team Coordination, Mutual Performance Monitoring, and Verbalizing Situational Information.

Figure 3.

Figure 3.

Correlation between performance scores. (A) There was a strong correlation between the novel Recognize, Respond, Reassess (3R) clinical performance score and completion of identified items on the American Heart Association Advanced Cardiovascular Life Support (AHA ACLS) checklist. (B) Teamwork Behavioral Rater (TBR) scores weakly correlated with clinical performance. Each dot represents the scores awarded by one of the five raters to one of the 16 groups in the evaluation simulation.

The correlation between the five raters in average overall teamwork score for the 16 groups showed a median Spearman's correlation coefficient (Rs) of 0.622 (range, 0.434–0.789; P values, 0.001–0.093) and median coefficient of determination (Rs2) of 0.387. Clinical performance scores using the novel Recognize, Respond, Reassess tool correlated strongly with the AHA ACLS checklist (rs = 0.822, Rs2 = 0.675, P < 0.001) (Figure 3).

Discussion

In this randomized controlled trial of three teamwork training modalities, demonstration-based training was superior to traditional didactics and similar to simulation in its influence on teamwork behavior among incoming internal medicine interns managing a simulated patient crisis. Simulation training showed a trend toward better teamwork than traditional didactics alone. The impact on teamwork was consistent across teamwork domains and was largely independent of learner satisfaction and clinical performance.

Despite the growing consensus that teaching teamwork skills is an important part of training future healthcare professionals (34), evidence supporting the best method of training healthcare teams is limited. The majority of prior trials compare a single modality of team training to no teamwork training (20, 3537). The few available randomized controlled trials directly comparing team training modalities have shown conflicting results. In a trial involving doctors, nurses, and midwives, Clay-Williams and colleagues (38) found that team training involving didactics and video vignettes was superior to simulation in improving objectively measured teamwork behaviors. Similarly, Shapiro and colleagues (39) found that, among a small group of emergency department physicians and nurses, the addition of simulation-based team training to didactic and demonstration-based methods did not significantly improve teamwork.

In contrast, when Riley and colleagues (40) randomized one hospital to control, one hospital to in-classroom didactic and demonstration-based training through the TeamSTEPPS program, and one hospital to both TeamSTEPPS and in situ simulation training, they observed decreased perinatal morbidity at the hospital exposed to simulation. Similarly, when Daniels and colleagues (18) randomized nurses and obstetric residents to lecture, videos, and low-fidelity simulation addressing the clinical aspects of eclampsia management versus high-fidelity simulation including education in crisis resource management principles, the group that received crisis resource management as a part of simulation training subsequently demonstrated better teamwork.

Given the limited and conflicting prior data, our study adds insight into the best methods by which to train healthcare professionals in teamwork. Watching critical care experts model teamwork behavior in a videotaped simulation case was similarly effective to participating in a simulation and more effective than being presented teamwork principles by traditional didactic slide presentation. These results reaffirm that formal didactics alone are inferior for teaching nontechnical skills (41) but also highlight the wide educational spectrum between traditional didactics and expensive, expert-dependent, time-intensive high-fidelity simulation. All of the interventions in this study, including the simulation arm, took less than 15 minutes on a single day to complete and occurred with minimal real-time input from expert faculty. Despite this low resource-intensity design, both the demonstration and simulation interventions appeared to measurably improve teamwork skills.

As evidence in teamwork education progresses, we may be better able to discriminate settings in which the costs of a comprehensive, high-fidelity simulation curriculum (4247) are justified; settings in which limited, low-fidelity simulation is adequately effective (4244, 4851); and settings in which demonstration-based team training via in situ expert modeling, locally developed video scenarios, or online videos may be the most effective way to educate trainees about teamwork. Many high-quality demonstration-based teaching tools have been developed to convey procedural skills or medical knowledge to trainees (52), and focusing on the development of similar high-quality tools for teaching crisis resource management might be an effective and cost-efficient way of improving healthcare teamwork.

In addition to the above findings relating teamwork training methods to behavioral outcomes, our study posits the novel Recognize, Respond, Reassess scoring system as an available measure of clinical performance during acute care simulation. The Recognize, Respond, Reassess score correlated highly with adherence to AHA ACLS guidelines. If validated in future studies, it may offer a rubric that can be translated across cases and studies independent of scenario clinical content.

Our study has several strengths. It is the first randomized trial to compare head-to-head the three established modalities for team training. Unlike prior studies, which frequently combined teaching modalities within a single arm or examined the additive effect of one modality to others, the teamwork training interventions in our study were clearly delineated. Teamwork behavior was evaluated by independent, trained, blinded observers using a measure of teamwork behavior previously validated in the context of acute care simulation (30, 31). With 52 participants, the study was powered to show differences in teamwork behavior in the range believed to be clinically meaningful (31). The overall Teamwork Behavioral Rater scores and difference in scores between groups in our study were similar to those achieved in a prior study of teamwork targeting experienced critical care physicians and nurses (31). Given that all study participants were ACLS certified and that each of the three interventions in our trial focused on teaching teamwork principles and not clinical management, the clinical performance scores offer a “negative control.” The finding of similar clinical performance between all groups reinforces that the differences in teamwork behavior were attributable to the interventions themselves rather than baseline differences in the groups or differences in familiarity with the simulation environment.

Our study also has several limitations. The trial was conducted at a single center with a homogenous group of learners at a single point in training and may not generalize to other settings. The intervention within each modality was designed to be representative but not necessarily superlative—use of different teaching approaches within a given modality may have yielded different results. The didactic arm may have fared better with a more engaging narration, a simpler message focused on key concepts, less text and more figures, repetition of key principles, listener participation, or other techniques to encourage active learning. The demonstration intervention might have been more impactful had it not simply presented a simulated scenario from start to finish but used shorter segments to contrast examples of “good” and “bad” teamwork behavior, repeated video segments to highlight key learning points, used graphics to overtly identify teamwork behaviors, and allowed time for audience discussion. The brief, minimally guided simulation intervention might have demonstrated even better outcomes with incorporation of elements such as repetitive practice, integration into a longitudinal curriculum, progressive levels of difficulty, and structured feedback, which have been highlighted in prior simulation studies (53, 54). Feedback, and in particular structured debriefing, has increasingly been identified as a potentially important component of effective simulation-based education (33, 53, 55, 56). Studies adding structured debriefing to simulation have demonstrated improved teamwork behavior (48, 5760), although the exact additive value of debriefing to high-fidelity simulation remains controversial (21, 61). Debriefing has similarly been successfully incorporated into didactic and demonstration-based teamwork training programs (13). Whether addition of structured debriefing to any or all of the educational interventions tested would have changed the effect on acquired teamwork skills is unknown. Similarly, many prior studies have combined modalities, mixing didactics, demonstration, and simulation. Our trial was rigidly structured to isolate, as much as possible, the impact of each educational modality by itself on teamwork behaviors and cannot inform the impact of combined approaches. Whether the differences in teamwork behavior seen in our evaluation simulation immediately after the training intervention persist over time, translate to the clinical environment, or influence patient outcomes remains unknown.

Our findings do not minimize the potential role for either traditional didactics or simulation in internal medicine and critical care training. However, they do extend the imperative within medical education to understand which approaches are most effective. With increased focus on work hour regulations and costs, future studies in teamwork training must go beyond simply translating techniques that were effective for air travel or the military and use rigorous scientific methods to understand what is both effective and cost-efficient specifically in medicine.

Conclusions

Expert demonstration appears similar to simulation and superior to didactics for teaching incoming interns teamwork skills. Clinical performance was largely independent of teamwork behavior and did not differ between training modalities.

Acknowledgments

Acknowledgment

The authors thank John Sergent, M.D., and the staff at the Center for Experiential Learning and Assessment facility at Vanderbilt University Medical Center for their contributions to making this study possible.

Footnotes

Author Contributions: M.W.S., R.D.K., T.W.R., and J.A.B. had full access to all the data and had final responsibility for the decision to submit for publication. J.K.C., M.E.P., A.N.M., A.P.W., T.W.R., and J.A.B.: study concept and design. J.K.C., M.E.P., A.N.M., K.T., A.B., A.P.W., T.W.R., and J.A.B.: acquisition of the data. M.W.S., R.D.K., J.K.C., M.E.P., K.T., A.B., T.W.R., and J.A.B.: data analysis and interpretation. M.W.S., R.D.K., M.E.P, and T.W.R.: manuscript preparation and drafting. M.W.S., R.D.K., and T.W.R.: statistical methods, statistical data analysis. J.K.C., M.E.P., A.N.M., K.T., A.B., A.P.W., T.W.R., and J.A.B.: manuscript critique and review. All authors approved the manuscript submitted.

This article has a data supplement, which is accessible from this issue's table of contents online at www.atsjournals.org

Author disclosures are available with the text of this article at www.atsjournals.org.

References

  • 1.Reader T, Flin R, Lauche K, Cuthbertson BH. Non-technical skills in the intensive care unit. Br J Anaesth. 2006;96:551–559. doi: 10.1093/bja/ael067. [DOI] [PubMed] [Google Scholar]
  • 2.Donchin Y, Gopher D, Olin M, Badihi Y, Biesky M, Sprung CL, Pizov R, Cotev S. A look into the nature and causes of human errors in the intensive care unit. Qual Saf Health Care. 2003;12:143–147. [Discussion pp. 147–148.]. doi: 10.1136/qhc.12.2.143. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Alvarez G, Coiera E. Interdisciplinary communication: an uncharted source of medical error? J Crit Care. 2006;21:236–242. [Discussion p. 242.]. doi: 10.1016/j.jcrc.2006.02.004. [DOI] [PubMed] [Google Scholar]
  • 4.Puntillo KA, McAdam JL. Communication between physicians and nurses as a target for improving end-of-life care in the intensive care unit: challenges and opportunities for moving forward. Crit Care Med. 2006;34:S332–S340. doi: 10.1097/01.CCM.0000237047.31376.28. [DOI] [PubMed] [Google Scholar]
  • 5.Pronovost P, Berenholtz S, Dorman T, Lipsett PA, Simmonds T, Haraden C. Improving communication in the ICU using daily goals. J Crit Care. 2003;18:71–75. doi: 10.1053/jcrc.2003.50008. [DOI] [PubMed] [Google Scholar]
  • 6.Reader TW, Flin R, Cuthbertson BH. Communication skills and error in the intensive care unit. Curr Opin Crit Care. 2007;13:732–736. doi: 10.1097/MCC.0b013e3282f1bb0e. [DOI] [PubMed] [Google Scholar]
  • 7.Manser T. Teamwork and patient safety in dynamic domains of healthcare: a review of the literature. Acta Anaesthesiol Scand. 2009;53:143–151. doi: 10.1111/j.1399-6576.2008.01717.x. [DOI] [PubMed] [Google Scholar]
  • 8.Howard SK, Gaba DM, Fish KJ, Yang G, Sarnquist FH. Anesthesia crisis resource management training: teaching anesthesiologists to handle critical incidents. Aviat Space Environ Med. 1992;63:763–770. [PubMed] [Google Scholar]
  • 9.Weaver SJ, Dy SM, Rosen MA. Team-training in healthcare: a narrative synthesis of the literature. BMJ Qual Saf. 2014;23:359–372. doi: 10.1136/bmjqs-2013-001848. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Helmreich RL, Merritt AC, Wilhelm JA. The evolution of Crew Resource Management training in commercial aviation. Int J Aviat Psychol. 1999;9:19–32. doi: 10.1207/s15327108ijap0901_2. [DOI] [PubMed] [Google Scholar]
  • 11.Gaba DM, Howard SK, Fish KJ, Smith BE, Sowb YA. Simulation-based training in anesthesia crisis resource management (ACRM): a decade of experience. Simul Gaming. 2001;32:175–193. [Google Scholar]
  • 12.Gaba DM. Crisis resource management and teamwork training in anaesthesia. Br J Anaesth. 2010;105:3–6. doi: 10.1093/bja/aeq124. [DOI] [PubMed] [Google Scholar]
  • 13.Neily J, Mills PD, Young-Xu Y, Carney BT, West P, Berger DH, Mazzia LM, Paull DE, Bagian JP. Association between implementation of a medical team training program and surgical mortality. JAMA. 2010;304:1693–1700. doi: 10.1001/jama.2010.1506. [DOI] [PubMed] [Google Scholar]
  • 14.Carne B, Kennedy M, Gray T. Review article: crisis resource management in emergency medicine. Emerg Med Australas. 2012;24:7–13. doi: 10.1111/j.1742-6723.2011.01495.x. [DOI] [PubMed] [Google Scholar]
  • 15.Kim J, Neilipovitz D, Cardinal P, Chiu M, Clinch J. A pilot study using high-fidelity simulation to formally evaluate performance in the resuscitation of critically ill patients: the University of Ottawa Critical Care Medicine, High-Fidelity Simulation, and Crisis Resource Management I Study. Crit Care Med. 2006;34:2167–2174. doi: 10.1097/01.CCM.0000229877.45125.CC. [DOI] [PubMed] [Google Scholar]
  • 16.Sarwani N, Tappouni R, Flemming D. Use of a simulation laboratory to train radiology residents in the management of acute radiologic emergencies. AJR Am J Roentgenol. 2012;199:244–251. doi: 10.2214/AJR.11.7892. [DOI] [PubMed] [Google Scholar]
  • 17.Andreatta P, Saxton E, Thompson M, Annich G. Simulation-based mock codes significantly correlate with improved pediatric patient cardiopulmonary arrest survival rates. Pediatr Crit Care Med. 2011;12:33–38. doi: 10.1097/PCC.0b013e3181e89270. [DOI] [PubMed] [Google Scholar]
  • 18.Daniels K, Arafeh J, Clark A, Waller S, Druzin M, Chueh J. Prospective randomized trial of simulation versus didactic teaching for obstetrical emergencies. Simul Healthc. 2010;5:40–45. doi: 10.1097/SIH.0b013e3181b65f22. [DOI] [PubMed] [Google Scholar]
  • 19.King HB, Battles J, Baker DP, Alonso A, Salas E, Webster J, Toomey L, Salisbury M.TeamSTEPPS: team strategies and tools to enhance performance and patient safety Henriksen K, Battles JB, Keyes MA, Grady ML.Advances in patient safety: new directions and alternative approaches. Vol. 3: Performance and tools. Advances in Patient Safety Rockville (MD)Agency for Healthcare Research and Quality; 2008[accessed 2014 Dec 11]. Available from: http://www.ncbi.nlm.nih.gov/books/NBK43686/ [Google Scholar]
  • 20.Morey JC, Simon R, Jay GD, Wears RL, Salisbury M, Dukes KA, Berns SD. Error reduction and performance improvement in the emergency department through formal teamwork training: evaluation results of the MedTeams project. Health Serv Res. 2002;37:1553–1581. doi: 10.1111/1475-6773.01104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Cook DA, Hatala R, Brydges R, Zendejas B, Szostek JH, Wang AT, Erwin PJ, Hamstra SJ. Technology-enhanced simulation for health professions education: a systematic review and meta-analysis. JAMA. 2011;306:978–988. doi: 10.1001/jama.2011.1234. [DOI] [PubMed] [Google Scholar]
  • 22.Satava RM. The revolution in medical education—the role of simulation. J Grad Med Educ. 2009;1:172–175. doi: 10.4300/JGME-D-09-00075.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Nasca TJ, Day SH, Amis ES. The new recommendations on duty hours from the ACGME Task Force. N Engl J Med. 2010;363:e3. doi: 10.1056/NEJMsb1005800. [DOI] [PubMed] [Google Scholar]
  • 24.Okuda Y, Bryson EO, DeMaria S, Jacobson L, Quinones J, Shen B, Levine AI. The utility of simulation in medical education: what is the evidence? Mt Sinai J Med. 2009;76:330–343. doi: 10.1002/msj.20127. [DOI] [PubMed] [Google Scholar]
  • 25.Issenberg SB, McGaghie WC, Hart IR, Mayer JW, Felner JM, Petrusa ER, Waugh RA, Brown DD, Safford RR, Gessner IH, et al. Simulation technology for health care professional skills training and assessment. JAMA. 1999;282:861–866. doi: 10.1001/jama.282.9.861. [DOI] [PubMed] [Google Scholar]
  • 26.Greenberg SB, Tokarczyk A, Small S. Critical care simulation. Dis Mon. 2011;57:715–722. doi: 10.1016/j.disamonth.2011.08.010. [DOI] [PubMed] [Google Scholar]
  • 27.Grenvik A, Schaefer JJ, III, DeVita MA, Rogers P. New aspects on critical care medicine training. Curr Opin Crit Care. 2004;10:233–237. doi: 10.1097/01.ccx.0000132654.52131.32. [DOI] [PubMed] [Google Scholar]
  • 28.Gaba DM.What does simulation add to teamwork training? Agency for Healthcare Research and Quality 2006[accessed 2014 Dec 11]. Available from: http://www.webmm.ahrq.gov/perspective.aspx?perspectiveID=20
  • 29.Pratt SD, Sachs BP.Team training: classroom training vs. high-fidelity simulationAgency for Healthcare Research and Quality. 2006[accessed 2014 Dec 11]. Available from: http://www.webmm.ahrq.gov/perspective.aspx?perspectiveID=21
  • 30.Weller J, Frengley R, Torrie J, Shulruf B, Jolly B, Hopley L, Hendersdon K, Dzendrowskyj P, Yee B, Paul A. Evaluation of an instrument to measure teamwork in multidisciplinary critical care teams. BMJ Qual Saf. 2011;20:216–222. doi: 10.1136/bmjqs.2010.041913. [DOI] [PubMed] [Google Scholar]
  • 31.Frengley RW, Weller JM, Torrie J, Dzendrowskyj P, Yee B, Paul AM, Shulruf B, Henderson KM. The effect of a simulation-based training intervention on the performance of established critical care unit teams. Crit Care Med. 2011;39:2605–2611. doi: 10.1097/CCM.0b013e3182282a98. [DOI] [PubMed] [Google Scholar]
  • 32.Committee ECC Subcommittees and Task Forces of the American Heart Association. 2005 American Heart Association guidelines for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation. 2005;112:IV1–IV203. doi: 10.1161/CIRCULATIONAHA.105.166550. [DOI] [PubMed] [Google Scholar]
  • 33.McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. A critical review of simulation-based medical education research: 2003–2009. Med Educ. 2010;44:50–63. doi: 10.1111/j.1365-2923.2009.03547.x. [DOI] [PubMed] [Google Scholar]
  • 34.Institute of Medicine (US) Committee on Quality of Health Care in America To Err Is Human: Building a Safer Health System Kohn LT, Corrigan JM, Donaldson MS.Washington, DC: National Academies Press; 2000[accessed 2014 Dec 11]. Available from: http://www.ncbi.nlm.nih.gov/books/NBK225182/ [PubMed] [Google Scholar]
  • 35.Fernandez Castelao E, Russo SG, Cremer S, Strack M, Kaminski L, Eich C, Timmermann A, Boos M. Positive impact of crisis resource management training on no-flow time and team member verbalisations during simulated cardiopulmonary resuscitation: a randomised controlled trial. Resuscitation. 2011;82:1338–1343. doi: 10.1016/j.resuscitation.2011.05.009. [DOI] [PubMed] [Google Scholar]
  • 36.Walker D, Cohen S, Fritz J, Olvera M, Lamadrid H, Carranza L. PRONTO low-tech obstetric simulation and team-training for obstetric and neonatal emergencies in Mexico leads to a decrease in cesarean delivery rates. Obstet Gynecol. 2014;123:177S. [Google Scholar]
  • 37.Nielsen PE, Goldman MB, Mann S, Shapiro DE, Marcus RG, Pratt SD, Greenberg P, McNamee P, Salisbury M, Birnbach DJ, et al. Effects of teamwork training on adverse outcomes and process of care in labor and delivery: a randomized controlled trial. Obstet Gynecol. 2007;109:48–55. doi: 10.1097/01.AOG.0000250900.53126.c2. [DOI] [PubMed] [Google Scholar]
  • 38.Clay-Williams R, McIntosh CA, Kerridge R, Braithwaite J. Classroom and simulation team training: a randomized controlled trial. Int J Qual Health Care. 2013;25:314–321. doi: 10.1093/intqhc/mzt027. [DOI] [PubMed] [Google Scholar]
  • 39.Shapiro MJ, Morey JC, Small SD, Langford V, Kaylor CJ, Jagminas L, Suner S, Salisbury ML, Simon R, Jay GD. Simulation based teamwork training for emergency department staff: does it improve clinical team performance when added to an existing didactic teamwork curriculum? Qual Saf Health Care. 2004;13:417–421. doi: 10.1136/qshc.2003.005447. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Riley W, Davis S, Miller K, Hansen H, Sainfort F, Sweet R. Didactic and simulation nontechnical skills team training to improve perinatal patient outcomes in a community hospital. Jt Comm J Qual Patient Saf. 2011;37:357–364. doi: 10.1016/s1553-7250(11)37046-8. [DOI] [PubMed] [Google Scholar]
  • 41.Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA. 1995;274:700–705. doi: 10.1001/jama.274.9.700. [DOI] [PubMed] [Google Scholar]
  • 42.Weinstock PH, Kappus LJ, Garden A, Burns JP. Simulation at the point of care: reduced-cost, in situ training via a mobile cart. Pediatr Crit Care Med. 2009;10:176–181. doi: 10.1097/PCC.0b013e3181956c6f. [DOI] [PubMed] [Google Scholar]
  • 43.Lapkin S, Levett-Jones T. A cost-utility analysis of medium vs. high-fidelity human patient simulation manikins in nursing education. J Clin Nurs. 2011;20:3543–3552. doi: 10.1111/j.1365-2702.2011.03843.x. [DOI] [PubMed] [Google Scholar]
  • 44.Leshikar DE, Pierce JL, Salcedo ES, Bola G, Galante JM. Do more with less: a surgery directed institutional model for resident central line training. Am J Surg. 2014;207:243–250. doi: 10.1016/j.amjsurg.2013.09.005. [DOI] [PubMed] [Google Scholar]
  • 45.Isaranuwatchai W, Brydges R, Carnahan H, Backstein D, Dubrowski A. Comparing the cost-effectiveness of simulation modalities: a case study of peripheral intravenous catheterization training. Adv Health Sci Educ Theory Pract. 2014;19:219–232. doi: 10.1007/s10459-013-9464-6. [DOI] [PubMed] [Google Scholar]
  • 46.Petscavage JM, Wang CL, Schopp JG, Paladin AM, Richardson ML, Bush WH. Cost analysis and feasibility of high-fidelity simulation based radiology contrast reaction curriculum. Acad Radiol. 2011;18:107–112. doi: 10.1016/j.acra.2010.08.014. [DOI] [PubMed] [Google Scholar]
  • 47.McIntosh C, Macario A, Flanagan B, Gaba DM. Simulation: what does it really cost? [abstract] Simul Healthc J. 2006;1:109–A1473. [Google Scholar]
  • 48.Welke TM, LeBlanc VR, Savoldelli GL, Joo HS, Chandra DB, Crabtree NA, Naik VN. Personalized oral debriefing versus standardized multimedia instruction after patient crisis simulation. Anesth Analg. 2009;109:183–189. doi: 10.1213/ane.0b013e3181a324ab. [DOI] [PubMed] [Google Scholar]
  • 49.Lois FJ, Pospiech AL, Van Dyck MJ, Kahn DA, De Kock MF. Is the “in situ” simulation for teaching anesthesia residents a lower cost, feasible and satisfying alternative to simulation center? A 24 months prospective observational study in a university hospital. Acta Anaesthesiol Belg. 2014;65:61–71. [PubMed] [Google Scholar]
  • 50.Heinrichs WL, Youngblood P, Harter PM, Dev P. Simulation for team training and assessment: case studies of online training with virtual worlds. World J Surg. 2008;32:161–170. doi: 10.1007/s00268-007-9354-2. [DOI] [PubMed] [Google Scholar]
  • 51.Hobgood C, Sherwood G, Frush K, Hollar D, Maynard L, Foster B, Sawning S, Woodyard D, Durham C, Wright M, et al. Interprofessional Patient Safety Education Collaborative. Teamwork training with nursing and medical students: does the method matter? Results of an interinstitutional, interdisciplinary collaboration. Qual Saf Health Care. 2010;19:e25. doi: 10.1136/qshc.2008.031732. [DOI] [PubMed] [Google Scholar]
  • 52.Ailon J, Mourad O, Chien V, Saun T, Dev SP.Ultrasound-guided insertion of a radial arterial catheter New Engl J Med[created 2014 Oct 9; accessed 2014 Dec 15]. Available from: http://www.nejm.org/doi/full/10.1056/NEJMvcm1213181 [DOI] [PubMed]
  • 53.Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005;27:10–28. doi: 10.1080/01421590500046924. [DOI] [PubMed] [Google Scholar]
  • 54.Weinger MB. The pharmacology of simulation: a conceptual framework to inform progress in simulation research. Simul Healthc. 2010;5:8–15. doi: 10.1097/SIH.0b013e3181c91d4a. [DOI] [PubMed] [Google Scholar]
  • 55.Cheng A, Eppich W, Grant V, Sherbino J, Zendejas B, Cook DA. Debriefing for technology-enhanced simulation: a systematic review and meta-analysis. Med Educ. 2014;48:657–666. doi: 10.1111/medu.12432. [DOI] [PubMed] [Google Scholar]
  • 56.Fanning RM, Gaba DM. The role of debriefing in simulation-based learning: Simul. Healthc. 2007;2:115–125. doi: 10.1097/SIH.0b013e3180315539. [DOI] [PubMed] [Google Scholar]
  • 57.Cheng A, Hunt EA, Donoghue A, Nelson-McMillan K, Nishisaki A, Leflore J, Eppich W, Moyer M, Brett-Fleegler M, Kleinman M, et al. EXPRESS Investigators. Examining pediatric resuscitation education using simulation and scripted debriefing: a multicenter randomized trial. JAMA Pediatr. 2013;167:528–536. doi: 10.1001/jamapediatrics.2013.1389. [DOI] [PubMed] [Google Scholar]
  • 58.Savoldelli GL, Naik VN, Park J, Joo HS, Chow R, Hamstra SJ. Value of debriefing during simulated crisis management: oral versus video-assisted oral feedback. Anesthesiology. 2006;105:279–285. doi: 10.1097/00000542-200608000-00010. [DOI] [PubMed] [Google Scholar]
  • 59.Morgan PJ, Tarshis J, LeBlanc V, Cleave-Hogg D, DeSousa S, Haley MF, Herold-McIlroy J, Law JA. Efficacy of high-fidelity simulation debriefing on the performance of practicing anaesthetists in simulated scenarios. Br J Anaesth. 2009;103:531–537. doi: 10.1093/bja/aep222. [DOI] [PubMed] [Google Scholar]
  • 60.Levett-Jones T, Lapkin S. A systematic review of the effectiveness of simulation debriefing in health professional education. Nurse Educ Today. 2014;34:e58–e63. doi: 10.1016/j.nedt.2013.09.020. [DOI] [PubMed] [Google Scholar]
  • 61.Tannenbaum SI, Cerasoli CP. Do team and individual debriefs enhance performance? A meta-analysis. Hum Factors. 2013;55:231–245. doi: 10.1177/0018720812448394. [DOI] [PubMed] [Google Scholar]

Articles from Annals of the American Thoracic Society are provided here courtesy of American Thoracic Society

RESOURCES