A Systematic Review of Teamwork Training Interventions in Medical Student and Resident Education

Abstract

Background

Teamwork is important for improving care across transitions between providers and for increasing patient safety.

Objective

This review’s objective was to assess the characteristics and efficacy of published curricula designed to teach teamwork to medical students and house staff.

Design

The authors searched MEDLINE, Education Resources Information Center, Excerpta Medica Database, PsychInfo, Cumulative Index of Nursing and Allied Health Literature, and Scopus for original data articles published in English between January 1980 and July 2006 that reported descriptions of teamwork training and evaluation results.

Measurements

Two reviewers independently abstracted information about curricular content (using Baker’s framework of teamwork competencies), educational methods, evaluation design, outcomes measured, and results.

Results

Thirteen studies met inclusion criteria. All curricula employed active learning methods; the majority (77%) included multidisciplinary training. Ten curricula (77%) used an uncontrolled pre/post design and 3 (23%) used controlled pre/post designs. Only 3 curricula (23%) reported outcomes beyond end of program, and only 1 (8%) >6weeks after program completion. One program evaluated a clinical outcome (patient satisfaction), which was unchanged after the intervention. The median effect size was 0.40 (interquartile range (IQR) 0.29, 0.61) for knowledge, 0.38 (IQR 0.32, 0.41) for attitudes, 0.41 (IQR 0.35, 0.49) for skills and behavior. The relationship between the number of teamwork principles taught and effect size achieved a Spearman’s correlation of .74 (p = .01) for overall effect size and .64 (p = .03) for median skills/behaviors effect size.

Conclusions

Reported curricula employ some sound educational principles and appear to be modestly effective in the short term. Curricula may be more effective when they address more teamwork principles.

BACKGROUND

Physicians have been trained in a culture that emphasizes and rewards individual accomplishments.¹ However, there is increasing recognition of the importance of teamwork in medical practice—from rapid response units, to surgical teams, to palliative care teams. Studies have noted associations between effective teamwork and improvements in quality of care,^2–4 and reductions in medical errors.^5–7 Teamwork in healthcare can be appreciated by patients, positively influences their satisfaction,^2,8 and augments provider satisfaction.⁹ Furthermore, collaborative professional relationships may decrease burnout among healthcare providers.¹⁰

Leading medical organizations endorse increased attention to teamwork. The Institute of Medicine has called for increased emphasis on teamwork in medical practice¹¹ and has proposed using teams to promote a culture of safety.^12,13 The Accreditation Council for Graduate Medical Education has included teamwork skills as part of its “Interpersonal and Communication Skill” core competency for house staff.¹⁴ Internal medicine residency program directors also advocate educational reforms that incorporate teamwork training.^15,16

OBJECTIVES

Previous reviews of teamwork training have not focused on physicians in training.^17–20 Hypothesizing that teamwork skills should be learned early in one’s professional development, we reviewed studies that described and evaluated educational interventions to teach teamwork skills to medical students and resident physicians. We sought to answer the following questions:

• What specific content was taught?

• Which teaching methods were used?

• How were the educational interventions evaluated?

• What was the efficacy of the interventions?

METHODS

Data Sources

We identified eligible articles through an electronic search of the following databases: PubMed, Education Resources Information Center, Excerpta Medica Database, PsychInfo, Cumulative Index of Nursing and Allied Health Literature, and Scopus. We also hand searched the reference lists of key review articles identified by the electronic search and the reference lists of all eligible articles (see Fig. 1). Searches were performed with the assistance of a medical librarian. To search PubMed, we combined the terms: [cooperation, interpersonal relationships, teamwork, or collaboration], with the terms: [medical education, internship, residency, or curriculum]. Medical Subject Headings were used when available. This strategy was adapted to search other databases. We initially selected articles that focused on teamwork by locating articles published in English between January 1980 and July 2006 with “team” in the title or abstract.

Figure 1.

Flowchart of search strategy. ERIC = Educational Resources Information Center; EMBASE = Excerpta Medica Database; CINAHL = Cumulative Index of Nursing and Allied Health Literature.

We abstracted articles that included medical students or house officers among the learners; described a curriculum involving teamwork training; and used either a pre/post evaluation design with or without controls, or a post-only design with a randomized control group. Articles were excluded if they did not contain original data (e.g., commentaries, editorials), or if the study did not report evaluation results (Fig. 1).

Data Extraction

After reviewing several published conceptual models of teamwork, we used the framework described by Baker, who described the knowledge, skills, and attitudes necessary to foster teamwork in medical education.²⁰ The principles within Baker’s competency framework have been used by other authors including Salas,²¹ Wilson,²² and Cannon-Bowers.²³ We transformed these principles into 8 items for the abstraction form and evaluated the articles for the presence or absence of these principles. We hypothesized that attention to these teamwork principles would be associated with curriculum effectiveness. Baker’s 8 teamwork principles include: (a) leadership training or an evaluation of leadership skills, (b) team members monitoring one another’s performance and providing feedback (mutual performance monitoring), (c) redistributing tasks upon demand by anticipating team member’s needs through accurate knowledge of their responsibilities (backup behavior), (d) ability to adapt to changing situations, (e) soliciting team member ideas in defining goals and objectives (team orientation), (f) trust, (g) communication training or an evaluation of communication skills, and (h) ensuring that team members are “on the same page” (shared mental models).

We also abstracted information from each curriculum on characteristics of the targeted learners, teaching methods, duration of intervention, evaluation design, outcomes measured, and evaluation results. We categorized the reported outcomes as knowledge, skill/behavior, attitude, or clinical.

Two reviewers (CC, RTB) extracted data using a data abstraction instrument. Kappa statistics were calculated to measure inter-rater reliability. In the case of disagreements, reviewers discussed the issue until consensus was reached.

Data Synthesis and Analysis

We used median effect size calculations to pool and compare heterogeneous educational outcomes.^24,25 Effect size, or standardized normal difference, is the difference between groups in a given measurement divided by the pooled standard deviation associated with that measurement.²⁶ We calculated effect sizes depending upon the study design reported in an article. For controlled curricula that reported pre/post results, we calculated the change in outcomes (post minus pre) of the intervention and control groups and divided this difference by the pooled standard deviation to find an effect size (Cohen’s d). For uncontrolled curricula that reported results before and after results, we divided the change in outcomes (post minus pre) by the standard deviation. To find the overall effect size for an article, we took the median value of all calculated effect sizes (median overall effect size). Thus for an article with 10 pre/post results, we calculated 10 effect sizes. The median of these 10 effect sizes became that article’s median overall effect size. The effect size is said to be small if Cohen’s d = .20, moderate if equal to .50, and large if equal to .80.²⁷

We assessed the association between each article’s median overall effect size and the number of teamwork principles described in the article using nonparametric Spearman correlations. We assessed the relationship between each individual teamwork principle and median overall effect sizes using the nonparametric Wilcoxon rank sum test. We performed subanalyses in which outcomes from each article were classified as knowledge, skill/behavior, or attitude. A similar method of calculating median effect size was used to obtain knowledge, skill/behavior, and attitude effect sizes for each article. We used Spearman correlations to assess relationships between median knowledge, skill/behavior, and attitude effect sizes and the number of teamwork principles described in each article. All data analyses were performed using Intercooled Stata 9.2 (STATA Corporation, College Station, TX, USA, 2006).

RESULTS

Of 1,120 citations that were identified, 13 studies fulfilled all criteria for inclusion and abstraction (Fig. 1). Inter-rater reliability was moderate (overall agreement = 81%, κ = .62) before consensus was reached through discussion. Consensus was reached in all cases. The most common areas requiring discussion were in assigning the principles of mutual performance monitoring, redistributing tasks according to need, and being “on the same page.”

Eleven (85%) of the 13 studies were published in 2004 and 2005.^28–38 The 13 articles described teaching a total of 3,137 learners. Learners were primarily internal medicine, anesthesiology, and emergency medicine residents (Table 1).

Table 1.

Curricular Characteristics of Teamwork Training Systematic Review Articles

Author (year)	Evaluation design	Duration of intervention	Learner characteristics (n)	Teaching methods described
Cooper H et al.33 (2005)	Single center, pre/post with control group	1-y course	2nd-year medical students, nursing and physical/occupational therapy students (237)	Lecture, case scenarios, nonmedical teambuilding exercises, facilitated reflection
DeVita MA et al.34 (2005)	Single center, pre/post, no control group	1-d course	Residents (anesthesia, internal medicine, emergency medicine), nurses, and respiratory therapists (138)	Lecture, mannequin simulation, facilitated reflection
Dienst ER and Byl N40 (1981)	Single center, pre/post, no control group	8-wk clerkship	4th-year medical students, nursing, pharmacy, and physical/occupational therapy students (116)	Patient encounters, debriefing, interspersed didactics
Epstein R et al.31 (2004)	Single center, pre/post, no control group	2-wk course	2nd-year medical students (98)	Mannequin simulation, role play, debriefing, facilitated reflection
Evans DV and Egnew TR39 (1997)	Multicenter, pre/post with control group	1-d workshop	Residents (family medicine) (78)	Nonmedical teambuilding exercises, debriefing
Featherstone P et al.35 (2005)	Multicenter, pre/post, no control group	1-d course	Physician specialties not described, nurses, and respiratory therapists (131)	Lecture, case scenarios
Fulmer T et al.32 (2005)	Multicenter, pre/post, no control group	9-mo course	Residents (anesthesia, internal medicine, emergency medicine), nursing, social work, physical/occupational therapy students, and administrators (537)	Case scenarios, video-based discussion, patient encounters
Grogan EL et al.28 (2004)	Single center, pre/post, no control group	1-d course	Residents (anesthesia, internal medicine, emergency medicine, surgery), nursing, administrators, ward personnel (489)	Lecture, role play in case scenarios, debriefing
Hope JM et al.36 (2005)	Single center, pre/post, no control group	11 3-h sessions over 1 y	Medical students (all years), nursing and physical/occupational therapy students, ward personnel (65)	Lecture, case scenarios, role play, facilitated reflection
Østergaard HT et al.29 (2004)	Multicenter, pre/post, no control group	1-d course	Residents (anesthesia, internal medicine, emergency medicine, pediatrics), nurses, ward personnel (234)	Lecture, video-based discussion, mannequin simulation, debriefing
Ponzer S et al.30 (2004)	Multicenter, pre/post, no control group	2-wk course	3rd-year medical students, nursing and physical/occupational therapy students (962)	Lecture, patient encounters, facilitated reflection
Shapiro MJ et al.37 (2004)	Single center, pre/post with control group	1-d course	Residents (emergency medicine), nurses (20)	Lecture, case scenarios, video-based discussions, mannequin simulation, debriefing
Stoller JK et al.38 (2004)	Single center, pre/post, no control group	1-d workshop	Residents (internal medicine) (32)	Lecture, nonmedical teambuilding exercises, debriefing

Content

Unless otherwise noted, all denominators for percentages are the 13 curricula included in this review (Table 2). The majority of the curricula (9, 69%) used a referenced teamwork model or group development framework with the crew/crisis resource management model being the most popular.^28,29,31,34 Nine curricula (69%) used a teamwork assessment instrument, 8 of which were previously validated.^{28,31–33,35–37,39}

Table 2.

Teamwork Principles Described, Model Used, and Overall Effect Size

Author (year)	Teamwork principles*	Teamwork model	Median overall effect size† [range]
Cooper H et al.33 (2005)	a, d, g	None described	0.32 [0.09, 0.56]
DeVita MA et al.34 (2005)	d, e, f, g	CRM‡	0.49 [0.23, 0.58]
Dienst ER and Byl N40 (1981)	a, d, e, g	None described	0.47 [0.13, 0.92]
Epstein R et al.31 (2004)	d	CRM‡	0.16 [0.15, 0.31]
Evans DV and Egnew TR39 (1997)	a, f, g	Group stages (Hackman)	0.47 [0.04, 0.97]
Featherstone P et al.35 (2005)	d, e, g	None described	0.41 [0.32, 0.49]
Fulmer T et al.32 (2005)	b, d, e, f	Teams skills list	0.44 [0.2, 0.61]
Grogan EL et al.28 (2004)	b, d, g, h	CRM‡	0.36 [0.05, 0.36]
Hope JM et al.36 (2005)	e, f	Group stages (Hackman)	0.37 [0.14, 0.49]
Østergaard HT et al.29 (2004)	a, b, c, d, e, g	CRM‡	–
Ponzer S et al.30 (2004)	c, e, g	none described	0.28 [0.17, 0.42]
Shapiro MJ37 (2004)	a, b, c, d, e	CRM‡	0.73 [0.42, 0.79]
Stoller JK et al.38 (2004)	a, f	Kouzes and Posner	0.11 [−0.70, 0.68]

* Eight teamwork principles: (a) the curriculum included leadership training or evaluated leadership skills; (b) team members monitor one another’s performance and provide feedback (mutual performance monitoring); (c) the curriculum addressed redistributing tasks upon demand by anticipating team member’s needs through accurate knowledge of their responsibilities (backup behavior); (d) the curriculum addressed the ability to adapt to changing situations; (e) the curriculum addressed soliciting team member ideas in defining goals and objectives (team orientation); (f) the curriculum addressed fostering trust between team members; (g) the curriculum included communication training or evaluated communication skills; (h) the curriculum addressed ensuring that team members are “on the same page” (shared mental models).

‡ CRM refers to crew/crisis resource management.

‡ For each outcome result, the mean pre/post difference was divided by the pooled standard deviation yielding an effect size. The median of all calculated effect sizes within a given article is the median overall effect size.

Table 2 identifies the specific teamwork principles described in each article. The most frequently included principle was adaptability (9, 69%),^{28,29,31–35,37,40} although team involvement in defining goals/objectives and communication were nearly as common (8, 62% each).^{29,30,32,34–37,40} Leadership (6, 46%)^{29,33,37–40} and trust (5, 39%)^{32,34,36,38,39} were stressed in several of the curricula. Team performance monitoring appeared in only 4 curricula (31%),^28,29,32,37 while redistributing tasks according to need was identified in only 3 curricula (23%).^29,30,37 A single curriculum (13%) described the principle of ensuring that team members are “on the same page.”²⁸

Educational Strategies

Most curricula placed medical trainees in multidisciplinary learning environments that included nurses, social workers, physical/occupational therapists, administrators, and pharmacists (10, 77%; Table 1).^{28–30,32–37,40} All curricula employed some active learning methods, defined as methods that required active involvement of learners. These included “critical incident” simulations such as respiratory arrest or traumas (4, 31%),^29,31,34,37 role play (23%),^28,31,36 case-based scenarios/discussion (6, 46%),^{28,32,33,35–37} and actual patient encounters (3, 23%).^30,32,40 No curriculum described using standardized patients in teamwork training. Three of 13 curricula employed nonmedical teambuilding exercises,³³ such as a rope course³⁹ or a survival game.³⁸

The majority of the curricula incorporated feedback as an integral part of active learning. Seven (54%) used formal debriefing sessions to provide feedback for learners in the curriculum.^{28,29,31,37–40} However, the articles did not elaborate on the specific details of feedback methods used by the course instructors. Five (38%) of the curricula used facilitated reflection as a technique to help learners gain a richer understanding of teamwork.^{30,31,33,34,36}

Evaluation Design

Three studies (23%) used a controlled pre/post evaluation design;^33,37,39 10 (77%) used an uncontrolled pre/post design.^{28–32,34–36,38,40} There were no randomized controlled trials. Five studies (38%) were multi-institutional.^{29,30,32,35,39} The time of the evaluation follow-up varied from immediately postintervention to 2 years, with 10 studies reporting only short-term (<30 days), end-of-program results.^{28–34,37,38,40} Two curricula reported medium-term results: 30 days after the completion of the course³⁹ and 6 weeks after the course.³⁵ A single study reported the results of narrative assessments completed by course participants 1–2 years after the intervention.³⁶

Evaluation Measures

Four (31%) of the curricula used self-reported measures of knowledge;^30,36,39,40 none used an objective test of knowledge (Table 3). None of the knowledge assessments described had been previously validated or reported reliability testing. Most commonly, knowledge outcomes demonstrated improvements in group awareness.^36,39,40 In one case, knowledge of multidisciplinary roles was increased.³⁰

Table 3.

Reported Outcomes of Teamwork Interventions

Author (year)	Outcomes expected/measured	Outcome results (K = knowledge, S =skill/behavior, A = attitude, C = clinical)
Cooper H et al.33 (2005)	Questionnaire of self-assessed teamwork skills and attitudes;* analysis of written reflections	S: acquisition of team skills did not improve (p > .1); A: appreciation for and awareness of positive professional relationships increased by 8% (p < .01)
DeVita MA et al.34 (2005)	Observer checklist of role-related task completion using videotapes of simulation exercises	S: task completion rate improved from 31% to 89% overall (p < .05); S: improved simulated patient survival (p < .05)
Dienst ER and Byl N40 (1981)	Questionnaire of self-assessed attitudes towards a team approach, knowledge about healthcare teams, and skills in applying team concepts to problem solving; patient interview and chart review	K: 38% increase in health care team concepts and roles (p < .05). S: 6.5% increase in team problem solving (p < .05); A: no change in attitudes towards teamwork (p > .1); C: team effectiveness did not correlate with patient satisfaction; C: chart completeness correlated with team satisfaction (p < .5)
Epstein R et al.31 (2004)	Questionnaire of self-assessed attitudes and teamwork skills;* observer checklist of team skills for students using a human patient simulator	S: high simulator performance team score (72%; 95% CI 0.66, 0.78); A: 71% of learners agreed or strongly agreed that simulator exercises were valuable 95% CI (62%, 80%)
Evans DV and Egnew TR39 (1997)	Questionnaire of self-assessed team effectiveness skills and group awareness knowledge;* questionnaire of self-assessed attitudes towards communication	K: knowledge of group roles improved by 13% (p < .001); S: group effectiveness improved by 8% (p < .05); A: communication confidence increased by 8.5% (p < .05)
Featherstone P et al.35 (2005)	Questionnaire of self-assessed attitudes about working in a multidisciplinary team*	A: confidence to work on an interdisciplinary team improved by 6% (p < .05)
Fulmer T et al.32 (2005)	Questionnaire of self-assessed team communication, interdisciplinary team skills, and attitudes towards health care teams;* observer ratings of team skills	S: 36% overall improvement in team skills and communication (p < .001); A: attitude towards healthcare teams improved by 11% (p < .001)
Grogan EL et al.28 (2004)	Questionnaire of self-assessed attitudinal shifts regarding behaviors emphasized in CRM modules*	A: improvement in 20 of 23 items of attitude scale (p < .01)
Hope JM et al.36 (2005)	Questionnaire of self-assessed knowledge of team roles and skill in accomplishing various teambuilding functions;* narrative assessments completed by course participants 1–2 y after the intervention	K: 36% increase in understanding of multidisciplinary team roles (p < .01); S: 44% improvement in self-assessed teamwork skills (p < .001)
Østergaard HT et al.29 (2004)	Questionnaire of self-assessed leadership and teamwork skills	S: 22% increase in number of participants rating teamwork skills as very good or good (statistics not reported)
Ponzer S et al.30 (2004)	Questionnaire of self-assessed knowledge of team roles and self-assessed attitudes towards teamwork	K: 62% of medical students reported improved knowledge of multidisciplinary roles (p < .05); A: 20% increase in number of learners rating teamwork important or very important (p < .001)
Shapiro MJ et al.37 (2004)	Observer-assessed ratings of teamwork skills and behavior*	S: 8% improvement in team dimension rating (p < .05) and a trend towards improved teamwork behavior ratings (p < .1).
Stoller JK et al.38 (2004)	Questionnaire of leadership and team attitudes	A: improved attitudes towards importance of leadership and to the contribution of all team members (p < .05)

* Study used previously validated instruments to assess outcome, or reliability testing was reported in manuscript.

Eight curricula (62%) showed improvements in self-assessed team skills (Table 3).^{29,31,32,34,36,37,39,40} Four curricula included observer assessments of team skills.^31,32,34,37 Six instruments that were used to assess teamwork skills had undergone reliability testing and had been previously validated: Rochester Communication Rating Scale,³¹ Team Skills Scale,³² leadership and group development assessment,³⁹ ALERT questionnaire,³⁵ the Team Development Wheel,³⁶ and the Team Dimensions Rating form.³⁷

Nine (69%) of the curricula assessed attitudinal changes (Table 3);^{28,30–33,35,38–40} all were self-assessed, and one used analysis of written reflections.³³ Three curricula assessed attitude outcomes using instruments that had undergone reliability testing and had been previously validated: the Human Factors Attitude Survey,²⁸ the Attitudes Towards Health Care Rating Scale,³² and the Readiness in Interprofessional Learning.³³ These assessments most frequently found improved attitudes toward interdisciplinary healthcare teams. Four studies found improved attitudes towards allied healthcare professionals’ roles (nursing, social work, case management, physical therapist, pharmacy) and their contributions to the team effort.^28,32,33,38 One study found no change in attitudes towards teamwork.⁴⁰

While patients were involved in 3 of the curricula,^30,32,40 only one examined clinical outcomes. This curriculum did not find a relationship between team effectiveness and patient satisfaction.⁴⁰

Efficacy

All but one article provided the statistical details required to calculate effect size.²⁹ The median overall effect size for each of the 12 articles ranged from 0.11 to 0.73 (see Table 2). The median effect size for knowledge outcomes was 0.40 (IQR 0.29, 0.61; 4 studies included), for skill/behavior outcomes 0.41 (IQR 0.35, 0.49; 9 studies included), and for attitudinal outcomes 0.38 (IQR 0.32, 0.41; 9 studies included).

When we looked at individual teamwork principles, none were significantly associated with overall effect size or separated knowledge, skill, or attitude effect sizes. The relationship between the total number of teamwork principles described in each article and the median overall effect size for each article achieved a Spearman correlation of .74 (p = .01; Fig. 2). We performed a sensitivity analysis by recalculating the Spearman correlation after removing the highest³⁷ and lowest³¹ data points, which still showed a trend towards a relationship (Spearman correlation .56, p = .09). While the number of teamwork principles described did not significantly correlate with individual knowledge effect sizes (Spearman correlation of .31, p = .68) or attitude effect sizes (Spearman correlation of .55, p = .13), skill/behavior effect sizes were associated with the number of principles taught (Spearman correlation of .64, p = .03).

Figure 2.

Scatter plot of median overall effect size (Cohen’s d)* versus the number of teamwork principles described in each article. The trend line represents the Spearman’s correlation. The data point numbers correspond to the article reference number. *For each outcome result, the mean pre/post difference was divided by the pooled standard deviation yielding an effect size. The median of all calculated effect sizes within a given article is the median overall effect size.

DISCUSSION

Teaching teamwork is increasingly relevant in medical education as many factors threaten the ability of trainees to coordinate their individual efforts to care for patients. Contemporary medical training places coworkers together for variable—often short periods of time. Each month, teams form and then disband, posing a challenge to team cohesion.⁴¹ Additionally, work-hour restrictions necessitate increasing emphasis on the coordination of roles, facilitation of transfers, and supervision. In their examination of medical claims between 1984 and 2004, Singh et al. reported that teamwork breakdowns stemming from communication breakdowns and handoff errors were a contributing factor in 70% of the cases.⁴² The authors suggested that the potential for medical errors would only increase with changes in work-hour regulations. Teaching trainees to work together effectively may play an important role in minimizing these risks.

All of the teamwork curricula reviewed used reasonable educational strategies, such as engaging subjects in active learning and the use of referenced teamwork models.^{28,29,31,34,37} Most used reflection or structured feedback on performance and appropriate educational strategies for skills training particularly within the context of relatively stable medical teams such as code teams. Few curricula, however, involved learners in experiences for greater than 8 weeks. Teamwork training may require longitudinal instruction to address common team-related problems in medicine. For example, coordinating patient handoffs by asynchronous individuals or participating in long-term projects such as continuous quality improvement (CQI) may benefit from teaching teamwork principles over time. Additionally, longitudinal curricula would also allow teamwork curricula to be combined with teaching of other skills (CQI) and allow trainees to apply their learning in a variety of medical contexts.⁴³ For teamwork skills to be incorporated into ongoing behavior they may need to be reinforced in real-life environments, in the informal or hidden curricula of training institutions.^44,45

In this article, we have used the framework described by Baker, which we feel is comprehensive, can provide guidance for curricular design, and lends itself to hypothesis testing. Incorporating a greater number of Baker’s principles was associated with larger overall median effect sizes, yet we note that 3 of these principles were present in 4 or fewer articles: backup behavior, mutual performance monitoring, and being on the same page. It is possible that incorporating these principles into a more comprehensive teamwork model will help those engaged in curriculum development to design more effective programs.

Another possibility is that the association of positive outcomes and comprehensive use of Baker’s principles may simply be the result of increasing the likelihood that the right teamwork principles were taught for a given context. Different teamwork principles may be more or less important depending on the context. For example, teaching teamwork may address handoffs more effectively if curriculum developers emphasize specific teamwork principles, such as mutual performance monitoring, shared mental models, and backup behavior, over principles such as leadership.

Although several curricula in this review used validated instruments^{28,31–33,35,37,39} to assess teamwork, evaluation methodologies were generally weak. No evaluations used randomization and few used a controlled design. Only one looked at a clinical outcome: patient satisfaction. Self-assessments of knowledge, perceived skill attainment, and a lack of long-term follow-up of learner outcomes further limit the conclusions that can be drawn from these evaluations.⁴⁶ Developing metrics that are specific to teamwork principles would allow investigators to characterize the relative importance of each teamwork principle and the degree to which a principle is present.

The strengths of this systematic review include a methodical literature search of several databases with the assistance of a medical librarian, the use of 2 independent reviewers to extract data, and the use of a teamwork training assessment tool derived from a published theoretical framework to synthesize our data. Certain methodological limitations, however, should be considered in interpreting the results. First, there may be publication bias in reviews that include only published articles. We did create a funnel plot of effect size versus sample size, which demonstrated approximately equal distribution of studies with low and high effect sizes at different sample sizes, providing some evidence against a significant publication bias. Second, our search strategy was limited to English-only articles. Third, we used median effect sizes to compare results across studies using different outcome measures, which have the disadvantage of giving equal weight to both strong and weak evaluations. However, these methods have been recommended by others^24–26 as a way to synthesize heterogeneous outcomes common in studies of medical curricula. Fourth, we could not address the potential correlation between teamwork principles because of the small sample size. Fifth, our assessment of Baker’s principles only counts presence or absence and not the quality of content, which was frequently not described. Finally, because of the small number of studies in our review, a few outlier studies may have excessively influenced the correlation between the number of teamwork principles used and the effect size of the results. To account for this, we conducted a sensitivity analysis that eliminated the highest and lowest data points and showed a similar but nonsignificant trend (p = .09).

Notwithstanding these limitations, this review demonstrates that reported curricula employ sound educational principles, appear to be modestly effective in the short term, and seem to be more effective when they address more teamwork principles. Future curricular work and evaluation should focus on context-specific teamwork training, the relation between specific curricular content/educational methods and outcomes, and improved evaluation strategies, including randomized trials, long-term follow-up, and stronger outcome measures.