Effects of simulation-based cardiopulmonary and respiratory case training experiences on interprofessional teamwork: A systematic review

Abstract

Objective

Simulation exercises are meant to provide an opportunity for health care workers to improve teamwork and develop clinical skills, among other goals. The objective of this systematic review was to determine whether simulated interdisciplinary activities in the health care or clinical setting improve interprofessional collaboration within health care teams that include respiratory therapists.

Methods

A systematic literature search of PubMed, EMBASE and CINAHL was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analyses reporting guideline to find relevant articles using both MeSH terms and free text. Filters were applied to include English-language studies; studies published within the last 10 years (2011–2021), and studies involving human participants. Studies were excluded if they did not assess the effects of simulation on aspects of teamwork, if participants were students, if teams did not include respiratory therapists, or if the training did not involve a simulated experience in a clinical setting. The search identified 312 articles, 75 of which were advanced to full-text review. Of those 75 articles, 62 were eliminated for not measuring teamwork in their outcomes. Two articles were excluded for being published before 2011, and one was eliminated for poor methodological quality. A risk of bias assessment using standardized qualitative and quantitative appraisal checklists was conducted on each of the remaining 10 studies selected for inclusion.

Results

A total of 10 studies met the inclusion criteria for this review (eight prospective, pre/post-test studies and two prospective observational studies). Randomization and participant/researcher blinding were not present in the majority of the studies and reporting bias was also found to be a concern throughout the literature. However, all of the studies noted increased teamwork scores post-intervention, though they differed in the tools used to evaluate this outcome.

Discussion

Collectively, the studies included in this review demonstrate that interprofessional simulation experiences including respiratory therapists enhance teamwork. The various tools used to assess change in teamwork had evidence of validity; however, studies varied in their outcomes measured, making quantitative analysis inappropriate. There are challenges involved in creating and assessing these simulations, particularly when performed within a clinical environment, which make it difficult to fully remove bias from the study design. It is unclear if the teamwork improvement can strictly be attributed to the simulation intervention or in part due to the general development of team members’ competencies throughout the research period. Additionally, the permanency of the effects cannot be evaluated based on the studies included and could be an area for future research.

Conclusion

Despite the limited number and methodological precision of studies included in this review along with the differing outcome evaluation methods, the authors conclude that positive teamwork improvement results are generalizable and agree with the broader base of research of the effectiveness of simulation on teambuilding.

INTRODUCTION

In fast-paced clinical environments, it is essential that health care providers work together to adequately care for their patients. This model of collaborative practice occurs “when multiple health workers from different professional backgrounds provide comprehensive health services by working with patients, their families, carers and communities to deliver the highest quality of care” [1]. In the last decade, Interprofessional Education has been used to prepare learners for collaborative clinical practice within the workforce. The Centre for the Advancement of Interprofessional Education defines Interprofessional Education as “learning that occurs when students or members of two or more professions learn, about, from and with each other to improve collaboration, and quality of services and care” [2]. Many health care authorities and regulating colleges require continuing education inclusive of interprofessional care skills.

Landmark reports, such as To Err is Human: Building A Safer Health System [3], highlight the importance of preventing medical errors, which in Canada accounts for approximately 28,000 deaths yearly [4]. As such, simulation-based training for health care providers has become a well-established practice, particularly for non-technical skills, to mitigate such instances. Non-technical skills can be described as the cognitive and interpersonal skills possessed by health care providers that, along with clinical knowledge, aid in the delivery of care [5]. These skills are known to be critical for effective teamwork and important in the prevention of errors and adverse events in hospitals [6]. Simulation is a tool that can be used as an immersive and experiential learning strategy to replicate real-patient experiences in safe practice environments [7]. In the literature, many existing reviews explore simulation-based team training in various health-related disciplines. However, there are few studies assessing the use of simulation relating to interprofessional teamwork in the workplace and even fewer that include respiratory therapists (RTs).

RTs are highly skilled members of the interprofessional health care team that work in many areas of medicine, including critical and emergency care, community care, diagnostics, rehabilitation, code teams, operating rooms, neonatology and paediatrics [8]. Their role is to assist in diagnosing and treating patients of all ages suffering from respiratory and cardiopulmonary disorders, which range in severity from routine to life-threatening [9]. This breadth of practice means that RTs must be functional in many different instances of collaborative practice. As such, the authors undertook this systematic review to evaluate a primary outcome of whether simulated interprofessional experiences improve teamwork within health care teams that include RTs. This systematic review also assesses secondary outcomes to include other measures related to teamwork/collaboration that took place in a health care/clinical setting, including improvements in team dynamics and functionality, individual and team confidence levels, closed-loop communication and the use of huddles and debriefings.

METHODS

This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analyses reporting guideline [10]. The Centre for Reviews and Dissemination, University of York outlines collaboration and teamwork as social outcomes and not direct health-related outcomes in patient care. Therefore, an International Prospective Register of Systematic Reviews (PROSPERO) registration number was not applicable or included in this review.

Search strategy for identification of studies

A systematic literature search of PubMed (2011 to 2021), EMBASE (2011 to 2021) and CINAHL (2011 to 2021) was conducted to find relevant articles using the Medical Subject Headings (MeSH) terms “Interdisciplinary Communication,” “Patient Simulation,” “High Fidelity Simulation Training,” “Simulation Training,” and “Patient Care Team.” To broaden the literature search to include articles without indexed MeSH terms, free-text was used to include the following keywords alone or in various combinations: respiratory, respiratory therapy, health care, communication, collaboration, simulation, interdisciplinary, simulation-based team training and teamwork. Boolean operators ‘AND’, ‘OR,’ and ‘NOT’ as well as wildcard placeholders (*) were used to further tailor search results. A final search of all databases was conducted on December 12, 2021. The following example search was entered into PubMed: (“Patient Simulation”[Mesh] OR “High Fidelity Simulation Training”[Mesh] OR “Simulation Training”[Mesh]) AND (“Clinician” OR “Health Care Worker” OR “Healthcare Worker” OR “Healthcare Staff” OR “Respiratory Therap*”) AND (“Teamwork” OR “Team work” OR “collaboration” OR “team” OR “communication”).

Study selection

Titles and abstracts were screened to identify potential studies for inclusion. For relevancy and applicability, filters were applied to include English-language, studies published within the last 10 years, and studies involving human participants. To be considered for this review studies needed to include RTs in their health care team; include clinical simulation-based experiences where RTs’ expertise was required and measure the effects of the simulation experience on aspects of teamwork. Studies were excluded if they did not assess the effects of simulation on aspects of teamwork, if participants were students, if teams did not include RTs, or if the training did not use a simulated clinical experience where RTs would be required. All study authors independently screened eligible studies by title, abstract, and content, and the decision to include or exclude studies was decided through group consensus. All authors worked collaboratively to synthesize key messages from the collected evidence. Consensus was achieved through discussion.

Risk of bias assessment

Risk of bias assessment was conducted across all included studies. The Qualitative Checklist from the Critical Appraisal Skills Program [11] was used, and a quantitative section was added, which was obtained from Viswanathan et al [12].

RESULTS

Description of included studies

Figure 1 documents the study selection process. The systematic literature search identified 312 articles. Following a review of titles and abstracts, 237 were eliminated, and 75 unique studies were advanced for full-text review. Two studies were eliminated because they were more than 10 years old. Sixty-two studies were eliminated because they did not include an assessment of teamwork change as an outcome measure. Eleven studies were assessed for quality using the Critical Appraisal Skills Program [11]. One study was eliminated due to poor methodological quality. Therefore, a total of 10 studies met the inclusion criteria for this review [13–22] (eight prospective, pre/post-test studies and two prospective observational studies) (Table 1).

FIGURE 1.

Study selection process flowchart

TABLE 1.

Characteristics of included studies

Study	Simulated clinical experience; Training setting; Study design	Description of intervention: Study population	Outcome assessment tool	Outcomes
Prospective, pre-post test study
Mahramus et al, 2016 [20] Southeastern USA	Simulated Clinical Experience: Cardiopulmonary Arrest Setting: Simulation Training Laboratory Study Design:Prospective, Pre-Post Test Study	Intervention: Brief Interventional Training Program: incorporating components of Crew Resource Management (CRM) and the TeamSTEPPS training programs.* Session length: 2 h using a Laerdal SimMan 3G high-fidelity manikin Study Population: Cardiopulmonary Arrest team members (n=73) 18 resident MDs 23 critical care RNs 30 RTs *Quasi experimental RCT design.	TEAM Survey Tool†	Baseline teamwork scores (mean SD on TEAM tool) improved from 2.57 to 2.72 out of 5 to 3.2 (0.5) and 3.7 (0.4), for the first and second simulation, respectively (P<0.001) Observers’ mean (SD) TEAM scores for the first and second simulations were 3.0 (0.5) and 3.7 (0.3) respectively (P<0.001). Overall, 2-h simulation-based teamwork training improved measures of teamwork from a multidisciplinary group who routinely respond to code events.
Sawyer et al, 2013 [22] Honolulu, Hawaii, USA	Simulated Clinical Experience: High Acuity Neonatal Resuscitation Setting: Simulated Delivery Room Study Design:Prospective, Pre-Post Test Study	Intervention: TeamSTEPPS training program* Computerized neonatal manikin. Study population: Neonatal Resuscitation Team members (n=42) 29 RNs 10 physicians 3 RTs	TeamSTEPPS tools ‡ TeamSTEPPS Team Performance Observation Tool TeamSTEPPS Teamwork Attitudes Questionnaire TeamSTEPPS Learning Benchmarks	Attitudes toward teamwork improved from a pretest average of 4.4 to a post-test average of 4.7 (95% CI 20.34 to 20.22; P<0.001). Teamwork knowledge of TeamSTEPPS Learning Benchmarks improved from a pretest average of 86.8% to a post-test average of 92.6% (95% CI 28.32 to 23.26; P=0.001). Significant improvements in teamwork skill after TeamSTEPPS training for team structure. Pretest 2.5 vs. post-test 4.2 (95% CI 22.0 to 21.4; P<0.001). Situation monitoring pretest 2.5 vs. post-test 4.3 (95% CI 22.2 to 21.5; P=0.001). Mutual support pretest 2.9 vs. post-test 4.3 (95% CI 21.8 to 21.0; P=0.001). Communication pretest 3.0 vs. post-test 4.4 (95% CI 21.6 to 21.1; P=0.001)
Gilfoyle et al, 2017 [14] Calgary, Toronto, Montreal, Canada	Simulated Clinical Experience: Resuscitations: Cardiogenic Shock; Asystole; Unstable Supraventricular Tachycardia Setting: Simulation Centres Multi-centred Study (51 teams) Study Design:	Intervention: 1-day simulation-based team training course, four simulated resuscitation scenarios followed by debriefing. Infant Mannequins Study Population: Interprofessional Paediatric Resuscitation Teams (n=300) 103 Resident physicians, 2 ICU nurse practitioners, 130 registered nurses, 65 Respiratory Therapists	Clinical Teamwork Scale Teamwork and communication skills measured using a 14-item scale ranging from 0 (unacceptable) to 10 (perfect). Clinical Performance Tool Measured change in adherence to Paediatric Advanced Life Support guidelines.	Teams improved in Clinical Performance Tool scores (67.3% to 79.6%; P<0.0001) Time to initiation of chest compressions (60.8% to 27.1%; P<0.0001), time to defibrillation (164.8–122.0 s, P<0.0001) Clinical Teamwork Scale scores (56.0% to 71.8%; P<0.0001). Positive correlation between Clinical Performance Tool and Clinical Teamwork scale (R=0.281; P<0.0001)
Figueroa et al, 2013 [15] Memphis, Tennessee, USA	Simulated Clinical Experience: Post paediatric Cardiac Surgical Emergency Scenarios Setting: Offsite Simulation Centre (PICCU patient room) Study Design:Prospective, Pre-Post Test Study	Intervention: TeamSTEPPS learning program (9-h program) High fidelity patient simulator mannequins: Newborn and Paediatric HAL (Gaumard) and SIM Man (Laerdal) Study Population: Multidisciplinary Paediatric Cardiac Intensive Care Unit Team (n=37) 23 RNs 5 cardiology/ critical care trainees 5 RTs 4 noncategorized	Purpose built undefined questionnaire assessing Team STEPPS and PPCS-CA outcomes	Course participants perceived themselves better prepared to participate in and lead future code events (P<0.05) Reported improved confidence and skill immediately after training and 3 months later in advanced airway management and cardioversion/defibrillation (P<0.05) Teams overall showed a statistically significant increase (P<0.05) in closed-loop communication, use of huddles and debriefings, perfection of mutual respect and a sense of empowerment.
Tsai, AC., et. al., 2016 [16] Boston, Massachusetts, USA	Simulated Clinical Experience: Emergency Airway Management Setting: Simulation Centre Study Design: Prospective, Pre-Post Test Study	Intervention: 2–15-min simulations each followed by 30–45-min debriefing Study Population: Emergency Airway Response Team (ERAT) (n=177) 10 attendings 76 residents 63 RNs 25 RTs 3 other	Purpose built questionnaire.	Pre- and post-simulation assessment of impact on team effectiveness, personnel confidence and protocol knowledge Residents increased team dynamics and confidence scores from 3.5 to 4.2 post simulation. Participants with <5 years’ experience improved team dynamics and confidence scores from 3.4 to 4.3 post-simulation Participants with 5–15 years of experience increased their scores from 4.2 to 4.8 Those with ≥15 years’ experience increased scores from 4.4 to 4.8 EART protocol knowledge scores increased from 70.6% to 90.9% (P<0.1)
Steinemann et al, 2011 [17] Hawaii, USA	Simulated Clinical Experience: Blunt trauma shock; patients >6 years. Setting: Hospital Trauma Centre Study Design: Prospective, Pre-Post Test Study	Intervention: Human patient simulation, in-situ team training (HPS) 3–15-min scenarios each with immediate video-enabled debriefing. SimMan 3D Mannequin (Laerdal) Study Population: Multidisciplinary groups of 5–8 (blinded to clinical endpoints but not study outcome) N=137 9 attending surgeon 21 attending ED physician 3 physicians’ assistant 24 residents 14 critical care RN 30 ED RN 23 RTs 13 ED tech	Trauma-NOTECHS (5 main teamwork domains illustrated by 47 behavioural exemplars); pre and post HPS measures. Measured real life resuscitation events pre and post HPS training (244 resuscitation)	Pre and post scenario assessment revealed significant improvement (P<0.05) in real life trauma teamwork from first to last scenario. T-NOTECHS scores revealed improvements in all participants. ≥7 of the eight tasks required – scores went from 32% to 84% (P≤0.05) Mean time in seconds to completion of real-life resuscitation pre and post HPS (6 months follow up) was 460 s to 353 s (P≤0.01)
Chang, YC., et al, 2019 [21] Taiwan	Simulated Clinical Experience: Transport Teams: Septic shock and concomitant adverse events related to physiologic instabilities, equipment failures Setting: In-situ simulations: intensive care units, emergency departments, elevators, in front of operation rooms and CT rooms Simulator mannequin with ECG rhythm and vital sign monitoring, abnormal breathing sounds, variable lung compliance and airway resistance. Study Design: Quasi experimental. Prospective, Pre-Post Test Study	Intervention: In-situ Interprofessional Simulation-based Training (IIST) model 3 sessions delivered over 3 months: 2-h session each with unique scenario followed by video feedback and focus group discussion Study Population: N=36 newly registered postgraduates 12 residents 12 nurses 12 RTs Randomly assigned to experimental (n=6/group) or control groups (n=6/group) Each team had one RT, one resident and one nurse.	Anaesthetists’ Non-Technical Skills (ANTS) checklist Measures task management, teamwork, situational awareness and decision-making	Pre- and post analysis determined that those who engaged in the IIST package achieved significantly higher level of team functionality improved compared with control group. (data not shown) Teams in experimental group continually advanced their non-technical skills with each exposure to the IIST activities.
Miller et al, 2012 [18] Albuquerque, New Mexico, USA	Simulated Clinical Experience: Shock following blunt abdominal trauma or penetrating chest wound Setting: In-situ simulations: Level 1 Trauma /Academic Tertiary Care Centre Study Design:Prospective, Pre-Post Test Study	Intervention: In situ Trauma Simulation Program Didactic session followed by eigth simulations plus debriefing experiences (1 per week) SimMan 3D Mannequin (Laerdal) Study Population: All trauma team members trauma surgery attendings and residents emergence department (ED) physician attendings and residents ED RNs technicians pharmacists clerks RTs	Clinical Teamwork Scale* Measured communication and teamwork pre didactic, intra simulation and post program through observation of 38 real life trauma activations	Improvement in 11 of 14 CTS measures during the simulation phase. Teamwork mean score of 6.1 at baseline improved to 7.8 during simulation. P=0.020 Overall communication mean score of 5.3 at baseline improved to 7.8 during simulation. P=0.003 During the potential decay phase, the scores for every measure returned to baseline phase values.
Observational Study – Pilot Prospective
Nishisaki et al, 2011 [13] Philadelphia, PA, USA	Simulated Clinical Experience: Infant and Paediatric Orotracheal Intubation Setting: Paediatric Intensive Care Unit- In situ Study Design:Observational Pilot Prospective	Intervention: Just In Time – Airway Management Training High-fidelity human simulation sessions with mannequins: Megacode Kid & Human Infant Simbaby (Laerdal) followed by brief debriefing Study Population: 54 paediatric residents 24 emergency residents 122 Pediatric Intensive Care Registered Nurses (PICU RN) 65 RTs	Just In Time – Paediatric Airway Provider Performance Scale 15 real time orotracheal paediatric intubations assessed to measure change in teamwork	Airway teams with two or more JIT-Trained team members scored significantly higher compared with the airway teams with fewer than two JIT-trained team members (127±6 vs. 116±9; P=0.131) in the technical domain, 71±5 vs. 64±7; P=0.057) in the behavioural domain. Linear regression analysis showed airway team performance scores were significantly associated with the number of team members who previously participated in JIT training sessions. Overall intubation success rate with fewer than three attempts were 89% for the team with two or more JIT-Trained team members vs. 67% for the team with fewer than two JIT-trained team members (P=0.525).
Colman N, et al, 2019 [19] Atlanta, GA, USA	Simulated Clinical Experience: Septic shock, refractory bronchospasm, and, pulmonary hypertensive crisis with cardiac arrest. Setting: Paediatric Intensive Care Unit- In situ Study Design:Observational Pilot Prospective	Intervention: three concurrent simulation scenarios using high-fidelity mannequins: Human Infant SimBaby (Laerdal) each followed by a debriefing session. Study Population: 165 PICU staff members participated in a 3-h SBTT workshop. 110 nurses 30 RTs 9 PICU attendings 16 PICU fellows	Clinical Training Scale Tool*	Intra session assessments were conducted and paired t-testing showed a statistically significant improvement in teamwork performance as each scenario progressed from scenario 1 to 2, and from scenario 2 to 3 (P<0.0001).

^*References for these programs [23,24];

^†Prior studies have demonstrated internal consistency reliability of the TEAM tool ranged from 0.94 to 0.97 [23,24];

^‡TeamSTEPPS tools: Tools used to measure the outcomes of the TeamSTEPPS program include: TeamSTEPPS Team Performance Observation Tool (T-TPOT); TeamSTEPPS Teamwork Attitudes Questionnaire (TTA-Q); TeamSTEPPS Learning Benchmarks. The TTA-Q has been validated with strong evidence of Internal Validity (Cronbach’s alpha of 0.70–0.83 and Pearson coefficient among the constructs of 0.36–0.63 [25]. The T-TPOT has interrater reliability with a mean overall Cohen’s kappa of 0.86 [23].

The 10 included studies were conducted in three countries (United States, Canada and Taiwan) between 2011 and 2019. Nine of the 10 studies outlined the number and role of participants, incorporating a total of 1471 subjects (summarized in Table 2). The minimum number of participants in a study was 37, and the maximum was 300. As per our study inclusion criteria, all studies included RTs. One study did not report the total number of participants but included trauma surgery attendings, residents, emergency department nurses, technicians, pharmacists, clerks and RTs.

TABLE 2.

Study population breakdown

Author	Total	Nurse	Doctors	Residents	RT	Nurse practitioner	Other allied*	Not mentioned
Mahramus et al	73	23	0	18	30	0	0	2
Sawyer et al	42	29	10	0	3	0	0	0
Nishisaki et al	265	122	0	78	65	0	0	0
Colman et al	165	110	9	16	30	0	0	0
Gilfoyle et al	300	130	0	103	65	2	0	0
Figueroa et al	37	23	0	0	5	0	5	4
Tasi et al	177	63	10	76	25	0	0	3
Steinemann et al	137	44	33	24	23	0	16	0
Chang et al	275	128	0	131	16	0	0	0
Miller et al†	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
Total	1471	672	62	446	262	2	21	9

^*Other Allied: ED technicians and physician assistants.

^†Miller et al did not specify participant breakdown but included trauma surgery attendings, residents, ED nurses, technicians, pharmacists, clerks and RTs.

Simulated clinical experiences

Descriptions of the simulated clinical experiences, training settings and study population characteristics are presented in Table 1. The included studies reported on simulated clinical experiences derived from a spectrum of settings (eg, delivery rooms, emergency departments, intensive care units, CT rooms, patient rooms and elevators) and clinical cases. Most simulation experiences involved emergent events requiring respiratory intervention. These include respiratory failure, cardiogenic shock leading to tachycardia and stroke, postoperative hemorrhagic stroke, postoperative Fontan with low cardiac output, thrombosed modified Blalock-Taussig shunt, tetralogy of Fallot, Ebstein anomaly, blunt abdominal trauma, penetrating chest injury, pericardiocentesis, intraperitoneal fluid, septic shock, refractory bronchospasm, cardiac arrest and pulmonary hypertensive crisis.

Most studies (n=9) utilized manikins for simulated events. Mahramus et al [20] used the Laerdal SimMan 4G high-fidelity manikin, whereas Miller et al [18] and Steinemann et al [17] used the SimMan 3G model. Gilfoyle et al [14] and Colman et al [19] used the Laerdal Baby Manikin. The Gaumard Newborn HAL and Paediatric HAL simulators were used by Figueroa et al [15], Sawyer et al [22] and Nishisaki et al [13]. Sawyer et al [22] and Nishisaki et al [13] referred to them as human infant manikins. Chang et al [21] did not specify which brand of manikin was used but mentioned a simulator mannequin with electrocardiogram rhythm and vital sign monitoring, abnormal breathing sounds, variable lung compliance and airway resistance. They also mentioned using plastic bags as simulated functional lungs. One study by Tsai et al [16] did not provide details of its simulation.

Tools and processes used to assess teamwork

Studies assessed changes in aspects of interprofessional teamwork using a variety of tools and approaches. A summary of outcome assessment tools can be found in Table 1.

Figueroa et al [15] and Sawyer et al [22] used teamSTEPPS tools developed by the Agency for Healthcare Research and Quality (AHRQ) and the US Department of Defense. This method evaluates five core competencies of teamwork including team structure, leadership, situational monitoring, mutual support and communication. Originally evaluated by the AHRQ, Sawyer et al [22] further evaluated the validity of teamSTEPPS in their study using measures of content, response process, relationship to other variables, internal structure and consequence validity. Internal structural validity and response validity were also assessed by Sawyer et al. The authors used a TeamSTEPPS Team Performance Observation Tool ( T-TPOT), which was validated for inter-rater reliability using Cohen’s kappa [23]. However, Figueroa et al [15] used undefined assessment tools and included no reference to validity.

Mahramus et al [20] used the TEAM survey tool. The TEAM survey tool was originally developed to assess teamwork skills during resuscitation events and consists of 11 items, 10 rated on a Likert scale of zero to four, and one item on a global rating of teamwork on a scale of 1 to 10. This study tool consisted of 33 questions and was a composite tool from the Observational Skill based Assessment tool for Resuscitation, validated in 2011 by Walker et al using face and content validity and reliability assessment using Cronbach’s alpha to assess inter-rater reliability [26].

Gilfoyle et al [14] measured successes based on Paediatric Advanced Life Support guidelines evaluated with the Clinical Performance Tool. This tool examines adherence to paediatric guidelines using four checklists to assess shock, respiratory failure, asystole and supraventricular tachycardia. This tool was validated by the authors of the study for reliability and construct validity. The tool was also validated by Donoghue et al [27]. Gilfoyle et al [14] also evaluated teamwork using the Clinical Teamwork Scale, which was validated by Guise et al [28].

Steinemann et al [17] used the Oxford Non-Technical Skills (NOTECHS) scale for trauma that uses five main teamwork domains of leadership, cooperation and resource management, communication and interaction, assessment and decision making, and situation awareness/coping with stress. This method uses 47 behavioural exemplars and was validated by others, including Mishra et al, in 2009 [29].

Miller et al [18] evaluated teamwork and communication skills using the Clinical Teamwork Scale. This scale assesses teamwork with a 14-item scale ranging from 0 to 10, 10 reflecting a perfect score. It has been validated by Guise et al [28] and measures five domains of clinical teamwork: communication, situational awareness, decision-making, role responsibility and patient friendliness.

Colman et al [19] used the same tool as validated by Guise et al [28] but called it the Clinical Training Scale Tool. As above, this is a validated tool using standards of teamwork using Pearson’s correlation coefficient and Kappa coefficient for inter-rater reliability.

Nishisaki et al [13] developed an evaluation tool named Just-In-Time Paediatric Airway Provider Performance Scale for paediatric intubation using the health care failure mode and effect analysis to identify important processes necessary for safe airway management in the paediatric intensive care unit. The study authors used Health care Failure Mode and Effect Analysis independent expert raters and a global team performance assessment to assess the tool’s reliability and validity.

Tsai et al [16] compared pre- and post-simulation questionnaire scores and analyzed data using ANOVA testing. The authors used a 5-point Likert scale to rate the responses from strongly disagree to strongly agree. The tool was constructed using a Delphi methodology, which is a methodology that uses a panel of experts to arrive at a group consensus.

Lastly, Chang et al [21] assessed four non-technical skills (task management, teamwork, situational awareness and decision-making) using a validated 15-item Anaesthetists’ Non-Technical Skills checklist [30]. The authors assessed the technical skill of RT trainees (new hire RTs) with a 28-item checklist that was examined for expert validity.

Findings

A summary of the study outcomes can be found in Table 1. The study by Mahramus et al [20] found increased teamwork scores from 2.57 to 2.27 out of 5 to 3.2 and 3.7 for the first and second simulation, respectively (P<0.001). In their study, the observer mean TEAM scored 3.0 and 3.7 for the first and second simulations respectively (P<0.001).

In the study by Sawyer et al [22], attitudes toward teamwork improved from 4.4 to 4.7 (95% CI 20.32 to 20.22, P=0.001). After simulation experiences, significant improvement in teamwork skills was reported after training for team structure from 2.5 to 4.2 (95% CI 22.0 to 21.4, P=0.001) were measured. Additionally, communication increased from 3.0 to 4.4 out of five (95% CI 21.8 to 21.5, P=0.001).

Miller et al [18] recorded communication score improvements from 5.0 to 7.8 during simulation (P=0.147), though all measures returned to baseline values during the potential decay phase.

Colman et al [19] found an overall teamwork rating improvement post-simulation as well as with individual teamwork skills assessed including team rating, closed-loop communication, directed communication and role clarity. Statistically significant teamwork improvement was noted in their study with the addition of simulated events (P<0.0001).

In the study by Nishisaki et al [13], airway teams with two or more JIT-trained team members scored higher compared with the teams with fewer trained team members 127±6 versus 116±9 (P=0.131) in the technical domain, 71±5 versus 64±7 (P=0.057) in the behavioural domain.

Gilfoyle et al [14] reported improved Clinical Performance Tool scores from 67.3% to 79.6% (P<0.0001), improved time to initiate chest compressions from 60.8s to 27.1 s (P<0.001), and time to defibrillation 164.8 to 122.0 s. The authors concluded that the clinical performance of the teams improved in a manner likely to lead to improved patient outcomes in real-life events.

Figueroa et al [15] published improved confidence and skill after training and 3 months later in advanced airway management, cardioversion and defibrillation (P<0.05). Increases in closed-loop communication, use of huddles and debriefings, mutual respect, and a sense of empowerment were also noted (P<0.05).

The study by Tsai et al [16] assessed team dynamics with varying experiences. They reported participants with one to 5 years’ experience improved team dynamics and confidence scores from 3.4 to 4.3 post-simulation. Participants with five to 15 years’ experience increased their scores from 4.2 to 4.8. Those with 15 years of experience increased scores from 4.4 to 4.8 (P<0.1).

Steinemann et al [17] noted significant improvement in trauma teamwork from the first to the last scenario of 32% to 84% (P<0.05) and mean time in seconds to completion of three common resuscitation tasks from 460.0 to 353.0 s (P<0.01). Similarly, Miller et al [18] noted the mean time to application of cardiac monitors decreased from 158.0 to 129.0 s during the simulation phase.

Chang et al [21] found that participants achieved a significantly higher level of both technical and non-technical skills which increased with each exposure to the simulation training. The team functionality of the experimental group was significantly higher than the control group in the post-test. For RTs specifically, the overall achieved skills (involving intrahospital transport of a simulated critically ill patient) in the experimental group was 82.1% compared with 55.4% in the control group.

Risk of bias assessment

Risk of bias assessment was conducted across all included studies (see Appendix 1¹). The Qualitative Checklist from the Critical Appraisal Skills Program [11] was used, and a quantitative section was added, which was obtained from Viswanathan et al [12].

Randomization and participant/researcher blinding were not completed in the majority of studies [14,16–20,22]. Of the three studies that blinded participants [13,15,21], Nishisaki et al [13] noted potential bias as observers were not blinded to each simulated event. Figueroa et al [15] de-identified evaluations to reduce potential bias, but course instructors worked with the course participants. A single quasi-experimental study by Chang et al [21] used randomization.

Risk of selection bias was also found in included studies because convenience sampling was the most common form of participant selection. The study by Sawyer et al [22] recruited participants through a hospital-wide quality improvement initiative. Similarly, Chang et al [21] enrolled participants and randomly assigned them to an experimental or control group. The sample size was small and conducted in a single hospital. Mahramus et al [20] reported difficulties in eliminating participants through convenience sampling of those who routinely respond to code events as members of a hospital resuscitation team. Miller et al [18] collected data as a convenience sample, mentioning that they reported on data collected on days only, not nights and weekends, though this would not have affected the composition of the trauma team.

Reporting bias was also identified in studies that utilized participant self-reporting. For example, Mahramus et al [20] noted that “participants can be more inclined to self-report better scores post-simulation as they expect their teamwork skills would improve.” Sawyer et al [22] also noted this potential for bias post-simulation to self-report having improved. According to Steinemann et al [17], “The improvement in teamwork scores following training may be attributable, in part, to the Hawthorne effect.” A limitation of the study by Tsai et al [16] is that all those who could potentially respond to an event are not included in their study. Additionally, the authors do not use a specific validated questionnaire for reporting, though this method was justified with a Delphi methodology. Likewise, the present study utilizes self-reporting for data collection.

DISCUSSION

Collectively, the studies included in this review demonstrate that interprofessional simulation experiences enhance teamwork, as assessed through various measures. The tools used in nine of the 10 studies to assess change in teamwork (excluding Figueroa et al [15]) provided evidence of validity as indicated in the Tools and Processes Used to Assess Teamwork section. However, most studies used different methods to evaluate this result.

The variety of assessment tools used in these studies make it difficult to compare results across studies; however, because numerous distinct validated tools resulted in positive results, the effects can be considered more likely to be true. One tool could be poor at assessing teamwork change, but it is less likely that they are all poor. Another strength of the studies is that the simulated clinical experience cases and treatment settings were quite diverse. The fact that teamwork was advanced regardless of the specific clinical cases and places is positive.

Due to the nature of the study designs included in this review, each study included some forms of bias (reported in Table 1). When interpreting results, it is important to consider that subjects may have altered their behaviour based on the knowledge they were being observed, otherwise known as the Hawthorne effect [31]. As studies varied in their outcome measures, it was not possible to meaningfully combine the data for analysis.

It is unclear whether the improvement recorded in teamwork can be strictly attributed to the simulation intervention cultivating better group dynamics or in part due to the development of team members’ bedside and clinical skills over time as the research was conducted [20]. The permanency of these effects is also unclear because only one study [15] reassessed teamwork abilities after the initial simulation session and evaluation. After a 3-month follow-up, the authors found participants still felt a sense of empowerment. As a result of the simulation exercise, changes to practice had been made to improve teamwork and communication, including debriefings after any significant event and the incorporation of bed/shift huddles [15]. Steinemann et al [17] also noted, “in addition to subjective improvements in teamwork skills, we observed improvements in clinical process in the 6 months following teamwork training of most of our trauma team members.” However, according to Miller et al [18], there was no continuation and the improved teamwork and communication were not sustained.

Six of the 10 studies [13,14,17,19,21,22] specifically indicated there is a gap in the literature for more comprehensive research into the role of simulation on health care team dynamics and the length of time that the improvements (if present) would be sustained. As a remedy to this “dilution of training effect” [13], five of the 10 studies [13,16–18,21] suggested refresher simulations or integrating simulations into more regular schedules.

The challenges involved in creating and assessing these complex events within a working hospital setting (or offsite) cannot be understated. Aside from the technical issues of running a high-fidelity session (two studies noted issues with sound and video quality of recordings), several studies noted the logistical difficulty of scheduling participants in acute patient care areas such as emergency departments or intensive care units at the same time, even though these are the providers who would get the most benefit from multidisciplinary team training [13]. In addition, as noted by Nishisaki et al [13], even if one group experienced an improvement in teamwork skills during simulation, those same members would not necessarily be together in a real-world situation.

Despite the challenges and uncertainty, most studies concluded that simulation training was feasible, showed results in improved teamwork and that the building of technical and non-technical skills can be applied to any interprofessional health care team. The benefits of simulation have come to the forefront during the COVID-19 pandemic, including increased safety through the reduced need for patient interaction, the development of skills to reduce errors during a stressful period when protocols can change quickly, and a sense of solidarity with interprofessional colleagues.

Study limitations

The limitations of this review include the limited number of studies that met the criteria for inclusion; the lack of methodological precision in most studies; and small sample sizes in three studies [13,18,21]. No studies reported negative results, which could be due to the method of using self-reporting questionnaires for assessment (as in the majority of included studies). This can lead to positive answers from participants, as reported in the Risk of Bias section. Additionally, the search terms used to find articles may not have captured all relevant literature and the studies that were included by nature of their designs may have inherent participant bias toward those who are more at ease or function well within a team setting. However, data from this review agrees with published literature on the effectiveness of simulation in team building in the clinical setting.

Future research

To our knowledge, there are no studies to date that compare teamwork scores with and without RTs as part of the health care team undergoing simulation. Firstly, there is the potential for future research to analyze the specific impact of the RT role being present/absent and any impact on teamwork scores. Secondly, there is potential for research on skill retention post-simulation. Related to this, the research could focus on the optimal length, format and interval for refresher training. Furthermore, analysis based on each institution’s needs and available resources is necessary to determine whether the benefit warrants the cost. Thirdly, there is potential for research on the effect of these interventions on clinical outcomes in patient care, as noted by Steinemann et al [17] and Sawyer et al [22]; as several studies noted it is considered to be a strong possibility [14].

A review by Lackie et al [32] discusses how to embed the psychological safety of participants into simulation activities. According to their findings, safety enablers include pre-briefing-debriefing by trained facilitators, no-blame culture and structured evidenced-based simulation designs, while barriers include hierarchy among/between professions, being observed, and fear of making mistakes [32]. As mentioned, patient actors were used in three studies in their simulation events [14,17,19]. Although outside the scope of this review, this is an important area that should be considered in the early planning stages of all future simulation research.

CONCLUSION

The objective of this systematic review was to determine whether simulated interdisciplinary activities in the health care or clinical setting improve interprofessional collaboration and teamwork within health care teams that include RTs. Based on this review, it is clear that simulation should be fostered in the clinical setting for interprofessional teams including RTs, notwithstanding the various challenges. In these studies, simulation improved not only technical skills but also non-technical skills that are important in the prevention of errors and adverse events in hospitals.

A few of the reported secondary outcomes include improvements in team dynamics and functionality, individual and team confidence levels, closed-loop communication, use of huddles and debriefings and mutual respect, as well as a sense of empowerment. Although the technical nature of the interventions varies, the act of working together in a safe, simulated environment as an interdisciplinary group has proven benefits in the short- and medium-term. Future research is needed in the field of simulation and respiratory therapy for improved collaboration. Studying qualitative skills is often difficult, and therefore the undertaking of a systematic and longitudinal evaluation is needed to accurately measure whether simulation meaningfully contributes to the improvement of these skills in the workplace. Furthermore, how the development of these skills through simulation impacts patient care should also be measured.

DISCLOSURES

Contributors

All authors contributed to the conception or design of the work, acquisition, analysis, or interpretation of the data. All authors were involved in drafting and commenting on the paper and have approved the final version. Carly Brockington is Managing Editor for the CJRT, but was blinded to the decision-making process.

Funding

The present study did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Competing interests

All authors have completed the ICMJE uniform disclosure form and declare no conflict of interest.

Ethical approval

Not required for this article type.