Development of a Simulation-Based Interprofessional Teamwork Assessment Tool

Zia Bismilla; Tehnaz Boyle; Karen Mangold; Wendy Van Ittersum; Marjorie Lee White; Pavan Zaveri; Leah Mallory

doi:10.4300/JGME-D-18-00729.1

. 2019 Apr;11(2):168–176. doi: 10.4300/JGME-D-18-00729.1

Development of a Simulation-Based Interprofessional Teamwork Assessment Tool

Zia Bismilla ¹, Tehnaz Boyle ¹, Karen Mangold ¹, Wendy Van Ittersum ¹, Marjorie Lee White ¹, Pavan Zaveri ¹, Leah Mallory ^1,^✉

Zia Bismilla, MD, MEd, is Assistant Professor of Pediatrics, University of Toronto, The Hospital for Sick Children, Toronto, Ontario, Canada; Tehnaz Boyle, MD, PhD, is Assistant Professor of Pediatrics, Boston University School of Medicine, and Associate Clinical Director of Pediatrics, Solomont Center for Simulation, Boston Medical Center; Karen Mangold, MD, MEd, is Assistant Professor of Pediatrics and Medical Education, Feinberg School of Medicine, Northwestern University; Wendy Van Ittersum, MD, is Associate Professor of Pediatrics, Northeast Ohio Medical University, and Division of Hospital Medicine, Akron Children's Hospital; Marjorie Lee White, MD, MPPM, MA, is Professor of Pediatrics, Medical Education and Health Services Administration, and Assistant Dean for Clinical Simulation, University of Alabama at Birmingham; Pavan Zaveri, MD, MEd, is Associate Professor of Pediatrics and Emergency Medicine, The George Washington University School of Medicine and Health Sciences, and Director of Education, Division of Emergency Medicine, Children's National Health System; and Leah Mallory, MD, is Associate Professor of Pediatrics, Tufts University School of Medicine, The Barbara Bush Children's Hospital, and Director of Simulation, The Hannaford Simulation Center, Maine Medical Center.

Funding: Funding was provided by a Charlton Academic Research Award from Tufts University School of Medicine and an Education Innovations Grant from Maine Medical Center Institute for Teaching Excellence.

Conflict of interest: The authors declare they have no competing interests.

The authors would like to thank the members of the Delphi panel for their contributions to developing this tool. The authors would also like to acknowledge the International Network for Simulation-based Pediatric Innovation, Research and Education (INSPIRE) and the Society for Simulation in Healthcare and the International Pediatric Simulation Society for supporting the biannual INSPIRE research meetings.

This study was presented as a poster at the Pediatric Academic Societies Meeting, San Francisco, California, May 6–9, 2017, and as an oral presentation at the International Pediatric Simulation Symposia and Workshops, Boston, Massachusetts, June 1–3, 2017.

^✉

Corresponding author: Leah Mallory, MD, Maine Medical Center, 22 Bramhall Street, Portland, ME 04102, 207.662.0514, mallol@mmc.org

Editor's Note: The online version of this article contains the SBP-3 pediatric subcompetency with milestone levels.

^✉

Corresponding author.

PMCID: PMC6476092 PMID: 31024648

Abstract

Background

The Accreditation Council for Graduate Medical Education (ACGME) Milestone projects required each specialty to identify essential skills and develop means of assessment with supporting validity evidence for trainees. Several specialties rate trainees on a milestone subcompetency related to working in interprofessional teams. A tool to assess trainee competence in any role on an interprofessional team in a variety of scenarios would be valuable and suitable for simulation-based assessment.

Objective

We developed a tool for simulation settings that assesses interprofessional teamwork in trainees.

Methods

In 2015, existing tools that assess teamwork or interprofessionalism using direct observation were systematically reviewed for appropriateness, generalizability, adaptability, ease of use, and resources required. Items from these tools were included in a Delphi method with multidisciplinary pediatrics experts using an iterative process from June 2016 to January 2017 to develop an assessment tool.

Results

Thirty-one unique tools were identified. A 2-stage review narrowed this list to 5 tools, and 81 items were extracted. Twenty-two pediatrics experts participated in 4 rounds of Delphi surveys, with response rates ranging from 82% to 100%. Sixteen items reached consensus for inclusion in the final tool. A global 4-point rating scale from novice to proficient was developed.

Conclusions

A novel tool to assess interprofessional teamwork for individual trainees in a simulated setting was developed using a systematic review and Delphi methodology. This is the first step to establish the validity evidence necessary to use this tool for competency-based assessment.

What was known and gap

The ACGME Milestone projects have created a need for additional competency-based assessment instruments to be used in different settings, including simulation.

What is new

An assessment tool, which can be used for multiple event types, to assess the competence of an individual in an interprofessional team.

Limitations

The tool was developed for a single specialty, reducing generalizable. The instrument has not been implemented and lacks validity evidence.

Bottom line

A 16-item instrument to evaluate key elements of interprofessional teamwork for individual trainees was created using a modified Delphi method.

Introduction

In the Accreditation Council for Graduate Medical Education (ACGME) Milestones projects, residents are assessed semiannually along specialty-specific competencies, from novice to mastery,¹ creating a need for additional competency-based assessment instruments with evidence of validity for different settings including simulation.²

A 2016 study surveyed simulation experts and pediatrics program directors to identify priority areas for developing simulation-based assessment tools.³ Respondents identified which of the 21 pediatrics milestone subcompetencies were the most difficult to assess using traditional methods and which were best suited to simulation-based assessment. Systems-based practice 3 (“Work in interprofessional teams to enhance patient safety and improve patient care quality”) emerged as 1 of the 3 subcompetencies best suited to simulation-based assessment.³ While many teamwork assessment tools exist, most of these tools assess either the leader alone or the team as a whole, and focus on acute events.

The goal of this study was to develop a tool independent of event type or acuity that could be used to assess the competence of not just the team leader, but also an individual in any role on a pediatric team.

Methods

Our team included simulation experts from pediatric emergency medicine, pediatric hospital medicine, and graduate medical education from 7 institutions across North America (The Hospital for Sick Children, Boston University School of Medicine, Feinberg School of Medicine, Northeast Ohio Medical University, The University of Alabama at Birmingham, The George Washington University School of Medicine and Health Sciences, and Tufts University School of Medicine). All team members are pediatricians, medical educators, and members of the International Network for Simulation-based Pediatric Innovation, Research, and Education (INSPIRE). Our initial literature review found many tools assessing the skills of a team leader or the function of a team as a whole; however, it failed to identify a tool to assess an individual working in an interprofessional team in a role other than team leader. We conducted a systematic review and used Delphi methodology to achieve expert consensus to develop a new tool.⁴^–⁶

Systematic Review

With a medical librarian, we conducted a systematic review to identify existing tools assessing interprofessionalism and/or teamwork using direct observation in a clinical or simulated setting. We searched PubMed, Ovid, and MedEdPORTAL databases, as well as unpublished and grey literature. Search terms included teamwork, simulation, assessment, interprofessional collaboration, interprofessional teamwork, and physician-nurse relations. The search included publications through October 2015. Titles and abstracts were screened by members of our study team.

Identified tools were independently reviewed by 2 investigators for inclusion in 2 stages. First, a tool was evaluated to determine whether it (1) assessed teamwork (including nontechnical elements); (2) was generalizable across scenarios; (3) was adaptable for use assessing individual performance; and (4) met ACGME criteria of “ease of use” (ie, is easily carried or accessed, requires minimal setup, and is completed in under 20 minutes) and “resources required” (ie, requires no resources beyond documentation tools, assessor training is less than an hour, and an individual assessor is sufficient for evaluation).⁷ Tools were independently nominated for inclusion in the second stage by the 2 reviewers. We used study group consensus to resolve discrepancies between reviewers.

In the second stage, we rated the nominated tools based on difficulty of adaptation (1, no adaptation necessary/ready for use, to 7, extensive adaptation necessary/multiple changes of multiple elements needed/would require excessive effort, time, or work). We had an a priori plan to include tools with an “adaptability rating” ≤ 3 (few changes needed/easily made with minimal effort, time, or work). We selected tools with an adaptability rating ≤ 3 to review.

From the selected tools, we first extracted items related to interprofessional teamwork. Similar items were combined and nonapplicable items (eg, specific procedural skills/unable to assess with simulation) were eliminated. Remaining items were included in the Delphi process, which was conducted between June 2016 and January 2017.

Modified Delphi Exercise

Our research team invited outside experts in simulation, teamwork, team performance, interprofessionalism, and assessment to participate in the Delphi panel through INSPIRE. The panel was reviewed and revised to ensure multidisciplinary membership that included physicians, nurses, physiotherapists, occupational therapists, and respiratory therapists, as well as geographic diversity across North America. We only invited experts who work in pediatrics, as we focused on a pediatric subcompetency.

Panelists were provided with a brief background on the Milestone projects, the original milestones (available as online supplemental material), how the list of elements was identified, and the likely steps of the project. The initial Delphi round was piloted with 3 nonpanelist members.⁸

In the first round, panelists were asked to rate each item on necessity of inclusion using a 7-point scale (1, unnecessary to include, to 7, definitely must include). In addition, panelists could offer suggestions on each element and generate additional items. After elements were edited or removed, panelists were instructed to rate items again on necessity of inclusion using the same scale in a second round. In the third round, items with similar concepts were grouped and panelists selected the “best” item in each group. In the fourth round, the panelists rated each item a final time on the 7-point necessity of inclusion scale.

To simplify the Delphi process, anchors were not included in the initial rounds. In the final round, the original elements' anchors were reviewed, noting whether descriptive anchors, brief anchors, or no anchors were provided. The study team created 2 types of anchors: descriptive and terse. Descriptive anchors were consistent with associated questions and utilized milestone ratings language. Terse anchors were based on frequency (never, occasionally, frequently, and consistently), but also included elements from each question. All anchors included only 4 levels, as the pediatrics milestone differentiates levels 1 to 4 and excludes level 5. In the fourth Delphi round, participants selected the preferred anchor option for each of the items.

This project was deemed exempt by the Maine Medical Center Institutional Review Board.

Data Analysis

All data were recorded through the REDCap data management system and maintained anonymously for analysis. Consensus for inclusion was defined as a median ≥ 6 on the 7-point rating scale. For round 3, selecting the best item within a group of similar items, the item selected most frequently was included in the next round. Any tied items were voted on anonymously by the authors.

Results

We identified 59 teamwork articles and 23 interprofessionalism articles by systematic review. Thirty-one unique tools were identified and underwent further review.⁹^–³⁹ Fifteen tools met initial inclusion criteria after the initial stage (Table 1).¹⁷^–³¹ After the second stage of review for adaptability of the tool, 5 tools remained. A total of 157 items were extracted and reduced to 81 discrete items for rating by Delphi panelists (Figure).

Table 1.

Interprofessional and Teamwork Tools Meeting Initial Inclusion Criteria

Reference	Subjects Assessed	Tool Structure
Cooper et al¹⁰	Emergency department code teams	• 11 items in 3 domains followed by global rating • 5-point anchored observational scale and global rating
Curran et al¹¹	Developed/validated with literature review and Delphi, later papers modified and piloted with residents	• 31 items in 6 domains • 4-point Likert scale
Flowerdew et al¹²	Emergency department clinicians	• 12 items in 4 domains • 9-point anchored Likert scale
Frankel et al¹³	Various teams (surgical, medical, obstetric), emergent and nonemergent	• 21 items in 4 domains • 3-point checklist (weighted)
Guise et al¹⁴	Obstetric teams	• 15 items in 5 domains • 11-point Likert scale
Kim et al¹⁵	Residents	• Ottawa GRS: 7-point anchored scale, 5 categories • Ottawa CRM: 12 items in 5 domains
Lambden et al¹⁶	Residents in ICU setting; assesses team leader only	• 32 items in 8 domains • 7-point anchored Likert scale
Malec et al¹⁷	Residents and nurses	• 16 items • 3-point behavioral checklist
Mishra et al¹⁸	Surgical teams	• 16 items in 4 domains (separate evaluation of all elements for surgical team, anesthesia team, nursing team) • 4-point Likert scale
Olupeliyawa et al¹⁹	Medical students	• 6 items in 3 domains • 5-point Likert scale
Reid et al²⁰	Pediatric resident and expert teams	• 95 total items • 3-point behavioral checklist (weighted)
Sigalet et al²¹	Medical, nursing, and respiratory therapy students	• 12 items • 5-point behaviorally anchored scale
Steinemann et al²²	Emergency department trauma teams	• 27 items in 5 domains • 5-point anchored Likert scale
Sutton et al²³	Hospital resuscitation teams	• 61 items in 3 domains • 7-point Likert scale
Wright et al²⁴	Medical and nursing students	• 6 domains • 3-point Likert scale

Open in a new tab

Note: Items from these tools in the bolded studies were extracted for the Delphi process.

Abbreviations: GRS, global rating scale; CRM, crisis resource management; ICU, intensive care unit.

Flowchart From Total Number of Articles Reviewed to Final Tools and Items Considered

Twenty-two of the 30 invited experts from 5 interprofessional domains participated in the Delphi process. Panelists included 9 nurses, 8 physicians, 3 physical/respiratory therapists, and 2 nonclinical educators. Their areas of expertise included simulation, patient safety, organizational behavior, and interprofessionalism. Four rounds of surveys were completed with response rates ranging from 82% to 100%. The 81 initial items were reduced to 16 items that reached consensus (Table 2). A 4-point global rating scale ranging from novice to proficient was selected by the Delphi panel for each item. The final instrument is shown in Table 3.

Table 2.

Summary of Delphi Exercise

Round	Participants	Elements at Start	Elements After Analysis
1	22	81	62
2	18	62	48
3	21	48	18
4	20	18	16

Open in a new tab

Table 3.

Simulation-Based Interprofessional Teamwork Assessment Tool

Question	Novice	Beginner	Competent	Proficient	Not Observed/Not Able to Evaluate
1. Communicates need for assistance as necessary.	Does not communicate need for assistance.	Occasionally communicates need for assistance.	Communicates need for assistance most of the time.	Consistently communicates need for assistance.	Not observed/ not able to evaluate.
2. Accepts responsibility.	Does not accept responsibility for the failure of collaborative goals.	Occasionally accepts responsibility for the failure of collaborative goals.	Frequently accepts responsibility for the failure of collaborative goals.	Consistently accepts responsibility for the failure of collaborative goals.	Not observed/ not able to evaluate.
3. Introduces self by name and role.	Does not introduce self by name or role.	Partial and/or inaudible introduction of self or role. Introduction occurs only with prompting.	Complete and audible introduction of self and role. Introduction may need prompting.	Complete and audible introduction of self and role. Introduction occurs without prompting.	Not observed/ not able to evaluate.
4. Team roles defined and maintained.	Team roles undefined throughout.	Leader defined initially, but not maintained. Few/no participant roles defined. Role change is disorganized and unclear.	Leader defined and maintained for most of the event. Most participant roles defined. Role change is disorganized and unclear.	Leader and participant roles defined and maintained throughout. Role change is organized and clear.	Not observed/ not able to evaluate.
5. Shares mental model with team efficiently and concisely at onset and as change occurs.	Does not share initial mental model with team. Does not adapt model as change occurs.	Shares initial mental model, but not concisely. Occasionally adapts model as change occurs.	Shares initial mental model concisely. Frequently adapts model as change occurs.	Shares initial mental model concisely. Consistently adapts model as change occurs.	Not observed/ not able to evaluate.
6. Shows an organized and efficient problem-solving approach.	Problem solving is disorganized, inefficient.	Problem solving is slow, sequential.	Problem solving is organized, concurrent.	Problem solving is organized, concurrent, and considers alternatives.	Not observed/ not able to evaluate.
7. Communicates patient parameters and/or changes in clinical condition.	Never communicates patient parameters and/or clinical change.	Occasionally communicates some parameters and/or clinical changes.	Frequently communicates some parameters and clinical changes.	Consistently communicates most parameters and clinical changes.	Not observed/ not able to evaluate.
8. Seeks advice and expertise of others.	Does not ask for input from others.	Occasionally asks for input from others.	Regularly seeks input from team members.	Proactively seeks input from multiple team members.	Not observed/ not able to evaluate
9. Effectively facilitates discussions among team members.	Does not engage in discussions among team members.	Occasionally engages in discussions among team members.	Regularly engages in discussions among team members.	Proactively facilitates discussions among team members using brief, huddle, or debrief.	Not observed/ not able to evaluate.
10. Participates in a collaborative relationship with other team members.	Does not establish collaborative relationships with others.	Occasionally establishes collaborative relationships with others.	Frequently establishes collaborative relationships with others.	Consistently establishes collaborative relationships with others.	Not observed/ not able to evaluate.
11. Practices active listening through closed-loop communication.	Does not use closed-loop communication.	Occasionally uses closed-loop communication.	Frequently uses closed-loop communication.	Consistently uses closed-loop communication.	Not observed/ not able to evaluate.
12. Works with other team members to shift roles to address urgent/emergent events when appropriate.	Does not establish collaborative relationships with others.	Occasionally establishes collaborative relationships with others.	Frequently establishes collaborative relationships with others.	Consistently establishes collaborative relationships with others.	Not observed/ not able to evaluate.
13. Listens respectfully to the expressed needs of all team members, including patient and family, in delivering care.	Does not listen to needs of team members, including patient and family.	Occasionally listens to expressed needs of team members, including patient and family.	Frequently listens to team members in an active and respectful manner.	Consistently listens to team members in an active and respectful manner and ensures a common understanding of care decisions.	Not observed/ not able to evaluate.
14. Provides care in a way that is mindful of the patient and their family.	Does not interact with patient/family.	Several examples where communication or care do not respond to patient/family needs.	Communicates treatment plan to patient/family, but fails to respond appropriately to their needs.	Most communication is appropriate and mindful of patient/family.	Not observed/ not able to evaluate.
15. Is open to opinions from other team members.	Does not acknowledge opinions of other team members.	Acknowledges some team member opinions, but ignores others.	Responds respectfully, but does not solicit others' opinions.	Solicits and responds respectfully to others' opinions.	Not observed/ not able to evaluate.
16. Contributes to team debriefing	Does not share information with the team. When asked, does not give input. Does not volunteer ideas despite prompting.	Shares information inconsistently with the team. When asked, gives input occasionally. Volunteers ideas occasionally with prompting only.	Shares information frequently with the team. When asked, gives input. Volunteers ideas with prompting.	Shares information consistently with the team. When asked, gives input. Volunteers ideas without prompting. Will give feedback if warranted.	Not observed/ not able to evaluate.

Open in a new tab

Discussion

Our systematic review found 31 unique tools to evaluate teamwork, many of them limited by not being applicable to any individual member of a team and by their length. Using these existing tools and a modified Delphi process with interprofessional pediatrics experts, we created a 16-item novel tool to assess interprofessional teamwork in a simulated setting.

Our tool sought to adhere to Grand and colleague's 4-step framework,⁴⁰ striving to be specific and capture observable individual behaviors as much as possible, thus distinguishing team process (team members' nontechnical skills) and team performance (including activities linked to a task or outcome). The initial 2 steps describe creating items for the assessment tool and distinguishing team process versus performance. The final 2 steps detailed guidelines for tool validation and implementation, the next phases of our project. We have started this process through outside cognitive interviewing with simulation and assessment experts not involved in the development process and initial piloting of the tool at selected sites. Preliminary feedback indicated that the tool is intuitive and easy to use. As we extracted items from 3 tools in other specialties for the Delphi panel, it is possible that this tool could have crossover to other disciplines.

Midway through our project, Thistlewaite et al reported their experience developing a tool for the assessment of individual performance on an interprofessional team (iToft).⁴¹ While this tool may meet the need initially identified by our team, validity evidence is not yet available. Our process and tool differ in the focus on simulation-based assessment and the inclusion of simulation experts in our Delphi panel. The overlap between the 2 tools lends support to them both, despite a lack of published validity evidence for either tool currently. While most items in the iToft tool are also represented within our tool, anchor scales between the 2 tools differ. The iToft uses a scale with 4 options: not applicable, inappropriate, appropriate, and responsive. Thistlewaite and colleagues handled novice versus advanced differentiation by using 2 different tools: a basic version and an expert version. For our tool, Delphi experts opted rather to use a descriptive anchor scale to mark progression from novice to proficient to better mirror the ACGME Milestones format as well as trends in competency-based medical education.

The study may be limited by our choice of experts: while we included an interdisciplinary group with varied expertise, it is possible our expert panel did not fully represent all stakeholders. Our tool was developed by pediatricians for use to evaluate a pediatrics milestone. While the tool was not intended to be generalizable beyond the pediatric context, interprofessional teamwork is not unique to pediatrics, and many other specialties have competencies in interprofessional teamwork. Further evaluation in other clinical specialties would be required before use in these contexts. Second, as previously noted, teamwork occurs in acute and nonacute situations. We sought to develop a single tool appropriate for use in varied contexts, and initial elements included in Delphi round 1 came from tools designed to assess acute and nonacute situations as well as both simulated and nonsimulated settings. It is possible that in practice the final elements translate with different effectiveness in varied contexts. Third, in analyzing each assessment tool identified by our systematic review, we utilized ACGME criteria to determine the quality of the assessment method using 2 reviewers for each tool, but some of these elements still have a component of subjectivity.¹³ Maintaining methodological rigor through the course of tool development and ending up with a practical tool that can be easily implemented is a challenging balance. Finally, the instrument has not yet been applied to assessment of trainees. This will be required to show validity evidence for the assessment.⁴²

Next steps involve formally evaluating the tool using the ACGME standards for evaluating the quality of assessment methods to assess reliability, validity, ease of use, resources required, ease of interpretation, and educational impact.⁷ Only with further testing will we know whether we have achieved our objective.

Conclusion

While existing teamwork tools assess either leadership of an individual or team function as a whole, particularly in acute care events, we found none were designed to assess the competence of an individual trainee working as a member of an interprofessional team in a non-leader role. Using a modified Delphi process, we developed a new 16-item instrument to evaluate key elements of this milestone in a comprehensive way.

Supplementary Material

Click here for additional data file.^{(97.6KB, pdf)}

References

1.Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system—rationale and benefits. N Engl J Med. 2012;366(11):1051–1056. doi: 10.1056/NEJMsr1200117. [DOI] [PubMed] [Google Scholar]
2.Carraccio CL, Englander R. From Flexner to competencies: reflections on a decade and the journey ahead. Acad Med. 2013;88(8):1067–1073. doi: 10.1097/ACM.0b013e318299396f. [DOI] [PubMed] [Google Scholar]
3.Mallory LA, Calaman S, Lee White M, Doughty C, Mangold K, Lopreiato J, et al. Targeting simulation-based assessment for the pediatric milestones: a survey of simulation experts and program directors. Acad Pediatr. 2016;16(3):290–297. doi: 10.1016/j.acap.2015.09.007. [DOI] [PubMed] [Google Scholar]
4.de Villiers MR, de Villiers PJ, Kent AP. The Delphi technique in health sciences education research. Med Teach. 2005;27(7):639–643. doi: 10.1080/13611260500069947. [DOI] [PubMed] [Google Scholar]
5.Graham B, Regehr G, Wright JG. Delphi as a method to establish consensus for diagnostic criteria. J Clin Epidemiol. 2003;56(12):1150–1156. doi: 10.1016/s0895-4356(03)00211-7. [DOI] [PubMed] [Google Scholar]
6.Glenn JC, Gordon TJ. Futures Research Methdology—Version 3.0. Washington, DC: The Millennium Project; 2009. [Google Scholar]
7.Swing SR, Clyman SG, Holmboe ES, Williams RG. Advancing resident assessment in graduate medical education. J Grad Med Educ. 2009;1(2):278–286. doi: 10.4300/JGME-D-09-00010.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
8.Humphrey-Murto S, Varpio L, Wood TJ, Gonsalves C, Ufholz LA, Mascioli K, et al. The use of the Delphi and other consensus group methods in medical education research: a review. Acad Med. 2017;92(10):1491–1498. doi: 10.1097/ACM.0000000000001812. [DOI] [PubMed] [Google Scholar]
9.Grant EC, Grant VJ, Bhanji F, Duff JP, Cheng A, Lockyer JM. The development and assessment of an evaluation tool for pediatric resident competence in leading simulated pediatric resuscitations. Resuscitation. 2012;83(7):887–893. doi: 10.1016/j.resuscitation.2012.01.015. [DOI] [PubMed] [Google Scholar]
10.Cooper S, Cant R, Porter J, Sellick K, Somers G, Kinsman L, et al. Rating medical emergency teamwork performance: development of the Team Emergency Assessment Measure (TEAM) Resuscitation. 2010;81(4):446–452. doi: 10.1016/j.resuscitation.2009.11.027. [DOI] [PubMed] [Google Scholar]
11.Curran V, Hollett A, Casimiro LM, Mccarthy P, Banfield V, Hall P, et al. Development and validation of the interprofessional collaborator assessment rubric (ICAR) J Interprof Care. 2011;25(5):339–344. doi: 10.3109/13561820.2011.589542. [DOI] [PubMed] [Google Scholar]
12.Flowerdew L, Brown R, Vincent C, Woloshynowych M. Development and validation of a tool to assess emergency physicians' nontechnical skills. Ann Emerg Med. 2012;59(5):376–385.e4. doi: 10.1016/j.annemergmed.2011.11.022. [DOI] [PubMed] [Google Scholar]
13.Frankel A, Gardner R, Maynard L, Kelly A. Using the communication and teamwork skills (CATS) assessment to measure health care team performance. Jt Comm J Qual Patient Saf. 2007;33(9):549–558. doi: 10.1016/s1553-7250(07)33059-6. [DOI] [PubMed] [Google Scholar]
14.Guise JM, Deering SH, Kanki BG, Osterweil P, Li H, Mori M, et al. Validation of a tool to measure and promote clinical teamwork. Simul Healthc. 2008;3(4):217–223. doi: 10.1097/SIH.0b013e31816fdd0a. [DOI] [PubMed] [Google Scholar]
15.Kim J, Neilipovitz D, Cardinal P, Chiu M. A comparison of global rating scale and checklist scores in the validation of an evaluation tool to assess performance in the resuscitation of critically ill patients during simulated emergencies (abbreviated as “CRM simulator study IB”) Simul Healthc. 2009;4(1):6–16. doi: 10.1097/SIH.0b013e3181880472. [DOI] [PubMed] [Google Scholar]
16.Lambden S, DeMunter C, Dowson A, Cooper M, Gautama S, Sevdalis N. The Imperial Paediatric Emergency Training Toolkit (IPETT) for use in paediatric emergency training: development and evaluation of feasibility and validity. Resuscitation. 2013;84(6):831–836. doi: 10.1016/j.resuscitation.2012.11.013. [DOI] [PubMed] [Google Scholar]
17.Malec JF, Torsher LC, Dunn WF, Wiegmann DA, Arnold JJ, Brown DA, et al. The mayo high performance teamwork scale: reliability and validity for evaluating key crew resource management skills. Simul Healthc. 2007;2(1):4–10. doi: 10.1097/SIH.0b013e31802b68ee. [DOI] [PubMed] [Google Scholar]
18.Mishra A, Catchpole K, McCulloch P. The Oxford NOTECHS System: reliability and validity of a tool for measuring teamwork behaviour in the operating theatre. Qual Saf Health Care. 2009;18(2):104–108. doi: 10.1136/qshc.2007.024760. [DOI] [PubMed] [Google Scholar]
19.Olupeliyawa AM, O'Sullivan AJ, Hughes C, Balasooriya CD. The Teamwork Mini-Clinical Evaluation Exercise (T-MEX): a workplace-based assessment focusing on collaborative competencies in health care. Acad Med. 2014;89(2):359–365. doi: 10.1097/ACM.0000000000000115. [DOI] [PubMed] [Google Scholar]
20.Reid J, Stone K, Brown J, Caglar D, Kobayashi A, Lewis-Newby M, et al. The Simulation Team Assessment Tool (STAT): development, reliability and validation. Resuscitation. 2012;83(7):879–886. doi: 10.1016/j.resuscitation.2011.12.012. [DOI] [PubMed] [Google Scholar]
21.Sigalet E, Donnon T, Cheng A, Cooke S, Robinson T, Bissett W, et al. Development of a team performance scale to assess undergraduate health professionals. Acad Med. 2013;88(7):989–996. doi: 10.1097/ACM.0b013e318294fd45. [DOI] [PubMed] [Google Scholar]
22.Steinemann S, Berg B, DiTullio A, Skinner A, Terada K, Anzelon K, et al. Assessing teamwork in the trauma bay: introduction of a modified “NOTECHS” scale for trauma. Am J Surg. 2012;203(1):69–75. doi: 10.1016/j.amjsurg.2011.08.004. [DOI] [PubMed] [Google Scholar]
23.Sutton G, Liao J, Jimmieson NL, Restubog SL. Measuring ward-based multidisciplinary healthcare team functioning: a validation study of the Team Functioning Assessment Tool (TFAT) J Healthc Qual. 2013;35(4):36–49. doi: 10.1111/jhq.12010. [DOI] [PubMed] [Google Scholar]
24.Wright MC, Segall N, Hobbs G, Phillips-Bute B, Maynard L, Taekman JM. Standardized assessment for evaluation of team skills: validity and feasibility. Simul Healthc. 2013;8(5):292–303. doi: 10.1097/SIH.0b013e318290a022. [DOI] [PMC free article] [PubMed] [Google Scholar]
25.Brett-Fleegler MB, Vinci RJ, Weiner DL, Harris SK, Shih MC, Kleinman ME. A simulator-based tool that assesses pediatric resident resuscitation competency. Pediatrics. 2008;121(3):e597–e603. doi: 10.1542/peds.2005-1259. [DOI] [PubMed] [Google Scholar]
26.Calhoun AW, Boone M, Miller KH, Taulbee RL, Montgomery VL, Boland K. A multirater instrument for the assessment of simulated pediatric crises. J Grad Med Educ. 2011;3(1):88–94. doi: 10.4300/JGME-D-10-00052.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
27.Clancy CM. TeamSTEPPS: optimizing teamwork in the perioperative setting. AORN J. 2007;86(1):18–22. doi: 10.1016/j.aorn.2007.06.008. [DOI] [PubMed] [Google Scholar]
28.Cicero MX, Riera A, Northrup V, Auerbach M, Pearson K, Baum CR. Design, validity, and reliability of a pediatric resident JumpSTART disaster triage scoring instrument. Acad Pediatr. 2013;13(1):48–54. doi: 10.1016/j.acap.2012.09.002. [DOI] [PubMed] [Google Scholar]
29.Donoghue A, Nishisaki A, Sutton R, Hales R, Boulet J. Reliability and validity of a scoring instrument for clinical performance during Pediatric Advanced Life Support simulation scenarios. Resuscitation. 2010;81(3):331–336. doi: 10.1016/j.resuscitation.2009.11.011. [DOI] [PubMed] [Google Scholar]
30.Gilfoyle E, Gottesman R, Razack S. Development of a leadership skills workshop in paediatric advanced resuscitation. Med Teach. 2007;29(9):e276–e283. doi: 10.1080/01421590701663287. [DOI] [PubMed] [Google Scholar]
31.Lurie SJ, Schultz SH, Lamanna G. Assessing teamwork: a reliable five-question survey. Fam Med. 2011;43(10):731–734. [PubMed] [Google Scholar]
32.Nadler I, Sanderson PM, Liley HG. The accuracy of clinical assessments as a measure for teamwork effectiveness. Simul Healthc. 2011;6(5):260–268. doi: 10.1097/SIH.0b013e31821eaa38. [DOI] [PubMed] [Google Scholar]
33.Nishisaki A, Donoghue AJ, Colborn S, Watson C, Meyer A, Niles D, et al. Development of an instrument for a primary airway provider's performance with an ICU multidisciplinary team in pediatric respiratory failure using simulation. Respir Care. 2012;57(7):1121–1128. doi: 10.4187/respcare.01472. [DOI] [PubMed] [Google Scholar]
34.Orchard CA, King GA, Khalili H, Bezzina MB. Assessment of Interprofessional Team Collaboration Scale (AITCS): development and testing of the instrument. J Contin Educ Health Prof. 2012;32(1):58–67. doi: 10.1002/chp.21123. [DOI] [PubMed] [Google Scholar]
35.Patterson PD, Weaver MD, Weaver SJ, Rosen MA, Todorova G, Weingart LR, et al. Measuring teamwork and conflict among emergency medical technician personnel. Prehosp Emerg Care. 2012;16(1):98–108. doi: 10.3109/10903127.2011.616260. [DOI] [PMC free article] [PubMed] [Google Scholar]
36.Plant JL, van Schaik SM, Sliwka DC, Boscardin CK, O'Sullivan PS. Validation of a self-efficacy instrument and its relationship to performance of crisis resource management skills. Adv Health Sci Educ Theory Pract. 2011;16(5):579–590. doi: 10.1007/s10459-011-9274-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
37.Russ S, Hull L, Rout S, Vincent C, Darzi A, Sevdalis N. Observational teamwork assessment for surgery: feasibility of clinical and nonclinical assessor calibration with short-term training. Ann Surg. 2012;255(4):804–809. doi: 10.1097/SLA.0b013e31824a9a02. [DOI] [PubMed] [Google Scholar]
38.Schildmeijer K, Nilsson L, Arestedt K, Perk J. Assessment of adverse events in medical care: lack of consistency between experienced teams using the global trigger tool. BMJ Qual Saf. 2012;21(4):307–314. doi: 10.1136/bmjqs-2011-000279. [DOI] [PubMed] [Google Scholar]
39.Walker S, Brett S, McKay A, Lambden S, Vincent C, Sevdalis N. Observational Skill-based Clinical Assessment tool for Resuscitation (OSCAR): development and validation. Resuscitation. 2011;82(7):835–844. doi: 10.1016/j.resuscitation.2011.03.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
40.Grand JA, Pearce M, Rench TA, Chao GT, Fernandez R, Kozlowski SW. Going DEEP: guidelines for building simulation-based team assessments. BMJ Qual Saf. 2013;22(5):436–448. doi: 10.1136/bmjqs-2012-000957. [DOI] [PubMed] [Google Scholar]
41.Thistlethwaite J, Dallest K, Moran M, Dunston R, Roberts C, Eley D, et al. Introducing the individual Teamwork Observation and Feedback Tool (iTOFT): Development and description of a new interprofessional teamwork measure. J Interprof Care. 2016;30(4):526–528. doi: 10.3109/13561820.2016.1169262. [DOI] [PubMed] [Google Scholar]
42.Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: a practical guide to Kane's framework. Med Educ. 2015;49(6):560–575. doi: 10.1111/medu.12678. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Click here for additional data file.^{(97.6KB, pdf)}

[i1949-8357-11-2-168-b01] 1.Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system—rationale and benefits. N Engl J Med. 2012;366(11):1051–1056. doi: 10.1056/NEJMsr1200117. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b02] 2.Carraccio CL, Englander R. From Flexner to competencies: reflections on a decade and the journey ahead. Acad Med. 2013;88(8):1067–1073. doi: 10.1097/ACM.0b013e318299396f. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b03] 3.Mallory LA, Calaman S, Lee White M, Doughty C, Mangold K, Lopreiato J, et al. Targeting simulation-based assessment for the pediatric milestones: a survey of simulation experts and program directors. Acad Pediatr. 2016;16(3):290–297. doi: 10.1016/j.acap.2015.09.007. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b04] 4.de Villiers MR, de Villiers PJ, Kent AP. The Delphi technique in health sciences education research. Med Teach. 2005;27(7):639–643. doi: 10.1080/13611260500069947. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b05] 5.Graham B, Regehr G, Wright JG. Delphi as a method to establish consensus for diagnostic criteria. J Clin Epidemiol. 2003;56(12):1150–1156. doi: 10.1016/s0895-4356(03)00211-7. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b06] 6.Glenn JC, Gordon TJ. Futures Research Methdology—Version 3.0. Washington, DC: The Millennium Project; 2009. [Google Scholar]

[i1949-8357-11-2-168-b07] 7.Swing SR, Clyman SG, Holmboe ES, Williams RG. Advancing resident assessment in graduate medical education. J Grad Med Educ. 2009;1(2):278–286. doi: 10.4300/JGME-D-09-00010.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b08] 8.Humphrey-Murto S, Varpio L, Wood TJ, Gonsalves C, Ufholz LA, Mascioli K, et al. The use of the Delphi and other consensus group methods in medical education research: a review. Acad Med. 2017;92(10):1491–1498. doi: 10.1097/ACM.0000000000001812. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b09] 9.Grant EC, Grant VJ, Bhanji F, Duff JP, Cheng A, Lockyer JM. The development and assessment of an evaluation tool for pediatric resident competence in leading simulated pediatric resuscitations. Resuscitation. 2012;83(7):887–893. doi: 10.1016/j.resuscitation.2012.01.015. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b10] 10.Cooper S, Cant R, Porter J, Sellick K, Somers G, Kinsman L, et al. Rating medical emergency teamwork performance: development of the Team Emergency Assessment Measure (TEAM) Resuscitation. 2010;81(4):446–452. doi: 10.1016/j.resuscitation.2009.11.027. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b11] 11.Curran V, Hollett A, Casimiro LM, Mccarthy P, Banfield V, Hall P, et al. Development and validation of the interprofessional collaborator assessment rubric (ICAR) J Interprof Care. 2011;25(5):339–344. doi: 10.3109/13561820.2011.589542. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b12] 12.Flowerdew L, Brown R, Vincent C, Woloshynowych M. Development and validation of a tool to assess emergency physicians' nontechnical skills. Ann Emerg Med. 2012;59(5):376–385.e4. doi: 10.1016/j.annemergmed.2011.11.022. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b13] 13.Frankel A, Gardner R, Maynard L, Kelly A. Using the communication and teamwork skills (CATS) assessment to measure health care team performance. Jt Comm J Qual Patient Saf. 2007;33(9):549–558. doi: 10.1016/s1553-7250(07)33059-6. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b14] 14.Guise JM, Deering SH, Kanki BG, Osterweil P, Li H, Mori M, et al. Validation of a tool to measure and promote clinical teamwork. Simul Healthc. 2008;3(4):217–223. doi: 10.1097/SIH.0b013e31816fdd0a. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b15] 15.Kim J, Neilipovitz D, Cardinal P, Chiu M. A comparison of global rating scale and checklist scores in the validation of an evaluation tool to assess performance in the resuscitation of critically ill patients during simulated emergencies (abbreviated as “CRM simulator study IB”) Simul Healthc. 2009;4(1):6–16. doi: 10.1097/SIH.0b013e3181880472. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b16] 16.Lambden S, DeMunter C, Dowson A, Cooper M, Gautama S, Sevdalis N. The Imperial Paediatric Emergency Training Toolkit (IPETT) for use in paediatric emergency training: development and evaluation of feasibility and validity. Resuscitation. 2013;84(6):831–836. doi: 10.1016/j.resuscitation.2012.11.013. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b17] 17.Malec JF, Torsher LC, Dunn WF, Wiegmann DA, Arnold JJ, Brown DA, et al. The mayo high performance teamwork scale: reliability and validity for evaluating key crew resource management skills. Simul Healthc. 2007;2(1):4–10. doi: 10.1097/SIH.0b013e31802b68ee. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b18] 18.Mishra A, Catchpole K, McCulloch P. The Oxford NOTECHS System: reliability and validity of a tool for measuring teamwork behaviour in the operating theatre. Qual Saf Health Care. 2009;18(2):104–108. doi: 10.1136/qshc.2007.024760. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b19] 19.Olupeliyawa AM, O'Sullivan AJ, Hughes C, Balasooriya CD. The Teamwork Mini-Clinical Evaluation Exercise (T-MEX): a workplace-based assessment focusing on collaborative competencies in health care. Acad Med. 2014;89(2):359–365. doi: 10.1097/ACM.0000000000000115. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b20] 20.Reid J, Stone K, Brown J, Caglar D, Kobayashi A, Lewis-Newby M, et al. The Simulation Team Assessment Tool (STAT): development, reliability and validation. Resuscitation. 2012;83(7):879–886. doi: 10.1016/j.resuscitation.2011.12.012. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b21] 21.Sigalet E, Donnon T, Cheng A, Cooke S, Robinson T, Bissett W, et al. Development of a team performance scale to assess undergraduate health professionals. Acad Med. 2013;88(7):989–996. doi: 10.1097/ACM.0b013e318294fd45. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b22] 22.Steinemann S, Berg B, DiTullio A, Skinner A, Terada K, Anzelon K, et al. Assessing teamwork in the trauma bay: introduction of a modified “NOTECHS” scale for trauma. Am J Surg. 2012;203(1):69–75. doi: 10.1016/j.amjsurg.2011.08.004. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b23] 23.Sutton G, Liao J, Jimmieson NL, Restubog SL. Measuring ward-based multidisciplinary healthcare team functioning: a validation study of the Team Functioning Assessment Tool (TFAT) J Healthc Qual. 2013;35(4):36–49. doi: 10.1111/jhq.12010. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b24] 24.Wright MC, Segall N, Hobbs G, Phillips-Bute B, Maynard L, Taekman JM. Standardized assessment for evaluation of team skills: validity and feasibility. Simul Healthc. 2013;8(5):292–303. doi: 10.1097/SIH.0b013e318290a022. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b25] 25.Brett-Fleegler MB, Vinci RJ, Weiner DL, Harris SK, Shih MC, Kleinman ME. A simulator-based tool that assesses pediatric resident resuscitation competency. Pediatrics. 2008;121(3):e597–e603. doi: 10.1542/peds.2005-1259. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b26] 26.Calhoun AW, Boone M, Miller KH, Taulbee RL, Montgomery VL, Boland K. A multirater instrument for the assessment of simulated pediatric crises. J Grad Med Educ. 2011;3(1):88–94. doi: 10.4300/JGME-D-10-00052.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b27] 27.Clancy CM. TeamSTEPPS: optimizing teamwork in the perioperative setting. AORN J. 2007;86(1):18–22. doi: 10.1016/j.aorn.2007.06.008. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b28] 28.Cicero MX, Riera A, Northrup V, Auerbach M, Pearson K, Baum CR. Design, validity, and reliability of a pediatric resident JumpSTART disaster triage scoring instrument. Acad Pediatr. 2013;13(1):48–54. doi: 10.1016/j.acap.2012.09.002. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b29] 29.Donoghue A, Nishisaki A, Sutton R, Hales R, Boulet J. Reliability and validity of a scoring instrument for clinical performance during Pediatric Advanced Life Support simulation scenarios. Resuscitation. 2010;81(3):331–336. doi: 10.1016/j.resuscitation.2009.11.011. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b30] 30.Gilfoyle E, Gottesman R, Razack S. Development of a leadership skills workshop in paediatric advanced resuscitation. Med Teach. 2007;29(9):e276–e283. doi: 10.1080/01421590701663287. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b31] 31.Lurie SJ, Schultz SH, Lamanna G. Assessing teamwork: a reliable five-question survey. Fam Med. 2011;43(10):731–734. [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b32] 32.Nadler I, Sanderson PM, Liley HG. The accuracy of clinical assessments as a measure for teamwork effectiveness. Simul Healthc. 2011;6(5):260–268. doi: 10.1097/SIH.0b013e31821eaa38. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b33] 33.Nishisaki A, Donoghue AJ, Colborn S, Watson C, Meyer A, Niles D, et al. Development of an instrument for a primary airway provider's performance with an ICU multidisciplinary team in pediatric respiratory failure using simulation. Respir Care. 2012;57(7):1121–1128. doi: 10.4187/respcare.01472. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b34] 34.Orchard CA, King GA, Khalili H, Bezzina MB. Assessment of Interprofessional Team Collaboration Scale (AITCS): development and testing of the instrument. J Contin Educ Health Prof. 2012;32(1):58–67. doi: 10.1002/chp.21123. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b35] 35.Patterson PD, Weaver MD, Weaver SJ, Rosen MA, Todorova G, Weingart LR, et al. Measuring teamwork and conflict among emergency medical technician personnel. Prehosp Emerg Care. 2012;16(1):98–108. doi: 10.3109/10903127.2011.616260. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b36] 36.Plant JL, van Schaik SM, Sliwka DC, Boscardin CK, O'Sullivan PS. Validation of a self-efficacy instrument and its relationship to performance of crisis resource management skills. Adv Health Sci Educ Theory Pract. 2011;16(5):579–590. doi: 10.1007/s10459-011-9274-7. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b37] 37.Russ S, Hull L, Rout S, Vincent C, Darzi A, Sevdalis N. Observational teamwork assessment for surgery: feasibility of clinical and nonclinical assessor calibration with short-term training. Ann Surg. 2012;255(4):804–809. doi: 10.1097/SLA.0b013e31824a9a02. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b38] 38.Schildmeijer K, Nilsson L, Arestedt K, Perk J. Assessment of adverse events in medical care: lack of consistency between experienced teams using the global trigger tool. BMJ Qual Saf. 2012;21(4):307–314. doi: 10.1136/bmjqs-2011-000279. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b39] 39.Walker S, Brett S, McKay A, Lambden S, Vincent C, Sevdalis N. Observational Skill-based Clinical Assessment tool for Resuscitation (OSCAR): development and validation. Resuscitation. 2011;82(7):835–844. doi: 10.1016/j.resuscitation.2011.03.009. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b40] 40.Grand JA, Pearce M, Rench TA, Chao GT, Fernandez R, Kozlowski SW. Going DEEP: guidelines for building simulation-based team assessments. BMJ Qual Saf. 2013;22(5):436–448. doi: 10.1136/bmjqs-2012-000957. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b41] 41.Thistlethwaite J, Dallest K, Moran M, Dunston R, Roberts C, Eley D, et al. Introducing the individual Teamwork Observation and Feedback Tool (iTOFT): Development and description of a new interprofessional teamwork measure. J Interprof Care. 2016;30(4):526–528. doi: 10.3109/13561820.2016.1169262. [DOI] [PubMed] [Google Scholar]

[i1949-8357-11-2-168-b42] 42.Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: a practical guide to Kane's framework. Med Educ. 2015;49(6):560–575. doi: 10.1111/medu.12678. [DOI] [PubMed] [Google Scholar]

PERMALINK

Development of a Simulation-Based Interprofessional Teamwork Assessment Tool

Zia Bismilla, MD, MEd

Tehnaz Boyle, MD, PhD

Karen Mangold, MD, MEd

Wendy Van Ittersum, MD

Marjorie Lee White, MD, MPPM, MA

Pavan Zaveri, MD, MEd

Leah Mallory, MD

Abstract

Background

Objective

Methods

Results

Conclusions

What was known and gap

What is new

Limitations

Bottom line

Introduction

Methods

Systematic Review

Modified Delphi Exercise

Data Analysis

Results

Table 1.

Figure.

Table 2.

Table 3.

Discussion

Conclusion

Supplementary Material

References

Associated Data

Supplementary Materials

ACTIONS

PERMALINK

RESOURCES

Cite

Add to Collections

PERMALINK

Development of a Simulation-Based Interprofessional Teamwork Assessment Tool

Zia Bismilla, MD, MEd

Tehnaz Boyle, MD, PhD

Karen Mangold, MD, MEd

Wendy Van Ittersum, MD

Marjorie Lee White, MD, MPPM, MA

Pavan Zaveri, MD, MEd

Leah Mallory, MD

Abstract

Background

Objective

Methods

Results

Conclusions

What was known and gap

What is new

Limitations

Bottom line

Introduction

Methods

Systematic Review

Modified Delphi Exercise

Data Analysis

Results

Table 1.

Figure.

Table 2.

Table 3.

Discussion

Conclusion

Supplementary Material

References

Associated Data

Supplementary Materials

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases