Better Decision-Making: Shared Mental Models and the Clinical Competency Committee

Abstract

Background

Shared mental models (SMMs) help groups make better decisions. Clinical competency committees (CCCs) can benefit from the development and use of SMMs in their decision-making as a way to optimize the quality and consistency of their decisions.

Objective

We reviewed the use of SMMs for decision making in graduate medical education, particularly their use in CCCs.

Methods

In May 2020, the authors conducted a narrative review of the literature related to SMMs. This review included the SMM related to teams, team functioning, CCCs, and graduate medical education.

Results

The literature identified the general use of SMMs, SMMs in graduate medical education, and strategies for building SMMs into the work of the CCC. Through the use of clear communication and guidelines, and a shared understanding of goals and expectations, CCCs can make better decisions. SMMs can be applied to Milestones, resident performance, assessment, and feedback.

Conclusions

To ensure fair and robust decision-making, the CCC must develop and maintain SMMs through excellent communication and understanding of expectations among members.

Introduction

When you think of your last clinical competency committee (CCC) meeting, did the group sometimes struggle with determining the Milestones level for a resident? Did different CCC members rate the same resident activity differently? There are many reasons for these challenges, but one strategy to help reduce these issues is through the creation of shared mental models (SMMs). The use of SMMs can help groups make better, more consistent decisions. CCCs make many decisions about learners that affect their advancement, learning plans, and professional development. SMMs of Milestones and the application of assessment tools can make these decisions clearer and more standardized.

The purpose of this article is to review the use of SMMs for decision making in graduate medical education (GME), especially the application to CCCs. In May 2020, the authors conducted a review of the literature related to SMMs using the terms “mental models,” “shared mental models,” “teams and team functioning,” “CCCs,” “Milestones,” and “GME.” Results of the search included the development and application of SMMs along with its benefits and challenges.

Research on SMMs focuses on team functioning.^1–5 An SMM refers to a team's common understanding of their task, interpretation of their environment, and required collaboration.^1,6 The Encyclopedia of Applied Psychology defines an SMM as: “shared understandings or representations of the goal of the team, individual team member tasks, and how team members will coordinate to achieve their common goals; individual team members can have varying degree of overlap or ‘sharedness' among their mental model of the team.”⁷

The growing interest in SMMs is prompted by their demonstrated benefits for team efficiency, functionality, and strategy.^1,3 In GME, the CCC is a required team.⁸ The learners are part of the larger CCC process and also benefit from SMMs related to the purpose of the CCC and the use of feedback to support their learning. An SMM does not require or guarantee complete agreement among members. Rather, with an SMM, all members of the team or CCC bring common understanding of their task and how they will conduct their work, which requires full participation working from the same set of expectations and open communication to share knowledge.

Shared Mental Model Research: An Exploration Outside GME

Research into SMMs examine the different types of models and their influence on team performance.^3,5 Green defined successful mental models as meeting 3 criteria: (1) an accurate reflection of the current reality, based on correct assumptions; (2) agreement among team members about the team goal and how the group will achieve goals; and (3) a description of how the group will work together to achieve goals.⁹ Achieving these criteria signals opportunity to apply SMMs to optimize the team's work; failure to achieve these should prompt the team to communicate, with the goal of reconciling discrepant opinions to achieve an SMM. Below we discuss the types of SMMs that can benefit team performance.

Understanding and Communicating About the Work

SMMs on teams help promote a cohesive understanding of upcoming projects and increase overall performance.³ Mental models can also be used to influence communication within a team. SMMs can be implemented in teams to help create a set of norms and guidelines for the team to follow.^1,5 These norms and guidelines can evolve as needed to serve as a useful basis for communication within a team. Without an SMM, individual members of the team may be working hard and committed to the work but operating in different ways that lead to uncoordinated and discrepant efforts. In contrast, a shared understanding within a team will guide members to know when and what to communicate and to be able to rely on mental models to predict what is needed by others.^1,5,10,11 This emphasis on clear communication to promote a common understanding of what is expected of the team influences overall task completion by establishing expectations and goals for team members to keep in mind when working toward project completion.

Team Training and Team Cognition

Creating SMMs through training or implementation of communication tools can promote improved team performance.^1–6 The training an organization conducts fosters mental models by providing the information needed to strengthen faculty's understanding of requirements and expectations.⁴ SMMs are an effective form of team cognition, that is to say, the way a team's knowledge is organized, represented, and distributed within the team. SMMs do not arise spontaneously. Team members may bring idiosyncratic mental models of team purpose and process. The group arrives at an SMM through co-construction of the team's work and how it will be done. This includes constructive conflict, in which team members confront and jointly work through differences in understanding, which enables teams to synthesize their understandings into an SMM.¹² A systematic review by Floren et al on strategies to develop SMMs within health care teams discusses interventions including team training, planning, leadership, and reflexivity.⁴ Training enables team members to understand others' roles to pursue shared team goals, with the support and alignment of the leader. Reflexivity entails reflecting upon the team's work and debriefing to maintain and refine SMMs.¹³ In this way, SMMs, reflecting shared cognition, can be used to examine team outcomes and the degree to which the team achieves its aims.¹⁰

Team understanding is a vital component of how members complete their tasks while communicating clearly with one another about expectations. Using an analysis of a team's mental models as feedback to the team can enhance the team's performance by providing a developmental framework.⁵ This analysis can be done through a comparison of charts, Venn diagrams, or other concept maps to identify the tasks that have an SMM. Similar to the consideration of SMMs for CCCs is the recent addition of “Teaming” as a component of the Accreditation Council for Graduate Medical Education's CLER (Clinical Learning Environment Review) Pathways to Excellence.¹⁴ The focus on purposeful interactions in which team members identify and capitalize on their various professional strengths to collaborate and share accountability for achieving results draws on many aspects of the SMM construct.¹⁴ The focus on mutual understanding, team communication, and shared accountability acknowledges the role of teams and SMMs to the optimization of the clinical learning environment in which faculty members practice and residents and fellows learn.

Tasks

Task-specific SMMs are the knowledge shared among group members about the nature of a task and the steps taken to complete it.¹⁰ When everyone within the team knows what needs to be done, the group can function more efficiently and cohesively. While not every team member may hold all knowledge relevant to the group's work, members should in that case know who on the team can do what so that the team has an SMM of complex tasks.¹⁰ Task-specific mental models may address the nature of the task and sequence of activities as well as who will do what.¹⁵ Ongoing research on SMMs can provide invaluable insight on team formation and project completion.

Research on Shared Mental Models in Graduate Medical Education

Clinical Competency Committees

Though SMM research previously arose in business and psychology literature, over the last 10 years a body of SMM research has emerged in the GME community. A shared understanding of the rationale for the CCC and the nature of its work can improve this group's functioning. Across GME, the CCC is required to provide, at a minimum, 2 Milestones-based evaluations of trainees annually.⁸ These evaluations must be used as a part of the program to determine a learner's overall level of competency and, at the completion of training, provide a recommendation for graduation. This requirement places a great deal of accountability on the CCC to ensure that a graduate is ready for unsupervised practice.^16,17

CCCs may struggle to define the purpose of their work and the best way of assessing resident performance.¹⁸ For example, a study of CCC decision-making in psychiatry using simulated resident cases revealed that program leaders exhibited overall low agreement about how to rate residents' performance. The relatively better agreement for ratings of medical knowledge and patient care than other competencies suggests that educators may have more shared understanding about performance expectations for these 2 competencies.¹⁹ Lack of a shared understanding about expectations could lead to misunderstandings, even legal challenges, should the CCC make an adverse decision about a learner.²⁰

Milestones and Assessment

Research in GME has characterized methods of articulating an SMM for defining and implementing decision-making about residents' achievement of the Milestones. In national emergency medicine Milestones ratings, the absence of “straight line scoring,” in which all Milestones are rated the same for a given resident, provides reassuring evidence that CCCs share understanding of the different constructs measured by the 6 competencies and associated Milestones.²¹ In order to improve committee members' decision-making around Milestones through greater alignment with a shared set of expectations and procedures, a system for collecting, synthesizing, and interpreting assessment data is needed. GME programs have approached this system design using various methods for selecting assessment tools and then designing a system. In emergency medicine residency training, educators across institutions collaborated to use a Delphi process to identify 11 EPAs for emergency medicine training as an intuitive method of operationalizing assessment of Milestones based on the essential tasks in the discipline.²² A system of observable professional activities mapped to the internal medicine Milestones and EPAs specifies the behaviors to be assessed during daily clinical care.²³ Recently, Park and colleagues described a mapping system to articulate the assessments and weighting for each Milestone to guide the committee in how to integrate multiple data points into a decision in a reproducible way for all residents.²⁴ Efforts to develop common Milestone language for competencies such as communication, professionalism, and systems-based practice contribute to shared understanding of these essential physician behaviors to guide CCC work across specialties.^25,26 These shared Milestones contribute to an SMM across the continuum of training and across disciplines. Taken together, these studies highlight the necessity of building SMMs into the work of the CCC and the importance of efforts to develop committee members' common understanding and a carefully designed assessment system to align data collection, interpretation, and decision-making.

Challenges

The ubiquity of the term “SMM” in discussions of assessment,^27–29 Milestones,^30,31 learning goals,³² entrustable professional activities,³³ entrustment,^34,35 feedback,^28,35,36 coaching,³⁷ and competency committees^38–40 might leave the impression that the concept is self-explanatory and that there is an SMM of SMMs. However, the familiarity and recognition of the popularity of the term should not be mistaken for understanding.⁴¹(p217) A principle challenge with mental models is the need to do the work to define them among team members in each instance, rather than to make a cursory assumption about their existence.^1–6 For example, CCCs must deal with tensions around varying mental models for the scope of attention (phases of education or a cohort of learners through all phases), of the best basis for judgment of competence (performance at a point in time or growth over time) and of the appropriate data for assessment (assessment with high psychometric validity or a program of assessment that includes data that may be less psychometrically persuasive). Mental models of performance assessment are challenged by individually idiosyncratic models of clinical excellence. Some assessors may focus more on the ability to establish patient relationships while others focus more on expertise.⁴² Moreover, individual assessors may apply preferences inconsistently or unfairly.^28,43 Evidence that Milestones and entrustable professional activities (EPAs) provide mental models shared between assessors and learners is mixed.^33,44–49 Part of the challenge is structural: Milestones and EPAs that are broadly defined are susceptible to variable interpretation.^49,50 This variability is affected by the larger challenge of the lack of an SMM of any instrument of assessment and the consequences of that assessment, which involves a process akin to negotiation.^45,46,51

The most important step in developing, maintaining, and using SMMs is to first acknowledge the complexity of a seemingly simple concept. Mental models are like Russian nesting dolls with each successive underlying model giving form to the next. The best result depends on careful attention through committee discussion and leadership at each level. For example, an engaged conversation of the sort described by Sargeant and colleagues, increases the likelihood that a resident or fellow will understand what is being assessed and the assessment itself and be able to develop a constructive response.^51,52

Complexity is added by the size of the CCC or the number of learners the CCC must evaluate. La Macchia et al demonstrated that large decision-making groups are viewed as less trustworthy than small ones and that trust is essential in evaluations and feedback.⁵³ This idea was furthered by Saap et al who described that if the number of learners to be assessed is high, even with an optimized number of CCC members, there is more risk of decision fatigue leading to groupthink about an individual learner.⁵⁴ If the number of learners and the number of members of the CCC are both high, there is an opportunity for distributed responsibility for the evaluation, which if not managed carefully may mean evaluation according to the mental model of an individual or subgroup rather than the SMM of the CCC as a whole.

The shared beliefs, understanding, and experiences on which SMMs are based are increasingly difficult to achieve as groups grow in size and are separated in time and space.⁵⁵ Those challenges are characteristic of large, professionally diverse (eg, area of practice, years of experience, location of practice) clinical departments and training programs. Although measures to promote and sustain SMMs under these circumstances are likely to be similar to those described in the next section, research is needed to know what will be effective.

Building Shared Mental Models in CCC Work

Methods of building SMMs include explicit training and ongoing committee discussion and reflection. Education for CCC members begins with introducing the concept of an SMM and how it can benefit and streamline the committee's work. Members' development of an SMM (Table) entails several aspects, including:

Table.

Application of Shared Mental Models (SMMs) for Learner Assessment in Graduate Medical Education

	Communication	Expectations	Guidelines
Resident Performance	SMM of desired performance among CCC members and between faculty and learner	Faculty discuss actual performance against a standard	Use of Milestones or EPAs to identify strengths and areas for improvement
Assessment	Between faculty and learner	Continuous improvement via faculty development	Identify intent of assessment tool and meanings of the rating or score
CCC	Clear communication among CCC members	Development of guiding principles and systematic discussions	ACGME requirements and decisions regarding learner advancement
Milestones	CCC members having open discussion	Discuss each Milestone for each learner	Shared understanding of Milestones in their program and how assessment tools map to those Milestones
Feedback	Two-way dialogue between learner and faculty	Milestone evaluations shared twice each year	Complete and thorough discussion that includes plan for next steps
Evidence	Working with other graduate medical educators	Knowledge of current research	Informing assessment and CCC work

Abbreviations: CCC, clinical competency committees; EPA, entrustable professional activities; ACGME, Accreditation Council for Graduate Medical Education.

1. Resident performance: an SMM of the desired resident performance and what this looks like enables members to assess actual performance against a standard. For some specialties and programs, EPAs may be used to define the specific expected tasks of the specialty. CCCs can devote time to discussing a typical resident at each level of training, and a resident who struggles or excels, and work backward to articulate how they identified those types of performance and what behaviors drove their characterizations of those residents.

2. Assessment: an SMM of the intent of an assessment tool, the meanings of the ratings and evidence generated with the tool, and how assessment evidence impacts the learner is needed. Although the CCC does not use the assessment tools to perform daily assessment of learners, they must interpret assessment information submitted by faculty and other assessors. As mentioned previously, assessors may themselves vary in their understanding of what is to be assessed and how to complete the assessment. Inconsistency or poor quality in ratings or narrative comments describing performance can be improved through the use of performance dimension training or frame-of-reference training to promote faculty understanding of what aspects of performance are being rated and what level of performance is expected.⁵⁶

3. Clinical competency committee: an SMM for the CCC encompasses the purpose and aims of group decision-making for residents' learning and advancement. Clear understanding of the benefits of group decision-making over individual decision-making, and the role of individual members, enables committee members to capitalize on the strengths of the group.⁵⁷ Articulating a set of guiding principles for the committee, combined with group members' clear understanding of the ACGME requirements for CCCs and for resident advancement, can enhance consistency in decision-making and adherence to requirements.

4. Milestones: CCC members must have a shared understanding of the discipline's Milestones and how they are assessed in order to reach consensus on residents' progress. To build the committee's understanding of the Milestones and how they are assessed within the program, committee members should receive, or conduct, mapping of the assessment tools in the program to the subcompetency Milestones. Discussing each Milestone and how it manifests in resident work is also a useful exercise for CCC members. Mapping assessments to Milestones within the system of assessment guides CCC members to know which evidence to use to assess progress on each Milestone. Adapting the specialty-specific Supplemental Guide with examples and assessment tools that exemplify your program is a direct way to create an SMM within the CCC.

5. Feedback: an important purpose of the CCC is to generate feedback for residents that will be shared semiannually by the program director, a mentor, or coach. The content and process for sharing this feedback is an area for attention within the program to promote consistent, complete, and effective feedback conversations. A common model for training faculty to discuss feedback in a bidirectional dialogue to promote learner change is recommended, such as the R2C2 (rapport and relationship building, reactions to feedback, content of feedback, coaching for change).⁵²

6. Evidence: CCC members should stay abreast of ongoing research on learner assessment and group decisions in GME. Collaboration with education scientists within the institution and journal clubs are strategies to identify and understand current literature and how it can inform the CCC work.

Revisiting these mental models at least annually helps ensure that the committee members, both longstanding and new, maintain a common understanding of their work. Without this ongoing attention to SMMs, committee members or the group as a whole may drift from their original purpose and approach.

Conclusions

This article reviews the use of SMMs in decision-making (see Figure). The literature related to SMMs is expansive in relation to teams and team functioning and is growing as it relates to GME and CCCs.

Figure.

Shared Mental Model Leading to Improved Decision-Making

In the context of the CCC, an SMM is a framework that can be used to bring together a group of individuals with unique perspectives to identify strengths and areas of concern and determine a path of professional development. For high-quality, consistent, and fair decision-making, the CCC must share understanding of expectations among members, the faculty who are assessing the learners, and the learners themselves.^38,58,59 The development of SMMs helps to move the CCC forward, but without doing the hard work to maintain and manage the shared knowledge, the results will be varied and potentially fall short of the aims of the CCC.

GME continues to evolve to optimize learning, and ultimately, trainees' patient care. The addition of the Milestones to the GME experience addressed the differences in how programs assessed their learners and offered an opportunity for consistency across programs aimed at achieving agreed-on outcomes of training. As faculty and CCCs develop SMMs of their assessment tools and procedures, Milestones, and decision-making tasks, there is ongoing need to ensure that learners are evaluated consistently and fairly within and across CCC meetings. Using clear communication and guidelines, and shared understanding of goals and expectations, CCCs can make better decisions through SMMs.