International Clinical Competency Committees: Maximizing Value for Faculty, Residents, and the Program

Rip Out Action Items

Clinical competency committees (CCCs) should:

1. Select CCC members invested in learner growth and development through evidence-based assessments.

2. Create a shared mental model of competency framework and periodically recheck.

3. Map current assessment tools to the milestones and identify gaps.

4. Ensure that all residents are deliberately discussed during CCC meetings.

5. Provide milestone data to the Program Evaluation Committee to support a review of curriculum and teaching.

The Challenge

Assessment of learners using a multifaceted approach is an essential component of the graduate medical education (GME) system across the globe. Clinical competency committees (CCCs) are expected to aggregate and synthesize individual assessments to inform faculty, program leadership, and learners of the given learner's progress toward unsupervised practice. The challenge for CCCs is that through these assessments, the committee must engage in a deliberate discussion of each learner, identifying areas of weakness or accelerated growth, and develop and implement individualized learning plans. Much remains to be learned about optimizing the CCC structure and process to facilitate robust developmental assessments, and actionable feedback to learners and, at the aggregate level, to program leaders. This Rip Out draws on studies of group process from other fields and published evidence to provide guidance regarding CCC structure, membership, and processes.^1,2

What Is Known

• Group Structure and Membership. Diverse groups tend to make better decisions than homogeneous ones. CCCs should ensure that membership is balanced in academic rank, gender, expertise, and roles within the program.² Nonphysician health professionals can bring an important perspective, especially in the domains of interpersonal skills and communication, and professionalism.

• Data. CCCs need access to rigorous and robust assessment data to facilitate their discussion of the learner progress, and ideally, training programs will have mapped their assessments to the educational milestones or entrustable professional activities (EPAs). This mapping enables initial analyses, and through dashboard tools such as radar plots and longitudinal milestone progression, can facilitate assessment of each competency (or EPA).³ Yet studies have suggested that CCCs infrequently use clinical performance data or patient experience survey data as a source of assessment.^4,5 All CCCs should look for opportunities to seek out and include this data, as it is essential for judging competency in communication skills, quality improvement, patient safety, and systems thinking.

• Meeting Process. The CCC chair needs to create a safe environment for all members to speak up and share their judgment and concerns, to allow all members to be engaged and invested in the work. Hierarchy can surface in CCC meetings.² The chair can ameliorate this by asking the most junior person (real or perceived) to share her or his thoughts first, before more senior members speak. Research has shown that the CCC process tends to fall into 2 patterns—predominately problem identification (only struggling residents are discussed) or developmental assessment.⁶ The goal of the CCC should be to discuss all residents and their professional development.^2,6

• Shared Mental Models. Research in group decisions in an education context has highlighted the importance of shared mental modes around a competency framework used to judge the professional development of learners.^2,4 CCCs should actively work to develop shared mental models through conversation and discussion of specific examples. This might require that each member take ownership of a subset of the competencies, offering teaching and coaching to other members.²

• Markers of Success. When CCCs function well, studies have shown that programs improve their individual and overall program assessments, uncover gaps in the curriculum, identify and assist struggling residents earlier, provide better and more specific feedback to all learners, and enable greater involvement and activation of learners through the creation of individualized learning plans.²

How You Can Start TODAY

1. Select Your CCC Members Wisely. Select faculty with an interest in assessment and feedback—those who are willing to dedicate the time necessary to review all of the assessments from the previous 6 months.

2. Develop a Shared Mental Model of Resident Performance. Before the CCC meets for the first time, convene the members in a workshop to create a shared mental model of the milestones. These decisions can be captured in a program-specific supplemental guide. The time to complete this work will be recouped through time savings in future CCC meetings and better assessments. Each meeting can begin with a short review and each member will be ready to offer appropriate decisions.

3. Use Data. Map your current assessment tools to the milestones and determine if there is sufficient opportunity for each milestone (or EPA) to be assessed. Ensure that the data being gathered, reviewed, and aggregated are used in multiple ways: feedback to the learner and evaluation of the program. For example, if learners struggle to attain a recommended level (eg, milestone level 4 for graduating residents), the curriculum or clinical experience may lack a critical component. Milestone data also can be compared to other program characteristics (eg, faculty turnover, certification rates).

4. Discuss Each Resident Deliberately. It is easy to focus on residents who are struggling and spend a great deal of time discussing them during the meeting. This is time well spent, as the intent is to provide appropriate feedback and a learning plan. In contrast, learners who outperform expectations sometimes are given a very quick review and may not receive guidance to allow them to deepen their learning and progress.

What You Can Do LONG TERM

1. Monitor your CCC for good group process and continued deliberate discussion of each resident. Identify and track your markers of success.

2. Implement a program of assessment and faculty development. Offer ongoing faculty development to ensure faculty maintain their shared mental model of the assessment tools.

3. Build a culture of ongoing assessment and feedback within your program. Invite residents to perform self-assessments with the milestones, discuss the differences between the CCC and resident assessment, and jointly create a learning plan. Share resources for CCCs and milestones (available from Accreditation Council for Graduate Medical Education International and others) with faculty and residents.

4. Ensure the ongoing use of CCC data in program evaluation and improvement and the continuous communication of the outcomes to residents and faculty.

Conclusion

Whether you are starting your first CCC or you want to ensure that your existing CCC maximizes the value of participants' time and feedback to learners, follow the steps above to creating a high-functioning group. A final way to add value is to ensure the assessment data gathered, reviewed, and aggregated are used in multiple ways including program evaluation and improvement.