Faculty Development for Milestones and Clinical Competency Committees

The goal of postgraduate training is the development of competency necessary for independent practice. To achieve this goal, the Accreditation Council for Graduate Medical Education (ACGME) in 2013 implemented the Milestones assessment framework and mandated the development of clinical competency committees (CCCs), tasked with biannual review of evaluation and assessment data of trainees.¹ The CCC, which consists of a chair and committee members, plays a critical role in assisting the program director in decision-making regarding resident or fellow advancement through the training period. Importantly, the committee makeup may span junior and senior faculty and include interprofessional representation, thus providing a spectrum of input.¹

To date, most of the literature regarding CCCs focuses on optimizing CCC function in terms of workflow and decision-making. Best practices for CCC function have been proposed, including use of varied assessment tools, structured meetings, and diverse members.^2,3 Other work has outlined strategies for effective group decision-making necessary for CCC function.^4–6 Ongoing faculty development has been proposed as an essential component of CCC function, yet little is known about which particular elements are necessary to optimize organization of meetings, decision-making, and to effectively utilize assessment processes.

In order for the CCC to optimally operationalize the Milestones for individual learner development, members of the committee require intentional faculty development. This must span multiple domains important for the CCC process, including knowledge of assessment (and the use of the Milestones), understanding of group decision-making, awareness of how biases interplay with these factors, and the effect of their decisions on patient care (Box). While it remains unknown how often and who should develop such faculty development sessions, we would advocate for at least an annual review of these topics given faculty turnover and need to maintain a shared mental model of best practices. Faculty development remains essential for both new and longstanding members, as participants will likely have different levels of understanding of the assessment process and Milestones themselves.²

Box Proposed Resources for Faculty Development of CCCs

Leadership Training for CCC Chair

• ▪AAMC Professional Development Leadership training https://www.aamc.org/professional-development/leadership-development
• ▪Leadership Education and Development (LEAD) Certificate Program https://www.aamc.org/professional-development/leadership-development/lead
• ▪Harvard Macy Institute: Leading Innovations in Health Care and Education https://www.harvardmacy.org/index.php/hmi-courses/leaders

Assessment and Evaluation

• ▪ACGME Faculty Development Course on Assessment https://dl.acgme.org/learn/course/an-introduction-to-assessment-1/introduction-to-assessment/course
• ▪AAIM Clinical Competency Committee Faculty Development Toolkit https://www.im.org/resources/ume-gme-program-resources/ccc
• ▪Harvard Macy Institute: A System Approach to Assessment in Health Professions Education https://www.harvardmacy.org/index.php/hmi-courses/assessment

ACGME Milestones

• ▪ACGME Faculty Development Course: Introduction to Milestones https://dl.acgme.org/courses/introduction-to-milestones
• ▪AAIM Clinical Competency Committee Faculty Development Toolkit https://www.im.org/resources/ume-gme-program-resources/ccc
• ▪Macy Foundation: Achieving Competency-Based, Time-Variable Health Professions Education https://macyfoundation.org/publications/achieving-competency-based-time-variable-health-professions-education

Bias Mitigation in Decision-Making

• ▪AAMC Resources: Unconscious Bias Resources for Health Professionals https://www.aamc.org/what-we-do/mission-areas/diversity-inclusion/unconscious-bias-training
• ▪Project Implicit: Implicit Association Test https://implicit.harvard.edu/implicit/takeatest.html

Group Decision-Making

• ▪Harvard Macy Institute: Leading Innovations in Health Care and Education https://www.harvardmacy.org/index.php/hmi-courses/leaders

We therefore aim to provide guidance for specific areas of faculty development for CCC members, including strategies for novel approaches to professional development for the committee. This includes: (1) the purpose and structure of the CCC itself; (2) the strategies, strengths, and weaknesses of the assessments employed by the program; and (3) the development of a shared mental model of the Milestones of the relevant specialty. Beyond this, the CCC chair will also need to understand (4) the best practices of an effective group discussion.⁷

Purpose and Structure of the CCC

Faculty need to understand the purpose of the CCC meetings, which often employ 1 of 2 strategies: problem identification, where the committee focuses on identifying learners who may require additional remediation, or a developmental approach, where the committee discusses all learners to identify next steps for advancement.³ Members also need to understand the process of decision-making, including who will make the recommendations regarding remediation.

Assessment Methods

In addition to understanding the intent of the Milestones, committee members must also be aware of the program's evaluation process (as related to each Milestone). Knowledge of the individual assessment methods and awareness of the data synthesis for each Milestone will improve a shared mental model and ideally minimize differences in individual interpretation of assessment data.⁶ This will also allow the committee to identify times when the assessment data is difficult to interpret requiring further group discussion.⁸

Relevant Training Milestones

Committee members should not only understand each of the relevant Milestones but also have a shared model of what a competent trainee looks like at their institution.⁹ It is evident that competencies are developed differently in different contexts and at different institutions.¹⁰ Therefore, a shared mental model is important for each CCC to minimize variation in how each member interprets the same assessment data.

Effective Group Practices

Effective committee groups understand the purpose, structure, and process of their work.⁴ However, sometimes groups encounter incomplete or otherwise difficult to interpret assessment data; therefore, group discussion is needed to draw a conclusion. The chair of the committee needs to lead the discussion and approach how decisions are made transparently. The chair should also be cognizant of the member composition of the group.⁴ A homogenous group is unlikely to present different views or opinions, which may be necessary in drawing fair conclusions.

Additional Faculty Development Topics

Additional faculty development topics for the CCC chair may be helpful in maximizing success of the committee. It is incumbent upon the chair to refine their own ability to synthesize feedback as well as interpret and gauge performance in specialty-specific Milestones. General leadership skills, such as how to effectively lead a committee or run a meeting, remain paramount.^11,12 It is important for the CCC chair to adapt their mindset from the frame of educator to that of the “educator-leader.” The CCC chair plays a pivotal role in ensuring that the committee functions through fair and informed decision-making processes based on the data and facts, rather than emotional decision-making.

Beyond this unique CCC training for the chair and members, all faculty educators must understand the intersection of biases on the assessment process. It is well-known that biases can manifest in a variety of ways through our traditional mechanisms of assessment. For example, in graduate medical education (GME), this has been highlighted in several settings showing differential Milestone achievement based on trainee gender,^13–15 as well as gender differences in narrative evaluation.¹⁶ Similarly, standardized patient assessments, which are often included in CCC discussions, have highlighted differential findings by gender, race, and ethnicity.¹⁷ Knowledge of such biases and the potential ramifications on Milestones ratings, narrative evaluation, remediation referrals, and progression decisions is imperative for faculty development in the CCC process.

Faculty development using bias reduction workshops or training, such as those employing the Implicit Association Test, could represent an avenue to initiate discussions,¹⁸ as these training programs have shown to be effective in mitigating biased decision-making in other settings.^18–20 The potential quantitative and qualitative differences in assessment should be considered in recurring CCC reviews. Additionally, faculty participants should be versed in the potential impact of system-wide and personal biases on trainee assessments. Beyond an individual training level, CCC structure, including the diversity of CCC members, as well as process measures, including program assessment and trends, should be regularly reviewed for biased tendencies.²

There are a multitude of topics important in faculty development for CCC chairs and members and implementation often requires novel solutions. One such approach was recently implemented within the MedStar Health GME Consortium in the Baltimore, MD and Washington, DC region, which includes more than 70 ACGME-accredited programs. The consortium conducted a half-day faculty development retreat specifically for CCC chairs. Topics addressed included a primer on evaluation of competence, a panel discussion of CCC best practices (for varied sizes of programs), small group discussions on CCC decision-making processes, and strategies for running effective meetings. The program also allowed for identification of distinct needs for various CCC stakeholders, as outlined in the Table. These sessions allowed for shared knowledge, highlighted knowledge gaps, and identification of distinct faculty training needs for success of CCCs. Such an approach allowed for sharing of best practices, strategies to overcome real or perceived challenges (such as time constraints to effective discussion in larger scale programs), and dissemination of knowledge throughout the sponsoring institution.

Table.

Proposed Topics for Faculty Development, Based on Position in CCC

Faculty Development Topic	PD Perspective of CCC Development Needs	Specific Needs of CCC Chair	Specific Needs of CCC Members
Knowledge about assessment, Milestones, and CCCs	Knowledge of ACGME requirements Knowledge of the intent of Milestones Understanding the role of the CCC and the role of the PD in determining resident achievement and advancement	Knowledge of ACGME requirements Knowledge of the intent of Milestones Understanding the role of the CCC and the role of the PD in determining resident achievement and advancement	Knowledge of ACGME requirements Knowledge of the intent of Milestones Understanding the role of the CCC and the role of the PD in determining resident achievement and advancement
Knowledge of remediation processes and practices	Understanding the role of the CCC and the role of the PD in determining resident achievement and advancement	Understanding the role of the CCC and the role of the CCC chair in determining resident achievement and advancement How to develop remediation strategies Knowledge about pathways for remediation	Understanding the role of the CCC in determining resident achievement and advancement Knowledge about pathways for remediation
Training in conducting effective meetings	Running an effective/efficient meeting Knowledge of CCC best practices Knowledge of group decision-making processes and pitfalls Strategies to navigate disagreements, whether intra-committee or between CCC and PD Allocating protected time (or equivalent) for CCC chair and members	Running an effective/efficient meeting How to utilize limited meeting time most effectively How to assure committee continuity and progress between meetings Knowledge of group decision-making processes and pitfalls Strategies to navigate disagreements, whether intra-committee or between CCC and PD	Adequate time for committee work/engagement How to adequately prepare for meetings, including pre-work and post-meeting follow-up
Interpreting data and metrics	Ability to utilize data and metrics effectively Differentiating between valid feedback and “grumblings” Mitigating bias Effective CCC “hygiene”: knowing what is appropriate and what is inappropriate to discuss at the CCC	Mitigating bias How to utilize CCC discussion that is not reflective of performance evaluations Best practices for documentation and production of meeting minutes Addressing personality variability and/or medical issues impacting performance	Mitigating bias Synthesizing data Successfully contributing to a shared mental model
Process of obtaining data	Coordinator or educational specialist resource to organize data for synthesis Resources for completion of evaluations and analytics, including data input and output	How to obtain valid metrics: ○ Evaluations ○ Improving completion, accuracy, quantity, and timeliness ○ How to utilize subjective feedback most effectively ○ How to solicit feedback from faculty who are not currently CCC members	How to incorporate feedback brought by faculty or others outside of the CCC structure

Abbreviations: CCC, clinical competency committee; PD, program director; ACGME, Accreditation Council for Graduate Medical Education.

Conclusions

Overall, the optimal functioning of a CCC requires dedicated faculty time, education, and investment in faculty development. CCCs form a critical component of GME assessment and play a pivotal role in ensuring readiness for independent practice. Given this critical role, we advocate for an appropriate fraction of protected time (or productivity) from clinical or other administrative duties for CCC chairs and similar key leaders to accomplish the important responsibilities of the role.

We have highlighted some overarching themes of essential faculty development for CCC success, but recognize that the needs are varied based on role within the CCC, experience level, and size of the program, among other considerations. It is critical to acknowledge differing faculty development needs across levels of experience of faculty (junior versus senior members), as well as the unique training needs of interprofessional members. Additionally, future scholarship is needed to determine the optimal timing, content, and subsequently the impact of such faculty development on CCC processes. We believe that the continued development of expertise for both the CCC chair and its members remain critical to the success of the Milestones assessment system.