Chronicling the Transition to Competency-Based Medical Education in a Small Subspeciality Program

Abstract

Background As medical education programs transition to competency-based medical education (CBME), experiences transitioning in the context of small subspecialty programs remain unknown, yet they are needed for effective implementation and continual improvements.

Objective To examine faculty and resident experiences transitioning to CBME in a small subspeciality program.

Methods Using a qualitative descriptive approach and constructivist lens, faculty and residents in McMaster University’s geriatric psychiatry subspecialty program were interviewed about their transition experiences between November 2021 and February 2022, after the program’s soft launch of CBME in 2020. Interviews were transcribed and data were analyzed using thematic analysis. Reflexive memo writing and investigator and data triangulation strategies were employed to ensure rigor and trustworthiness of the data.

Results Ten of the 17 faculty members (59%) and 3 residents (100%) participated. Six themes were developed: (1) Both faculty and residents see themselves as somewhat knowledgeable about CBME, but sources of knowledge vary; (2) More frequent feedback is beneficial; (3) Aspects of CBME that are challenging for residents are beneficial for faculty; (4) Competence committees are perceived positively despite most participants’ limited firsthand experience with them; (5) Small program size is both a barrier and facilitator to providing and receiving feedback; and (6) Suggestions for improvement are centered on helping manage faculty and resident workload imposed by CBME.

Conclusions Incongruent expectations surrounding entrustable professional activity management were highlighted as an area requiring support. Collegial relationships among faculty and residents made it difficult for faculty to provide constructive feedback but improved residents’ perceptions of the feedback.

Introduction

Graduate medical education (GME) programs across the world are transitioning to competency-based medical education (CBME) with the aim of enhancing patient care by improving assessment and learning in medical education.^1,2 To date, few studies have evaluated transition progress, and the few that exist^3-5 focus primarily on resident experiences in larger, procedural specialties and often generalize findings to all GME programs, despite contextual differences between programs that can impact implementation.⁶ As more specialties and subspecialties transition to CBME, knowledge from other programs’ transition experiences is used to inform implementation and transition decisions in similar contexts.⁷ With no studies to our knowledge evaluating faculty and resident experiences with the transition in smaller programs, it is unclear how the transition is progressing in these contexts.

One study of neurological surgery residents found that Competence by Design, the Canadian version of CBME,¹ allowed for more detailed and specific feedback, but completing all the entrustable professional activities (EPAs), which reflect the core skills of a specialty, was a time-consuming process.³ Another study of otolaryngology–head and neck surgery residents found that EPAs were completed at a very slow rate (ie, 1 per 4 weeks) and suggested assessment fatigue as a possible contributor.⁵ Several studies also suggest that while CBME implementation has led to an increase in feedback quantity, feedback quality remains unchanged.^4,6,8-11

The initial benefits and challenges of CBME depicted in the literature are often generalized to all GME programs. This is troublesome, as programs will differ in the timing of the transition, number of required EPAs, and available support, among other things. Additionally, faculty buy-in is integral to successful implementation of CBME, as faculty assess EPAs, participate in competence committees, and coach residents as they move through the curriculum. However, few studies have examined faculty perceptions of the transition.⁴

With the transition to CBME yet to be evaluated in the context of small programs, the objective of this study was to examine resident and faculty experiences transitioning to a CBME framework in a geriatric psychiatry subspecialty program.

Methods

This exploratory study used a qualitative descriptive approach¹² with a constructivist epistemological lens to gain firsthand knowledge of faculty and resident experiences with CBME implementation. Constructivism views knowledge as actively constructed by people through their unique perspectives, experiences, and interactions, emphasizing the importance of understanding and interpreting the subjective realities of research participants.¹³ This study was conducted as part of a larger program evaluation initiative within the Division of Geriatric Psychiatry in the Department of Psychiatry and Behavioural Neurosciences at McMaster University in Hamilton, Ontario, Canada between November 2021 and February 2022. All faculty (N=17) and residents (N=3) within the division were invited to participate in interviews via email, including several follow-up emails if a response was not received. Informed consent was obtained and video-based semistructured interviews were conducted. The interview guide (provided as online supplementary data) was created by a research assistant (T.A.S.) and education scientist (A.A.) with expertise in qualitative research and some familiarity with the program. The interview guide contained open-ended questions based on a review of the literature and consultation with program directors. Interviews were conducted by T.A.S. and S.O., who were trained in qualitative data collection methods, and supervised by A.A.

Interviews occurred over Zoom and were audio-recorded, transcribed, and de-identified. Reflexive thematic analysis was conducted using the steps outlined by Braun and Clarke,¹⁴ and themes were identified inductively. Following these steps, T.A.S. and S.O. familiarized themselves with the data, independently coded 3 randomly selected interviews (2 from faculty and 1 from a resident) to generate initial codes and compared and discussed codes to resolve interpretation differences. The remaining interviews were divided among the researchers who further collated codes into potential themes, refined each theme, and extracted representative examples. The research team met regularly during the analytic process to discuss interpretations of the data and ensure that codes were not constructed based on existing knowledge and assumptions but supported by the data. Reflexive memo writing and investigator and data triangulation strategies were also used to ensure rigor and trustworthiness of the data.

The team comprised T.A.S. (research assistant and undergraduate student in psychology) and A.A. (education scientist with expertise in qualitative and quantitative research methods), who offered nonclinical and education research viewpoints, while S.O. (MD/PhD candidate in neuroscience) and D.D. (academic geriatric psychiatrist) provided clinician and health researcher perspectives. T.A.S., S.O., and A.A. are all external to the Division of Geriatric Psychiatry, thus bringing “outsider” perspectives, while D.D. is a member of the Division and brought an “insider” perspective to the analytic process. The diverse education and training backgrounds of the research team helped further ensure rigor and trustworthiness.

This study was designated as program evaluation and was exempt from review by the Hamilton Integrated Research Ethics Board. Participants provided consent verbally.

Results

Ten of the 17 faculty members (59%) and 3 residents (100%) participated in interviews. Most faculty who did not participate indicated that they had not recently supervised residents. Nine participants identified as women and 4 as men, which reflects department demographics. Six themes were developed. Representative quotes are listed in the Table.

Table.

Themes and Representative Quotes

Theme	Faculty Quote	Resident Quote
Both faculty and residents see themselves as somewhat knowledgeable about CBME, but sources of knowledge vary.	“I would say that I’m fairly familiar because I also supervise family medicine residents, and I believe they transitioned before the geriatric psychiatry program. And I’ve read about it and learned about it also in the context of other projects.”	“So, as I mentioned, I only feel like I have a basic understanding of [CBME]. And given competing demands of other requirements of my training, I have not spent a lot of time researching it or reading about it in depth … I basically know what information has been sent to me in orientation packages and spoken with staff about it as needed.”
More frequent feedback is beneficial.	“I think the only way that [feedback has] changed is in the frequency of feedback. I’m giving feedback much more frequently based on doing things like EPAs, than I would have before.”	“I guess getting more frequent feedback can be helpful as well. Earlier on, rather than just at the end of a rotation, like [at the] beginning and end.”
Aspects of CBME that are challenging for residents are beneficial for faculty.	“The number of successful EPAs that our residents need to complete seems to be very reasonable and doable in the time allowed.”	“I think it asks a lot of residents … The goal shouldn’t be keeping track of the EPAs … it should be keeping track of what you need to do to become a better learner and a better clinician.”
Competence committees are perceived positively despite most participants’ limited firsthand experience with them.	“I don’t know very much at all [about competence committees], other than there is going to be a committee that will oversee the completion of the EPAs.”	“I know I’ve been reviewed by committees, but I’ve never been at the committees while I’ve been reviewed … So, I’m not aware of what occurs at the committee meetings … it’s the type of thing that’s always nice to know that it’s happening and that you are kind of being reviewed and any concerns would be addressed. So, I think it’s helpful to have them, but I haven’t been aware of the inner workings.”
Small program size is both a barrier and facilitator to providing and receiving feedback.	“I think the tendency to provide honest, constructive feedback is harder in a small program because we all know each other and … we sort of work together in a very close environment. And in many instances, the subspecialty trainee is 6 months away from becoming a colleague.”	“I feel like it makes it easier, like if it’s a supervisor you know well, you value their opinion, you know that clinically you have worked with them. So, you know how they work. And I find that I value their feedback even more when I know them, and I know how experienced they are … I wonder if [faculty] feel more or less comfortable sharing [constructive] feedback. If they’ve known you so well and you have that kind of relationship already. Not sure.”
Suggestions for improvement are centered on helping manage faculty and resident workload imposed by CBME.	List of feedback/coaching phrases Ongoing training Ongoing technology support Centralized directory for resources and tools	Joint check-ins Automated EPA tracking

Abbreviations: CBME, competency-based medical education; EPA, entrustable professional activity.

Theme 1: Both Faculty and Residents See Themselves as Somewhat Knowledgeable About CBME, But Sources of Knowledge Vary

All participants felt that they were at least somewhat knowledgeable about CBME, but sources of knowledge varied across participants. For some, knowledge about CBME came primarily from informational documents provided by the program. For others, experience implementing CBME in other contexts (ie, other programs) formed the basis of their understanding.

Theme 2: More Frequent Feedback Is Beneficial

Participants commonly realized the benefit of feedback being provided more frequently in the context of CBME. However, for some, informal feedback was provided frequently prior to CBME implementation, so this benefit was anticipatory rather than actually experienced.

Theme 3: Aspects of CBME That Are Challenging for Residents Are Beneficial for Faculty

Despite relatively positive perceptions overall, residents reported increased workloads from the additional administrative work with CBME, such as triggering assessments and keeping track of EPAs. Some residents developed systems for manually tracking EPAs but found this additional administrative work to be cumbersome and a distraction from clinical training. While residents found it burdensome, faculty members were pleased with CBME being more resident driven and with residents taking initiative to find opportunities to complete EPAs.

Theme 4: Competence Committees Are Perceived Positively Despite Most Participants’ Limited Firsthand Experience With Them

Overall perceptions of competence committees, which are a component of CBME programs that make group-based decisions about resident progress, were positive despite most participants having limited firsthand experience with them. Perceptions appeared to stem mostly from the theoretical benefits of competence committees rather than benefits actually experienced.

Theme 5: Small Program Size Is Both a Barrier and Facilitator to Providing and Receiving Feedback

Prior to CBME implementation, the small program size was beneficial, as it facilitated close relationships among faculty and residents and, as a result, a preexisting culture of informal feedback. This benefit continued but created some new challenges for faculty giving critical or constructive feedback within CBME. Concerns were raised about the potential for feedback to negatively impact competence committee decisions about promoting a resident to the next stage of training. Faculty also described hesitancy providing critical feedback to residents they viewed as coworkers and with whom they had close working relationships. Some faculty noted that residents were accustomed to achieving excellence; thus, they avoided giving critical feedback that risked harming resident morale. Residents generally felt that they received honest and helpful feedback, and valued feedback from faculty with whom they had close working relationships, as it gave them confidence that the faculty wanted to help them improve.

Theme 6: Suggestions for Improvement Are Centered on Helping Manage Faculty and Resident Workload Imposed by CBME

Suggestions were provided for improving the CBME transition. Both faculty and residents discussed the need for a centralized directory for locating resources and tools, as well as ongoing training and support in CBME. Faculty suggestions focused on guidance for providing constructive feedback, such as a list of example feedback/coaching phrases. To relieve the administrative burden and increased workload of needing to develop their own systems and manually track EPAs, residents suggested an automated system for completion and tracking of EPAs.

Discussion

The present findings suggest that small subspecialty programs may be well suited for the transition to CBME, although not without some unique challenges. Geriatric psychiatry residents had more positive perceptions about the CBME transition than residents described in the literature,^8,15,16 which may be explained by the close working relationships and existing culture of informal feedback, as well as the smaller number of EPAs for subspecialty programs. Despite the challenges experienced, residents cited benefits they anticipated from CBME.

Frequently discussed were the opposing views about tracking and initiating EPAs. Residents described difficulties with faculty expectations for initiating EPAs and uncertainty about when to initiate them. Incongruent expectations surrounding EPAs are not unique to geriatric psychiatry^4,17,18 and reveal areas where the responsibilities and tasks of CBME can be clarified and where program supports can be improved. As one participant suggested, opportunities for open dialogue between residents and faculty to share experiences would allow differing views to be addressed and for residents to receive clarity and support.

Faculty members described challenges with providing constructive feedback to residents, which were attributed primarily to collegial relationships, desire to protect resident morale, and concerns about negatively impacting competence committee decisions. These challenges are likely common in programs where faculty and residents work closely and have a more balanced relationship than the typical teacher and learner. Residents were receptive to receiving constructive feedback and suggested that their close relationships with faculty facilitated more meaningful feedback conversations. Other research supports this notion, suggesting that supportive and trustworthy relationships between teachers and learners helps enable productive feedback.^19,20

The present sample is bound by the context of a single academic center and the small nature of the subspecialty program. Further research should assess if these findings are transferable to other small specialties and subspecialties. Applying CBME-specific evaluation frameworks, such as the Core Components Framework, may be helpful as efforts to evaluate this new curriculum continue.²¹

Conclusions

Faculty and residents in geriatric psychiatry were generally pleased with the transition to CBME, but residents reported more challenges than faculty, some of which were unique to smaller programs. Suggestions for ongoing support and additional resources were made to improve the transition and reconcile some of the differing expectations and experiences between faculty and residents.

Editor’s Note

The online supplementary data contains the interview guide used in the study.

Author Notes

Funding: This study was funded by the Education Innovation Award from the Department of Psychiatry and Behavioural Neurosciences at McMaster University.

Conflict of interest: The authors declare they have no competing interests.

This work was previously presented at McMaster University Department of Psychiatry and Behavioural Neurosciences 34th Annual Research & Scholarship Day, May 3, 2023, Hamilton, Ontario, Canada, and Norman Education Research Day 2023, June 7, 2023, Hamilton, Ontario, Canada.