The Feasibility and Impact of Implementing Brief Educator Development Sessions in Underresourced Residency Training Sites

ABSTRACT

Background Busy physician educators teaching in rural and underserved communities face challenges finding time and resources to support their professional development.

Objective We developed and assessed the feasibility and utility of delivering 25-minute virtual educator development sessions for educators in residency training programs located in rural and underserved communities.

Methods This study evaluated a monthly 25-minute web-based education program designed to develop residency program directors’ and other educators’ teaching and assessment skills in training programs located in rural and underresourced communities. Participants included educators from Oregon and California between September 2021 and October 2023. Feasibility was evaluated by educator attendance and technology use. Utility was measured by continuing medical education credit requests regarding educational value and participant-reported application of skills.

Results Ninety-one educators participated, 59 from Oregon and 32 from California. Most were female (64.8%, 59 of 91), between the ages of 30 and 49 (68.1%, 62 of 91), White (85.7%, 78 of 91), and non-Hispanic/Latinx (90.1%, 82 of 91). Physicians made up 75.8% (69 of 91), while 18.7% (17 of 91) were other program educators. Feasibility in terms of technology use was found to be 50% or greater by desktop computer connection, with the remaining by mobile devices. Regarding utility, continuing medical education credit was requested for 48 (29.8%) of the 161 viewed sessions.

Conclusions Brief, web-based educator development sessions are feasible to implement in residency training sites located in remote and underresourced communities. Participants self-reported their intention to apply what they learned in their work with trainees.

Introduction

The number of medical students and residents training in rural and underresourced sites is growing.^1-3 However, little is known about how to engage busy community-based physicians toward developing their skills as educators. We studied the feasibility of implementing a 25-minute “bite-sized” continuing medication education (CME) credit-eligible program for educators in residency training programs located in underserved areas in Oregon and California to address barriers to participating in professional development. This program was developed as part of the American Medical Association Reimaging Residency Initiative, California Oregon Partnership to Address Disparities in Rural Education and Health (COMPADRE). COMPADRE’s overarching aim was to reduce health disparities by transforming the physician workforce to be better prepared, more equitably distributed, and more deeply connected to underresourced communities in Oregon and northern California.⁴ An important component of COMPADRE was to develop residency program directors, educators, and community preceptors.

Methods

Program Development and Implementation

One author (A.M.J.) reviewed literature related to faculty development programs and solicited feedback from COMPADRE grant team members, community preceptors, and community GME training program leaders to identify barriers to and facilitators of participating in educator development opportunities for busy clinicians in COMPADRE settings. The bite-sized program structure addressed barriers to access and content expertise by keeping the sessions short (25 minutes), teaching tangible skills that could be immediately applied in training practices, improving accessibility, and providing tools for reference.^5,6

Sessions were presented in real time by national or local content experts. Presenters were instructed to provide 15 minutes of content using active teaching strategies and to allow 10 minutes for discussion, either at the end of or woven throughout the session. Sessions occurred on the third Tuesday of most months during a conventional lunch period (12:05 pm-12:30 pm). Each session was live-streamed virtually and recorded for later asynchronous access. The annual calendar of topics and links to access real-time and past sessions were posted on a learning management system accessible to study participants. Session topics were chosen based on input from rural and COMPADRE site educators.

During the study period, 91 individuals received calendar appointments and an email reminder one week prior to each session and were encouraged to share these with others at their respective sites. Sessions were advertised through a monthly COMPADRE newsletter, and participation was encouraged for anyone interested in developing their teaching skills, regardless of their role (eg, physicians, advance practice clinicians, program coordinators). Other than CME credit, no other incentives were offered. During each session, a survey link to claim CME credit was posted in the chat (provided as online supplementary data). The survey included demographic questions as well as an open-ended question asking participants to describe what they learned and will take back to their teaching practice. After each session, a link to the session recording and a link to the CME survey was sent to all enrolled COMPADRE educators.

Study Design, Data Capture, and Evaluation Approach

This longitudinal observational study assessed educator demographics as well as feasibility and utility of the program. To assess feasibility, we tracked attendance during live-streamed and asynchronous sessions, mean attendance time in minutes, and type of user interface connection used to attend via Webex version 43.4 (Cisco). Utility was defined as educational value and assessed the number of participants (1) requesting CME credit, (2) indicating they learned new information to apply in teaching practice, and (3) attending more than one session, all of which was collected using Qualtrics XM (Qualtrics). Sessions occurred between September 21, 2021, and October 17, 2023.

Data Analyses

Descriptive statistics, including percentiles, means, medians, and ranges were used to analyze both feasibility and utility. All numeric data were analyzed using RStudio (R version 4.2.1) and Microsoft Excel. The mean attendance duration for each individual session topic was applied to the asynchronous session length. Further, we used classical content analysis to analyze text responses on new information learned to identify emergent themes.⁷ This involved coding text responses using open and axial approaches, collapsing or separating codes during consensus meetings (A.M.J., and P.A.C.), identifying and describing emergent themes, and selecting exemplars reflecting themes. All study activities were approved by Oregon Health & Science University’s and University of California, Davis’s Institutional Review Boards (OHSU IRB# 20668, UC Davis IRB# 1666474-8).

Results

Over the 25-month period, 20 bite-sized sessions were offered, and 91 educators registered to participate (Table 1). Of the educators, 75.8% (69 of 91) were physicians while 17.6% (16 of 91) were other program educators.

Table 1.

Demographic and Clinical Training Characteristics of Enrolled COMPADRE Educators

Demographic Characteristics	All COMPADRE Enrollees Combined (N=91), n (%)	Oregon (OR)		California (CA)
		All OR Enrollees n=59 (64.8%), n (%)	Bite-Sized Participant Enrollees (n=16) (17.6%), n (%)	All CA Enrollees n=32 (35.2%), n (%)	Bite-Sized Participant Enrollees (n=12) (13.2%), n (%)
Sex
Female	59 (64.8)	33 (55.9)	9 (56.3)	26 (81.3)	11 (91.7)
Male	31 (34.1)	25 (42.4)	7 (43.8)	6 (18.8)	1 (8.3)
Prefer not to answer	1 (1.1)	1 (1.7)	0 (0.0)	0 (0.0)	0 (0.0)
Age
Under 30	0 (0.0)	0 (0.0)	0 (0.0)	0 (0.0)	0 (0.0)
30-39	29 (31.9)	21 (35.6)	4 (25.0)	8 (25.0)	4 (33.3)
40-49	33 (36.3)	22 (37.3)	6 (37.5)	11 (34.4)	0 (0.0)
50-59	19 (20.9)	9 (15.3)	3 (18.8)	10 (31.3)	5 (41.7)
>60	9 (9.9)	6 (10.2)	3 (18.8)	3 (9.4)	3 (25.0)
Prefer not to answer	1 (1.1)	1 (1.7)	0 (0.0)	0 (0.0)	0 (0.0)
Race
White	78 (85.7)	53 (89.8)	15 (93.8)	25 (78.1)	10 (83.3)
Asian/Pacific Islander	7 (7.7)	4 (6.8)	1 (6.3)	3 (9.4)	1 (8.3)
More than one race	2 (2.2)	1 (1.7)	0 (0.0)	1 (3.1)	1 (8.3)
Prefer not to answer	4 (4.4)	1 (1.7)	0 (0.0)	3 (9.4)	0 (0.0)
Ethnicity
Hispanic	4 (4.4)	1 (1.7)	0 (0.0)	3 (9.4)	2 (16.7)
Non-Hispanic	82 (90.1)	55 (93.2)	16 (100)	27 (84.4)	10 (83.3)
Prefer not to answer	3 (3.3)	1 (1.7)	0 (0.0)	2 (6.3)	0 (0.0)
Missing	2 (2.2)	2 (3.4)	0 (0.0)	0 (0.0)	0 (0.0)
Disadvantages status
Economically disadvantaged	23 (25.3)	12 (20.3)	3 (18.8)	11 (34.4)	2 (16.7)
Educationally disadvantaged	27 (29.7)	16 (27.1)	3 (18.8)	11 (34.4)	4 (33.3)
Clinical Training Characteristics
Role During Training
MD/DO	69 (75.8)	47 (79.7)	13 (81.3)	22 (68.8)	8 (66.7)
Other (MSW, Coordinator, MA)	16 (17.6)	8 (13.6)	2 (12.5)	8 (25.0)	3 (25.0)
Physician’s Assistant	1 (1.1)	0 (0.0)	0 (0.0)	1 (3.1)	1 (8.3)
Missing	5 (5.5)	4 (6.8)	1 (6.3)	1 (3.1)	0 (0.0)
Training Site, n
No. of residency programs	26	15	10	11	7
Attended >1 session	21 (23.1)		12 (75.0)		9 (75.0)

Abbreviations: COMPADRE, California Oregon Partnership to Address Disparities in Rural Education and Health, MSW, Master of Social Work; MA, Master of Arts.

Two to 14 learners viewed each session, either synchronously or asynchronously. A total of 3805 minutes (approximately 63.4 hours) of viewing time occurred among all attendees, including asynchronous sessions. Feasibility, as assessed by user interface connections (eg, desktop, mobile app, the web, or recording), was found to be 50% or greater by desktop app connection for all except one session. Sixteen of 20 (80%) sessions received comments in response to the question, “What new information did you learn that you intend to apply in teaching practice?” Table 2 contains direct quotes and select themes from the qualitative analysis. The themes revealed that participants learned new clinical teaching strategies, identified tools and approaches to support educator development, and recognized that they can participate in education scholarship (Table 2). Only 23.1% (21 of 91) of the participants requested CME credit.

Table 2.

Emergent Themes From Open Ended Responses Regarding New Information to Apply in Teaching Practice

Bite-Sized Session Topic	Number Indicating They Learned New Information to Apply in Teaching Practice	Emergent Themes and Description of New Information to Apply in Teaching Practice	Exemplars
Active Learning Strategies for Didactic Teaching	4	Spacing learning and testing: Concept that spacing out teaching sessions and then testing enhances long-term memory and knowledge retention	“I was surprised about the concept that… spacing out learning is more helpful. I also learned more about study than test. I would like to incorporate this with board review questions from prior topics discussed at didactics to review topics and ‘test.’” [Participant #4] “I need to spend more time ensuring that we test on topics before and following didactics and not just the day of.” [Participant #31] “Testing as a teaching tool is something I probably underutilize. Spacing is another strategy that we could use for certain key and fundamental learning objectives for our didactics program.” [Participant #49]
Basics of Producing Scholarly Work	1	Scholarship is not just possible, but can be enjoyable	“Patty made the process of research and publishing seem like fun, I loved her ideas of breaking it up into manageable chunks and working with others.” [Participant #34]
Bias in Assessment	3	Reflective learning on awareness of and how language used can generate bias	“It was a good review on bias in assessment and the updated literature. I specifically appreciated the list of words and the discussions that came around how and why to use other language.” [Participant #67] “Bias can impact our assessments of learners, which may have a negative impact on their review. Many of these biases that we hold we are not even aware of.” [Participant #34]
Clinical Teaching	2	Teaching tools and strategies that enhance effectiveness as an educator	“Using teaching tools has been helpful for introducing more structure into clinical teaching. I find that when precepting a resident is feeling scattered, this is a great way to get the conversation back on track. It has improved communication between myself and residents and made for more useful feedback.” [Participant #15] “There are multiple methods of teaching, and some are more appropriate than others depending on the level of the learner and the flow of your own clinic. It is important to give feedback.” [Participant #34]
Coaching	0	—	—
Collaborative Scholarly Work	2	Tools and tips to work collaboratively on scholarly activities were valuable	“This was a great presentation on working on scholarly activity as a collaborative project, the manuscript outline was incredibly helpful, as was the offer by Patty to help be a mentor on scholarly activities.” [Participant #34]
Community Immersion in Your Program	7	Sharing ideas on ways learners can engage with community groups was perceived as benefitting both the learners and community groups	“Our…faculty discussed various ways to engage more with the community. We really liked the idea of setting residents up with service organizations (ie, Rotary, Soroptomist) and will work on getting that into our curriculum. We also liked the community walk around suggestion for students and new residents. Great talk with a lot of ideas today.” [Participant #25] “This was helpful to reflect on what community engagement looks like for… [our specific] students versus what all students at UCD SOM would benefit from.” [Participant #48]
Engaging in Feedback 1	1	Learning to provide actionable feedback (provided in enough detail that the learner understands exactly what they need to do to improve) was perceived as valuable by individual participants	“It is important to give feedback that is specific and timely to the situation, so it can be implemented by the learner. Feedback that is too general is not helpful.” [Participant #34] “Providing feedback is sometimes daunting! I appreciate the feedback tips, including BEST and ARTful feedback. I think we may try to put it into practice at future faculty meetings.” [Participant #3]
Engaging in Feedback 2	3
How to Develop Your CCC (Clinical Competency Committee)	2	Learning how to develop, evolve, and maintain a CCC is considered valuable for the residency	“I learned what is necessary to develop a CCC, who should participate, that the residency director should not chair it, and they should advise, especially regarding competency of residents based on regular evaluations and what those evaluations should be structured like.” [Participant #27] “Our CCC has undergone many changes over the past year. This presentation provided an excellent framework for organizing the process that we have started and making our goals more clear.” [Participant #69]
Individualized Learning Plans (ILP)	1	New undertaking where educational topic was timely and valuable for the residency	“We are just starting the process of rolling out ILPs for every resident. It was great to get this perspective and model to bring back to our faculty.” [Participant #69]
Learner Assessment	3	Reflective learning where stimulating recall of what participants knew but have forgotten to apply in current practice is valuable to individuals Alternative approaches to goal-setting with learners where switching from SMART (specific, measurable, attainable, relevant, timely) to WOOP (wish, outcome, obstacle, plan) was valuable to educators	“Some of the information presented during the presentations is not new. However, it is so valuable because I am able to reflect on what I’m doing and not doing. Then I can make changes that are appropriate in real time that increase my ability to teach and coach more effectively.” [Participant #62] “I liked learning about WOOP as an alternative to SMART. I will be using this in the future to help frame coaching conversations with residents and attendings. As mentioned at the beginning of the talk, coaching strategies can be relevant to many different situations and do not have to be focused on work/career issues.” [Participant #37] “The WOOP versus SMART goal setting. We will discuss this option with our learners and encourage them to download the app. This will help with learner feedback and improving preceptor teaching.” [Participant #25]
Master Adaptive Learner (MAL) Conceptual Framework	7	A nuanced and powerful way to think about learning—the MAL Framework was perceived as a valuable way to “stimulate” learning rather than “delivering” education	“The discussion on being curious with the learner was different. Asking the learner to be curious about the issue at hand and sort through what they do and do not know is a good tip.” [Participant #25] “Interesting to change the focus to encouraging residents to become master learners and teach them how to find the information rather than teach them all the information.” [Participant #34] “Appreciated the MAL framework and the concept that we are ‘really bad’ at self-assessments. Need to validate external assessments.” [Participant #39] “Learned the structure of the MAL and considering adapting my precepting technique for clinical encounters.” [Participant #42] “How to triage and prioritize learning. How to identify gaps via self-assessment.” [Participant #62]
Match Reversions	4	Learning how to utilize Match reversions to retain residents they want to keep in their program was perceived as valuable for both the residency and their communities	“We are highly motivated to create and maintain a pathway for students to return to communities of high need once they graduate. We are thinking of utilizing Match revisions to facilitate COMPADRE students to match to and train at our program.” [Participant #3] “Thank you for summarizing how to do a reversion with NRMP. This is the next best option besides a Match waiver for getting students into the program. We are considering doing a reversion in the future and it was helpful to hear how to set it up and why to set it up. Nice to know this is available.” [Participant #25] “It was very interesting to hear about residency programs using more than one tract to match residents, saving a certain number of spots for specific groups of students such as COMPADRE students.” [Participant #34]
Mental Skills Training	5	Exercises included in this session were perceived as beneficial for becoming better educators when working with residents in clinical settings	“Our faculty participated today live and we all felt that visualization and going through each procedure step with the learner is beneficial. You can use these techniques when preparing for a difficult patient encounter, end of life talk, or other challenging discussion with colleagues. Mindfulness is an excellent way to help others and yourself and my goal is to become certified in this technique.” [Participant #25] “I really like the idea of incorporating mental skills training prior to performing a new procedure. I see the value in using these skills to help residents before entering a stressful situation.” [Participant #34] “I anticipate using guided visualization as part of upcoming skills training with residents, prior to engaging in hands on practice, and helping them build this into their preparation for procedures as a way to make mental checklists, so that they can remember each part of the procedure.” [Participant #15]
Peer Observation of Teaching	3	Peer-to-peer feedback is perceived as novel and not something that is routinely done in residency training programs but should be	“I would love to institute peer observations of teaching styles. We do not currently have this as a process in our program and I think it would be beneficial for both the observer and the observee.” [Participant #34] “I thought the BEST and ART mnemonics were helpful. I will use them to help deliver more effective feedback in the future. I will especially work on making certain that my feedback is timely.” [Participant #62] “A lot of teaching in our program is often just on self-reflection or resident evaluation of attendings to the program after their specific rotation. We have not instituted attending-to-attending feedback on teaching skills in the past and this seems like it may be a worthwhile endeavor to improve our quality of teaching in a variety of settings.” [Participant #3]
Reflective Practice and Life-Long Learning	0	—	—
Supporting Struggling Learners	0	—	—
Using the ACGME Clinician Educator Milestones	3	Perception that using milestones will benefit all those working in the residency regardless of the type of training or role in the program they have	“I would like to use these milestones to inform faculty feedback and learning plans for new faculty. I think this can be a good resource for us to guide our faculty development initiatives.” [Participant #69] “Clinician educator milestones apply to not just clinicians but everyone working with learners. Not all milestones apply to all educators. The milestones are for self-assessment not accreditation.” [Participant #34]
What Is an EPA	0	—	—

Abbreviations: UCD SOM, University of California, Davis School of Medicine; COMPADRE, California Oregon Partnership to Address Disparities in Rural Education and Health; NRMP, National Resident Matching Program; ACGME, Accreditation Council for Graduate Medical Education; EPA, entrustable professional activity.

Discussion

Findings from this exploratory study indicate that it is feasible to provide interactive web-based faculty development in resource-limited settings and that a virtual program offering 25-minute development opportunities has utility for participants. Most participants took part in real time and utilized a desktop device, which suggests they could access the sessions in their clinic offices, precepting rooms, or at home. We did find that mobile connections were also used, suggesting that participants may login from several locations, depending on connectivity.

Based on the wide-ranging comments received about the sessions, it appears they held educational value for many participants. Notable examples include that participants felt like the instruction on feedback helped them better understand how to provide actionable feedback and that reflective learning provided an opportunity for stimulated recall of what participants already knew but had not been applying in their clinical practice.

This study makes important contributions to the literature by providing an understanding of the feasibility and utilization of implementing distance learning for educators in underresourced settings. Distance learning has been studied in other countries, such as Canada and India, where large medical schools depend on various community-based settings to train physicians in rural physician shortage areas.^8-10 The 2003 Canadian study by Curran¹⁰ is most like the study we conducted, though it is now 20 years old. The investigators conducted 31 one-hour audioconferences, each of which was attended by approximately 14 educators. They found that 67% (12 of 18) accessed the program either at work or from home and that the presentation materials and ability to submit evaluations online were beneficial. Similar research using the same method of continuous professional development has not been conducted in the United States, where distributed medical education is becoming increasingly common.¹¹

Because we have demonstrated feasibility and utility, there is value in continuing to evolve this important work. Areas for future development include conducting more in-depth needs assessments that include current faculty knowledge of and comfort with teaching, and assessing what impediments exist to attending in real time or in reviewing the recordings. Additionally, identifying more rural clinicians who may be naïve to teaching students and residents and more rigorously evaluating the quality and educational value of sessions offered will be important for future studies. Plans are for the network of COMPADRE residency programs to sustain this program after the funding period is complete; however, administrative and faculty coordinator support will continue in-kind.

Limitations include a small sample size, which limits generalizability to other programs and settings. Furthermore, participants were asked to self-report what they learned and intend to apply to their teaching practice. Conducting an additional assessment of whether participants applied what they learned in actual teaching practice will strengthen future studies. Finally, data collection on the reliability of mobile connectivity could give insight into additional options to access virtual content.

Conclusions

Brief, web-based educator development sessions are feasible to implement in residency training sites located in remote and underresourced communities. Participants self-reported their intention to apply what they learned in their work with trainees.

Author Notes

Funding: This work was funded by the American Medical Association Reimagining Residency grant program.

Conflict of interest: The authors declare they have no competing interests.

Editor’s Note

The online supplementary data contains the survey used in the study.