Abstract
Background Resident-as-teacher initiatives are traditionally specialty-specific and performed in-person, limiting ability to disseminate essential teaching skills to all residents.
Objective The aim of this study was to develop, implement, and evaluate a resident-as-teacher interactive e-learning module on growth mindset and coaching.
Methods The module was designed and implemented between August 2022 and March 2023. It was distributed to postgraduate year (PGY) 1 residents in all specialties at a large academic institution. Completion rates, Likert ratings, and answers to 2 open-ended questions were used for assessment. Descriptive statistics and 1-way analysis of variance with Sîdák correction for multiple comparisons were performed on Likert ratings. Responses to open-ended questions were evaluated using content analysis.
Results The module was completed by all 277 PGY-1 residents (100%), with the evaluation completed by 276 of 277 (99.6%) residents. Mean rating of the module’s relevance to the role of resident teacher was 4.06±0.90 (5-point Likert scale), with general surgery residents rating the module less favorably compared to all specialties (3.28±1.06; P<.01; 95% CI 0.26-1.30). Open-ended comments revealed that residents most liked the delivery of relevant teaching strategies and the interactive design of the module. The most common area for suggested improvement was the addition of content such as teaching in challenging situations. Time needed for design, implementation, and evaluation was 80 hours total.
Conclusions An e-learning module offers an interactive platform for teaching skills and was found to be an acceptable method of instruction for residents.
Introduction
The Liaison Committee on Medical Education requires that residents are adequately prepared for their teaching roles.1 While in-person approaches have proven effective in imparting essential educational skills, guaranteeing universal exposure to this information is challenging, given the diverse schedules and clinical obligations residents face.2 The increasing use of e-learning has changed the ability to disseminate educational concepts to a wide audience.
E-modules have previously been successfully utilized to teach clinical and procedural knowledge in graduate medical education (GME), showing the potential of technology to foster the development of competent clinician educators.3-10 Concurrent with the rise of e-learning, novel educational concepts such as growth mindset and coaching have emerged.11-16 These concepts align with competency-based education, redirecting the focus from achieving perfection to fostering a culture of continuous learning.11
Our goal was to design, implement, and evaluate acceptability of an interactive resident-as-teacher e-learning module focused on utilizing growth mindset and coaching techniques in teaching medical students. We hypothesized that by employing an e-learning platform and incorporating contemporary educational concepts, we could achieve a learning experience that was accessible and relevant to residents across specialties.
Methods
Setting and Participants
This study was performed at a large academic institution with 21 accredited residency programs, containing 277 postgraduate year (PGY) 1 residents across all specialties. E-learning module development, implementation, and evaluation were framed with a logic model (provided as online supplementary data).17
Inputs
The module was created by the GME Resident-as-Teacher Subcommittee faculty chair (B.M.R.) and members of the Department of Education, Innovation, and Technology (S.J.L., G.C.B.), utilizing an existing resource within the institution, Articulate 360, a web-based e-learning course creation application.
Activities:
From August to December 2022, the module was created using the 5E learning model of engage, explore, explain, elaborate, and evaluate for instructional design, incorporating content on psychological safety, growth mindset, and coaching.11,15,18,19 It featured the R2C2 method, a reflective model for facilitating performance feedback that consists of 4 phases: relationship building, exploring reactions, exploring content, and coaching.20
Residents who had been recognized as exceptional educators also recorded 30-second video vignettes to give institution-specific examples of best practices. Formative assessments were included throughout the module, and although residents were able to see immediate feedback, their answers were unable to be tracked for data collection.
Outputs:
In February 2023, the module was released via Blackboard Learn (Anthology Inc), a learning management system, with the requirement that all PGY-1 residents complete it within 1 month. Completion was tracked by the GME administrative team.
Outcomes Measured
Completion data, resident specialty, and answers to a 3-question evaluation (provided as online supplementary data) were obtained through Blackboard, preserving anonymity of respondents. The evaluation consisted of rating the module’s relevance (5-point Likert scale) and 2 questions asking residents to identify an aspect of the module they liked and an aspect they would change.
Analysis of Outcomes
Descriptive statistics were performed for response rate and overall module rating. One-way analysis of variance with Sîdák correction was used to compare Likert ratings of individual specialties with all specialties. Content analysis was used to review comments for patterns by 2 team members (B.M.R., A.E.R.).21 Both independently reviewed the comments, met to create categories, and then independently coded comments. In a final joint discussion, the reviewers achieved 90% concordance on aspects that residents liked and 82% concordance on aspects that residents would change, with coding then modified to resolve any differences.
Time for this project was supported by the Baylor College of Medicine Norton Rose Fulbright Educational Grant ($4,929). This study was approved by the Baylor College of Medicine Institutional Review Board.
Results
A completion rate of 100% was achieved, with 271 of 277 (97.8%) completing within the designated 1-month period, and 6 additional residents (2.2%) completing during a remediation period. The Likert evaluation was completed by 276 of 277 (99.6%) residents, with an average rating of 4.06±0.90 on a 5-point Likert scale. Table 1 shows the distribution of residents’ ratings by specialties. The module was rated highest by family medicine (4.36±0.50) and pediatrics (4.36±0.62) and lowest by general surgery (3.28±1.06) and surgical subspecialties (3.63±1.05), with general surgery residents’ ratings significantly lower (P<.01, 95% CI 0.26-1.30) compared to all specialties.
Table 1.
Resident Likert Rating Response to “How Relevant Is This Module to Your Role as a Resident Teacher?”
| Specialtya | Total Respondents, n (%) | Mean Rating±SD | Not at All Relevant (1), n (%) | A Little Relevant (2), n (%) | Somewhat Relevant (3), n (%) | Very Relevant (4), n (%) | Extremely Relevant (5), n (%) | P valueb | 95% CI |
| All specialties | 276 (100) | 4.06±0.90 | 2 (0.7) | 15 (5.4) | 47 (17.0) | 112 (40.6) | 100 (36.2) | ||
| Anesthesiology | 13 (4.7) | 3.69±0.63 | 1 (7.7) | 2 (15.4) | 10 (76.9) | .74 | -0.34-1.08 | ||
| Emergency medicine | 16 (5.8) | 3.93±0.93 | 1 (6.2) | 4 (25.0) | 6 (37.5) | 5 (31.3) | .98 | -0.52-0.77 | |
| Family medicine | 11 (4.0) | 4.36±0.50 | 7 (63.6) | 4 (36.4) | .86 | -1.08-0.47 | |||
| Internal medicine | 70 (25.4) | 4.20±0.83 | 2 (2.9) | 12 (17.1) | 26 (37.1) | 30 (42.9) | .89 | -0.47-0.19 | |
| Internal medicine/pediatrics | 8 (2.9) | 4.25±0.89 | 2 (25.0) | 2 (25.0) | 4 (50.0) | .98 | -1.09-0.71 | ||
| Neurology | 9 (3.3) | 4.11±0.78 | 2 (22.2) | 4 (44.5) | 3 (33.3) | .98 | -0.90-0.80 | ||
| Obstetrics and gynecology | 12 (4.3) | 4.00±0.95 | 1 (8.3) | 2 (16.7) | 5 (41.7) | 4 (33.3) | .98 | -0.67-0.80 | |
| Pathology | 7 (2.5) | 3.71±1.38 | 2 (28.6) | 1 (14.3) | 1 (14.3) | 3 (42.9) | .89 | -0.61-1.31 | |
| Pediatrics | 68 (24.6) | 4.36±0.62 | 5 (7.4) | 33 (48.5) | 30 (44.1) | .11 | -0.65-0.03 | ||
| Psychiatry | 13 (4.7) | 4.15±0.98 | 1 (7.7) | 2 (15.4) | 4 (30.8) | 6 (46.2) | .98 | -0.81-0.62 | |
| Surgery (general) | 25 (9.0) | 3.28±1.06 | 1 (4.0) | 5 (20.0) | 8 (32.0) | 8 (32.0) | 3 (12.0) | <.01c | 0.26-1.30 |
| Surgical subspecialties | 24 (8.7) | 3.63±1.05 | 1 (4.2) | 2 (8.3) | 7 (29.2) | 9 (37.5) | 5 (20.8) | .18 | -0.09-0.97 |
Two pediatrics and 2 family medicine residency programs at different sites were merged under their general specialty for comparison. Combined programs that were an integration of 2 specialties (ie, internal medicine/genetics), having 6 or fewer residents, were placed under the larger of the 2 specialties. Similarly, residents in neurosurgery, orthopedic surgery, otolaryngology, plastic surgery, thoracic surgery, urology, and vascular surgery were combined into one group named surgical subspecialties.
One-way analysis of variance with Sîdák correction for multiple comparisons in relation to “all specialties.”
Indicates significance at P<.05.
A total of 256 of 277 (92.4%) residents provided comments for “What did you like about this module?” and 230 of 277 (83.0%) provided comments for “What would you change about this module?” Some comments had multiple patterns for a total of 354 coded comments for the first question and 256 coded comments for the second question. Content analysis and representative examples are displayed in Table 2. The most common areas residents liked were relevant teaching strategies (30.2%, 107 of 354), interactive design (21.5%, 76 of 354), and efficient delivery (20.1%, 71 of 354). The most requested change was addition of content such as teaching in challenging situations, more interactive scenarios, or a summary page (36.0%, 91 of 256).
Table 2.
Open-Ended Questions Content Analysis
| Categories | n (%) | Representative Comments |
| “What did you like about this module? Name at least 1 element.” (N=354) | ||
| Teaching strategies | 107 (30.2) | |
| Relevant tips | 82 (23.2) | “Relevant ways to incorporate teaching into my daily practice as a resident.” “I liked having examples of how to express certain ideas or sentiments. I have a few phrases I can use to introduce ideas delicately now.” “Relevance to our daily training. “ |
| R2C2 | 25 (7.1) | “I thought the example of the R2C2 model was very informative and applicable.” “I enjoyed the R2C2 method discussion.” |
| Interactive design | 76 (21.5) | |
| Interactive | 48 (13.6) | “Was engaging and interactive.” “I liked how interactive the module is.” |
| Visual design | 28 (7.9) | “I liked the layout and design—kept me engaged.” “The graphic design was nice.” |
| Efficiency | 71 (20.1) | |
| Concise | 38 (10.7) | “I liked that it was concise—it got all of the information across in an engaging but not overly detailed way.” “It was short and to the point.” |
| Ease of completion | 33 (9.3) | “It was easy to follow.” “Easy navigation.” |
| Educational concepts | 43 (12.1) | |
| Coaching | 28 (7.9) | “I really liked the emphasis on giving regular feedback through a coaching approach throughout a rotation as a form of teaching students and helping them improve.” “Liked discussion on coaching.” |
| Growth mindset | 10 (2.8) | “I really enjoyed the part about growth vs fixed mindset and encouraging learners to have a better outlook on learning and encouraging them to be comfortable making mistakes.” “Focusing on growth mindset.” |
| Psychological safety | 5 (1.4) | “I liked the graph about psychological safety and accountability, really puts things into perspective, and reflects how I’ve felt learning as a medical student and now as an intern.” “I liked the visual of the triangle breakdown of types of threats that can prevent a student from succeeding in their learning environment.” |
| Resident testimonials | 37 (10.5) | “I liked the involvement of current residents at [our medical school] who can speak to the various elements that the module is conveying.” “I liked that there were examples of good teaching from my peers.” |
| Sentiment | 20 (5.6) | |
| Positive | 18 (5.1) | “The content of the module is great.” “Very informative.” |
| Negative | 2 (0.6) | “I would have preferred an in person interactive lecture, honestly, if [the medical school] truly intends to get this point across to residents.” “Unneeded.” |
| “What would you change about this module? Name at least 1 element.” (N=256) | ||
| Add content | 91 (36.0) | |
| Additional examples | 64 (25.0) | “I think it would be helpful to add strategies to effectively teach on busy services when time is not abundant.” “It could perhaps benefit from a section on how to coach students who do not have a growth mindset.” “Providing student experiences that have been negative—would be a good way to actualize the impact residents have on students.” |
| Interactive | 20 (7.8) | “I would provide more interactive real-life scenarios.” “I would make it more interactive.” |
| Summary | 7 (2.7) | “A summary page at the end to briefly look over or possibly print would be nice.” |
| Positive comments/no changes | 67 (26.2) | “Nothing, it was very straightforward and organized.” “There is nothing I would change about the module.” “I would not change anything; I found the course very engaging.” |
| Reduce content | 48 (18.8) | |
| Length | 29 (11.3) | “[I would] change the length…less reading.” “It’s too long.” |
| Videos | 19 (7.4) | “Could shorten the video segments further to include just salient points, instead of multiple videos, each emphasizing one main point.” “Probably don’t need as many video testimonials at the beginning.” |
| Procedural changes | 25 (9.8) | “Adding subtitles to the videos would help with accessibility.” “More reminders about completion and deadline.” “The attestation should be contained within the module.” “Blackboard can be tough to navigate.” |
| Cumbersome | 19 (7.4) | |
| Clicking | 11 (4.3) | “Many separate clicks required for interactive portions.” |
| Belabored | 8 (3.1) | “Some parts of the lesson seemed a bit belabored.” |
| Not applicable | 6 (2.3) | “Not sure how much it will change/help improve my daily practice of working with students.” |
Total time commitment for creation, implementation, and evaluation of the module was 45 hours for the GME faculty involved in this project, 30 hours for the members of the Department of Education, Innovation, and Technology, and 5 hours for residents.
Discussion
By utilizing resident testimonials and engaging representatives from diverse specialties, we implemented a comprehensive approach to create an institution-wide curriculum. In total, this module was accessible and relevant to PGY-1 residents, although less so for general surgery residents.
Three important lessons learned were to have trainees as co-collaborators, involve specialists in technology early, and be inclusive of all specialties. The TPACK (technological, pedagogical, and content knowledge) model organized our approach to introducing novel technology into our resident-as-teacher curricula.22 This framework emphasizes the 3 essential components of technological, pedagogical, and content knowledge for educational technology integration. While GME trainees and educators provided the content and pedagogical knowledge, collaboration with our internal Department of Education, Innovation, and Technology allowed creation of an interactive and engaging educational experience, with open-ended comments showing a desire for more content.
Although surgical subspecialty residents were included in module creation, general surgery residents were not. General surgery residents face unique challenges in instructing novice students on operating room protocols and basic surgical principles, ranging from suturing to trauma management.23,24 In the future, modules will include content applicable to techniques used frequently in surgical specialties, such as teaching procedural skills, teaching under time constraints, and teaching in high-stakes environments.25
By utilizing a learning management system, module compliance and reminders were easily performed by the GME office, and there was limited burden on program directors and coordinators. The PGY-1 module will continue, and PGY-2 and PGY-3 modules will be launched, with continued focus on coaching and growth mindset and additional procedural content to ensure applicability to all specialties. Other institutions could also use this template featuring their own resident videos to maintain a personal touch.
This project focused on exploring residents’ satisfaction with an e-module, ensuring it is user-centered and engaging. Our initial concern was that these modules would be met with cynicism; however, open-ended comments showed that the residents desired more information about these novel educational topics. A limitation is that we did not assess the module’s effectiveness. Future directions include adding assessments of knowledge gains and resident behavior changes.
Conclusions
This study demonstrates the successful development, implementation, and acceptability of an e-module for resident-as-teacher training. Delivery of relevant teaching strategies in an interactive manner was found to be effective for PGY-1 residents across specialties.
Supplementary Material
Editor’s Note
The online supplementary data contains a logic model for design and implementation of resident-as-teacher e-module and the survey used in the study.
Author Notes
This work was previously presented as an abstract at the Baylor College of Medicine Showcase of Educational Scholarship, Houston, Texas, USA, September 15, 2023.
Funding: Work on this project was supported by the Norton Rose Fulbright Educational Grant Fund from Baylor College of Medicine.
Conflict of interest: The authors declare they have no competing interests.
References
- 1.Liaison Committee on Medical Education Functions and structure of a medical school: standards for accreditation of medical education programs leading to the MD degree. Accessed July 1, 2023. http://lcme.org/publications/
- 2.Yu T-C, Hill AG. Implementing an institution-wide resident-as-teacher program: successes and challenges. J Grad Med Educ . 2011;3(3):438–439. doi: 10.4300/JGME-03-03-29. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Han H, Resch DS, Kovach RA. Educational technology in medical education. Teach Learn Med . 2013;25(suppl 1):39–43. doi: 10.1080/10401334.2013.842914. [DOI] [PubMed] [Google Scholar]
- 4.Han ER, Yeo S, Kim MJ, Lee YH, Park KH, Roh H. Medical education trends for future physicians in the era of advanced technology and artificial intelligence: an integrative review. BMC Med Educ . 2019;19(1):460. doi: 10.1186/s12909-019-1891-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Chretien KC, Yarris LM, Lin M. Technology in graduate medical education: shifting the paradigm and advancing the field. J Grad Med Educ . 2014;6(2):195–196. doi: 10.4300/JGME-D-14-00157.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Bettendorf B, Quinn-Leering K, Toth H, Tews M. Teaching when time is limited: a resident and fellow as educator video module. Med Sci Educ . 2019;29(3):631–635. doi: 10.1007/s40670-019-00731-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.de Leeuw R, de Soet A, van der Horst S, Walsh K, Westerman M, Scheele F. How we evaluate postgraduate medical e-learning: systematic review. JMIR Med Educ . 2019;5(1):e13128. doi: 10.2196/13128. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Kaplovitch E, Otremba M, Morgan M, Devine LA. Cost-efficient medical education: an innovative approach to creating educational products. J Grad Med Educ . 2019;11(6):713–716. doi: 10.4300/JGME-D-19-00155.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Farr E, Roth E. Virtual learning in graduate medical education: applying learning theory for effective educational videos. J Grad Med Educ . 2021;13(6):757–760. doi: 10.4300/JGME-D-21-00245.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Wittich CM, Agrawal A, Cook DA, et al. E-learning in graduate medical education: survey of residency program directors. BMC Med Educ . 2017;17(1):114. doi: 10.1186/s12909-017-0953-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Richardson D, Kinnear B, Hauer KE, et al. Growth mindset in competency-based medical education. Med Teach . 2021;43(7):751–757. doi: 10.1080/0142159X.2021.1928036. [DOI] [PubMed] [Google Scholar]
- 12.Wolcott MD, McLaughlin JE, Hann A, et al. A review to characterise and map the growth mindset theory in health professions education. Med Educ . 2021;55(4):430–440. doi: 10.1111/medu.14381. [DOI] [PubMed] [Google Scholar]
- 13.Klein J, Delany C, Fischer MD, Smallwood D, Trumble S. A growth mindset approach to preparing trainees for medical error. BMJ Qual Saf . 2017;26(9):771–774. doi: 10.1136/bmjqs-2016-006416. [DOI] [PubMed] [Google Scholar]
- 14.Osman NY, Sloane DE, Hirsh DA. When I say…growth mindset. Med Educ . 2020;54(8):694–695. doi: 10.1111/medu.14168. [DOI] [PubMed] [Google Scholar]
- 15.Brown LE, Rangachari D, Melia M. Beyond the sandwich: from feedback to clinical coaching for residents as teachers. MedEdPORTAL . 2017;13:10627. doi: 10.15766/mep_2374-8265.10627. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Rangachari D, Brown LE, Kern DE, Melia MT. Clinical coaching: evolving the apprenticeship model for modern housestaff. Med Teach . 2017;39(7):780–782. doi: 10.1080/0142159X.2016.1270425. [DOI] [PubMed] [Google Scholar]
- 17.Balmer DF, Rama JA, Simpson D. Program evaluation models: evaluating processes and outcomes in graduate medical education. J Grad Med Educ . 2019;11(1):99–100. doi: 10.4300/JGME-D-18-01084.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Jun WH, Lee EJ, Park HJ, Chang AK, Kim MJ. Use of the 5E learning cycle model combined with problem-based learning for a fundamentals of nursing course. J Nurs Educ . 2013;52(12):681–689. doi: 10.3928/01484834-20131121-03. [DOI] [PubMed] [Google Scholar]
- 19.Tsuei SHT, Lee D, Ho C, Regehr G, Nimmon L. Exploring the construct of psychological safety in medical education. Acad Med . 2019;94(suppl 11 Association of American Medical Colleges Learn Serve Lead: Proceedings of the 58th Annual Research in Medical Education Sessions):28–35. doi: 10.1097/ACM.0000000000002897. [DOI] [PubMed] [Google Scholar]
- 20.Sargeant J, Lockyer J, Mann K, et al. Facilitated reflective performance feedback: developing an evidence- and theory-based model that builds relationship, explores reactions and content, and coaches for performance change (R2C2) Acad Med . 2015;90(12):1698–1706. doi: 10.1097/ACM.0000000000000809. [DOI] [PubMed] [Google Scholar]
- 21.Stemler S. An overview of content analysis. Pract Assess Res Eval . 2001;7(1):17. doi: 10.7275/z6fm-2e34. [DOI] [Google Scholar]
- 22.Youm J, Corral J. Technological pedagogical content knowledge among medical educators: what is our readiness to teach with technology? Acad Med . 2019;94((suppl 11 Association of American Medical Colleges Learn Serve Lead: Proceedings of the 58th Annual Research in Medical Education Sessions)):69–72. doi: 10.1097/ACM.0000000000002912. [DOI] [PubMed] [Google Scholar]
- 23.Gifford E, Galante J, Kaji AH, et al. Factors associated with general surgery residents’ desire to leave residency programs: a multi-institutional study. JAMA Surg . 2014;149(9):948–953. doi: 10.1001/jamasurg.2014.935. [DOI] [PubMed] [Google Scholar]
- 24.Pelletier M, Belliveau P. Role of surgical residents in undergraduate surgical education. Can J Surg . 1999;42(6):451–456. [PMC free article] [PubMed] [Google Scholar]
- 25.Geary AD, Hess DT, Pernar LIM. Resident-as-teacher programs in general surgery residency: context and characterization. J Surg Educ . 2019;76(5):1205–1210. doi: 10.1016/j.jsurg.2019.03.004. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.