Abstract
Purpose
We sought to create a resident educator program using a Train-the-Trainer (TTT) approach with adaptable curricula at a large tertiary health care center with a medical school and 60 accredited residency programs.
Methods
The Resident Educator And Life-long Learner (REALL) Program was designed as a 3-phase model. Phase 1 included centralized planning and development that led to the design of 7 teaching modules and evaluation tools for TTT and resident sessions. Phase 2 entailed the dissemination of the TTT modules (Learning Styles, Observational Skills, Giving Feedback, Communication Skills: The Angry Patient, Case-Based Teaching, Clinical Reasoning, Effective Presentations) to faculty trainers. In phase 3, specific modules were chosen and customized by the faculty trainers, and implemented for their residents. Evaluations from residents and faculty were collected throughout this process.
Results
A total of 45 faculty trainers representing 27 residency programs participated in the TTT program, and 97% of trainers were confident in their ability to implement sessions for their residents. A total of 20 trainers from 11 residency programs implemented 33 modules to train 479 residents, and 97% of residents believed they would be able to apply the skills learned. Residents' comments revealed appreciation of discussion of their roles as teachers.
Conclusion
Use of an internal TTT program can be a strategy for dissemination of resident educator and life-long learner curricula in a large academic tertiary care center. The TTT model may be useful to other large academic centers.
Editor's Note: The online version of this article contains the planning checklist, the agenda outlines, 3 faculty surveys, and a resident evaluation for giving feedback (131KB, doc) .
Background
Resident physicians contribute substantially to medical students' education, with one study indicating at least one-third of medical students' knowledge was gained through clinical interactions with residents.1,2 Research suggests that students find residents to be valuable teachers;3–5 and for teaching practical skills residents were rated as more effective than faculty.5 Residents also teach junior residents, and supervising residents are interns' first-line teachers, acting as role models, answering questions on call, and providing guidance during work rounds.6,7 Residents derive benefit for their education and professional development from acting in the role of teacher, evaluator, and role model.8,9 Recognition of the impact of resident teaching on medical students and junior residents has led the Liaison Committee on Medical Education and the Accreditation Council for Graduate Medical Education to require preparation of residents for this teaching role.
Although program directors acknowledge residents' close involvement with the education of medical students, surveys reveal that less than 55% of programs offer instruction on teaching.10,11 One possible reason that so few programs have developed formal “resident as teacher” curricula may be that they may lack adequate time, resources, and training. We propose a centralized development and distributed implementation model featuring a Train-the-Trainer (TTT) approach for the development of a resident educator curriculum. TTT programs have been used in medicine for national dissemination to improve faculty clinical teaching (Stanford Faculty Development Program),12 and for enhancing physician communication skills (American Academy of Communications in Healthcare).13 Two literature reviews addressing resident as teacher curricula did not mention use of TTT programs as a specific strategy to teach residents, although both reviews concluded that resident teaching courses improve resident confidence and teaching skills.14,15
The contribution of this study to the literature on TTT programs is that it expands the program evaluation to include curriculum content, and dissemination and implementation strategies in an environment where faculty have limited time and resources. Our goal was to develop, distribute, and evaluate an adaptable resident educator and life-long learner curriculum in 60 accredited residency programs at the Cleveland Clinic, a teaching institution with nearly 1000 residents. This article describes the development for the Resident Educator And Life-long Learner (REALL) Program and the results of this curriculum on faculty trainers (FTs) and residents.
Methods
Participants
Faculty participants, or FTs, were program directors and specialty-specific faculty selected for their excellence in teaching. Resident participants attended sessions taught by FTs. The Institutional Review Board declared this study exempt for human subject research approval.
Curricular Interventions/Program Description
The REALL Program was designed as a 3-phase program (f i g u r e). Phase 1 encompassed the planning process. Phase 2 was a faculty TTT program. Phase 3 was FTs' implementation of the core curriculum to residents, known as resident training sessions.
Phase 1
Phase 1 established a planning group, known as the REALL Committee. The group was composed of 7 faculty members (6 MDs and 1 PhD) representing medical education, internal medicine, geriatrics, allergy, ophthalmology, pediatrics, and psychiatry. The committee chose a centralized development and distributed implementation model, based on 3 factors. First, residents as teachers share common instructional needs, such as giving feedback and case-based teaching. Second, duplication of efforts could be reduced by designing “generic” modules that could later be tailored by FTs. Third, the committee believed that specialty-specific faculty would have greater credibility and impact on residents within their program.
A search of “resident as teacher” in the medical education literature and a critique of various resident educator programs available online provided sources to determine key topics and potential implementation models. After a thorough review, the committee determined that the TTT model would be the best approach for our institution. Seven core topics (Learning Styles, Observational Skill, Giving Feedback, Communication Skills: The Angry Patient, Case-Based Teaching, Clinical Reasoning, Effective Presentations) were approved for development by the Cleveland Clinic GME Council (GMEC), an oversight committee in graduate medical education.
The module Giving Feedback was designed by a medical education specialist (C.A.T.). It was used as a prototype for the development of the 6 other modules. Module designers were given a curriculum development template to assure a scholarly and standardized approach to instructional module design (b o x 1). Materials for each module included: behavioral objectives, lesson plan, prereading, annotated PowerPoint (Microsoft, Redmond, WA) presentation, group activities, handouts, references, resident evaluation forms, and trigger videos when appropriate. Novel aspects of the module materials were the lesson plan, annotated PowerPoint slides, planning checklist (provided as online supplemental material) and tentative outline agenda (provided as online supplemental material). All material components were reproduced and made available to FTs on CDs and print materials in binders.
BOX 1 Curriculum Design Template for Module Giving Feedback
UNIT NAME: Redefining, Giving, and Receiving Feedback
COURSE OBJECTIVE ADDRESSED: Observe and provide meaningful verbal feedback to students and more junior residents
EDUCATIONAL OBJECTIVES:
By the conclusion of the Giving Feedback module, the residents will be able to:
Define “useful feedback.”
Describe a “feedback agreement” and explain why it is important in creating a learning environment.
List the characteristics of “Quick” and “Formal” feedback and describe the indications for using both.
Use the ASK-TELL-ASK method to give feedback to a peer during a “simulation.”
COMPETENCIES ADDRESSED:
Communication skills
KEY ISSUES:
“Useful feedback”: defined as information that leads to improved performance
Shared goals of “improved performance”
Sources of feedback—paying attention to the environment as well as teachers
ASK-TELL-ASK method
Feedback agreement/setting expectations
THE INSTRUCTOR/FACILITATOR WILL:
Provide a prereading.
Lead the discussion of “useful feedback” concept.
Introduce the ASK-TELL-ASK method.
Provide the video triggers, simulation cases, and materials.
Provide feedback and suggestions.
THE RESIDENTS WILL:
Read the prereading.
Participate in discussion and exercises.
EVALUATION
Level one: Residents' opinion of the usefulness of the workshop
Level two: Residents' recall of major concepts
Level three: Monitor student's assessments of resident ability to give feedback
RESOURCES AVAILABLE WITH THIS UNIT
PowerPoint presentation
Laminated cards
Video triggers (to be developed for resident)
Paper case scenario
Phase 1 took 6 months to complete. The cost of designing, developing, and implementing one module was $5550, determined by itemizing faculty support to prepare the module, administrative support, production of module materials (CD copying, DVD copying for video triggers), and expenses for food and beverages. FTs used their allotted educational administration time for residency training to attend the TTT modules and to implement resident sessions. Use of CDs allowed modification of materials and kept audio-visual and information technology needs to a minimum (equipment for viewing PowerPoint slides and DVD videos).
Phase 2
Phase 2 began with identification of FTs. Announcements describing the program were made at quarterly program directors' meetings. An e-mail was sent to program directors, who designated trainers to represent their disciplines. Participation was voluntary. Phase 2 occurred throughout 7 months, with monthly sessions held at 7 am. Registration and reminders were coordinated by medical education administration. Each 90-minute session included large-group review of materials, followed by discussion of teaching approaches, then opportunities to practice small-group activities. Additional teaching strategies included lectures, roleplay, and review of videotaped scenarios. TTT modules were taught by 2 faculty members (the module designer and a coteacher with expertise in faculty development and workshop implementation).
Phase 3
In phase 3 the FTs assumed responsibility for training. FTs chose which modules to implement, scheduled the resident sessions, and adapted the training materials as needed. This approach allowed for individualization of modules and application of materials to unique teaching environments. Members of the REALL Committee were available for consultation. Phase 3 was completed 12 months after the start of the first TTT module.
Outcome Measures, Evaluation, and Analysis
Evaluation occurred at each phase of the REALL Program (f i g u r e). Evaluation of the TTT individual modules occurred at the completion of each module, with Faculty Survey A (provided as online supplemental material). Additional information was collected 2 months after the completion of the TTT program, with Faculty Survey B (provided as online supplemental material), and 1 year after first session of the TTT session, with Faculty Survey C (provided as online supplemental material). Descriptive statistics were used to portray the participants' views. All surveys provided an opportunity to add comments. Comments were aggregated and common themes identified. Residents also completed a standardized evaluation form after the FT's presentation of a resident training session (provided as online supplemental material).
Results
Implementation of TTT Sessions
A total of 45 FTs from 27 residency programs participated in at least 1 of the 7 TTT series. This represented 45% (27 of 60) of accredited residency programs. Faculty Survey A was distributed to FTs attending the individual sessions (71% response rate). Key findings indicated that most FTs (99%) believed that the 7 modules met learning objectives, that the topics were important for residents (98%), and that as a result of attending the session, they were confident in their ability to implement the REALL module (97%).
Faculty Survey B was distributed at the end of the TTT program to all 45 participants (55% response rate). The purpose of the survey was to elicit the participants' views concerning the usefulness of the individual components of the TTT program. On a 4-point scale, 3 module components (large-group review, annotated PowerPoint slides, and handouts) were found to be very useful (rating range, 3.52–3.73), with annotated PowerPoint slides the most useful. There was wide variance in the rating of small-group breakouts (average rating, 3.1). Some of the FTs appreciated the opportunity to practice activities. Others cited the diversity of residency programs and variety of experiences as reasons that breakout activities were not as useful.
Implementation of Resident Teaching Sessions
Phase 3 of the REALL Program was accomplished by 20 trainers from 11 residency programs who implemented 33 resident sessions, teaching a total of 479 residents (table). Implementation rate by programs was 41% (11 of 27 programs). FTs presented Learning Styles and Giving Feedback most frequently to residents (7 times and 6 times, respectively; table). Resident sessions were integrated into regularly scheduled conferences (noon conference, department grand rounds), whereas some programs held retreats to implement 3 or more sessions at one time.
Table.
Train-the-Trainer Modules
At the conclusion of each resident session, evaluations were collected. Of the 479 participating residents, 385 (80%) completed the evaluation. Overall, residents across disciplines believed they would be able to apply the skills learned in their practice (97% agreed or strongly agreed) and that the specific objectives of resident sessions were met. Analysis of resident comments revealed 2 major themes. Residents appreciated the opportunity for open discussion about their role as teachers, and they valued the use of clear, practical examples and cases. When used, the residents supported the use of video triggers as a method to stimulate discussion.
Faculty Survey C was distributed to FTs who implemented at least one session (N = 20; 60% response rate). The purpose was to elicit feedback on the process of implementing resident sessions. Key findings indicated that trainers had little trouble implementing the sessions (100% agreed or strongly agreed), that the annotated PowerPoint slides were the most valuable component in the module binder (ranked first by 83%), and that FTs modified core curriculum most frequently by changing the case scenarios and examples to fit their specialty. The most common problem noted was lack of time to teach sessions because of competing clinical duties.
Discussion
Our results show that an internal TTT program is an effective method for disseminating resident educator curricula to faculty and residents at a large tertiary care academic institution. FTs' knowledge and confidence were advanced as evidenced as modules' objectives were met, a finding consistent with other TTT programs.12,15 FTs acquired skills during the TTT series, allowing trainers to teach resident sessions with “generic” materials and adapt components to suit their specialty and resident needs. This ability to modify resident educator materials is an important finding, indicating that FTs felt confident in adjusting materials to fit their specialty context. The goal of the REALL curriculum to residents was partially achieved, with an implementation rate of 41% (11 of 27) by participating programs. A total of 10 of the 11 programs that implemented REALL sessions had no prior training for resident as teacher, and 10 programs continue to implement REALL modules (electronic survey). Resident evaluations demonstrated that objectives were met and that residents were confident in their ability to apply new skills, mirroring other studies' results of resident as teacher programs.16–18 Finally, evaluation of our 3-phase model allowed identification of lessons learned for implementation and delivery of a resident educator curriculum program (b o x 2).
BOX 1 Lessons learned for implementation and delivery of resident educator and life-long learner (reall) program.
Strategies Important to the Success of REALL Program
Developing an active, engaged planning committee (phase 1)
Promoting the use of a curriculum development template by module designers (phase 1)
Gaining “buy-in” of graduate medical education council and residency program directors (phase 1)
Offering multiple Train-the-Trainer opportunities to allow faculty trainers time to coordinate with planning of yearly resident conferences (phase 2)
Providing a rich resource of both paper and electronic course material (phase 2)
Providing follow-up and curriculum adaptation assistance (phase 3)
Evaluating rate of implementation to determine benefits, challenges, and success
Our REALL Program had novel aspects. Use of a curriculum development template has not been described in resident as educator teaching methods. Inclusion of lesson plans, annotated speaker notes, outlined agendas, and planning checklists are innovative for providing faculty a comprehensive framework to teach residents. We believe the decision to develop residency-specific faculty to teach key aspects of the resident as educator was an important strategy because the involved faculty were already specialty role models and able to modify modules.
Our study has limitations. The program participation rate in the TTT program was 45%, likely because participation was voluntary. A GMEC review is pending to determine whether participation in the REALL Program should become mandatory. The decision of FTs to present any TTT module was complex. FTs felt the 7 topics were important and were confident in their ability to implement, yet individual implementation rate of the 7 topics varied from 46% to 13% (table). Learning Styles was presented the most, perhaps because it was the first TTT module presented and enthusiasm was high; however, Effective Presentations had the highest rate of implementation. The findings suggest that FTs may have gravitated to the curriculum considered most relevant for residents in their educator role. A faculty survey for FTs who attended the TTT sessions but did not present any resident sessions revealed that common barriers were lack of time and the need to schedule the residency curriculum 1 year in advance. Finally, although widely used in evaluation, self-assessments and satisfaction surveys do not measure changes in behavior. Use of Objective Structured Clinical Examination and teaching evaluations of residents may provide a more accurate reflection of acquired skills.
Use of this model is ongoing in our institution. Program participation in the TTT program has increased to 67% (40 of 60); implementation rate of modules remains at 40% (16 of 40). Changes have been made based on evaluation forms. FTs now have the opportunity to attend a half-day TTT retreat with continuing medical education credits. New videotapes have been produced, demonstrating specific examples for teaching in pediatrics and in the operating room. Costs for these modifications are offset by reduced faculty time needed in designing and developing module materials that remain constant (key objectives, lesson plan, group activities, handouts). New topics for module development focus on the competencies of professionalism and system-based practices. We plan to study benefits to FTs who have participated in the faculty development and to measure the program's effectiveness with use of medical students' evaluations of resident teaching behaviors in selected specialties.
Conclusions
We believe our study adds to the resident as educator literature by outlining an internal dissemination and development model that other large teaching institutions may wish to consider. Our model reached our residents, who needed guidance to learn and improve clinical teaching skills, and simultaneously improved the proficiency of a community of residency faculty educators who are thought to be leaders in our society.
Figure.
REALL Program Design Process and Evaluation Measures
Footnotes
Lily C. Pien, MD, is Faculty Development Associate in the Education Institute, Cleveland Clinic, and Staff Attending in the Respiratory Institute, Cleveland Clinic; Christine A. Taylor, PhD, is Director of Faculty Development in the Education Institute, Cleveland Clinic, and Professor of Medicine at the Cleveland Clinic Lerner Medical College of Case Western Reserve University; Elias Traboulsi, MD, is Head of the Department of Pediatric Ophthalmology and Strabismus, Cole Eye Institute, Chairman of Graduate Medical Education at the Cleveland Clinic, and Professor of Ophthalmology at the Cleveland Clinic Lerner Medical College of Case Western Reserve University; and Craig A. Nielsen, MD, is Director of the Internal Medicine Residency Program, Cleveland Clinic, Vice Chair of Education in the Medicine Institute, Cleveland Clinic, and Assistant Professor of Medicine at the Cleveland Clinic Lerner Medical College of Case Western Reserve University
Funding: The authors report no external funding source.
The authors would like to thank Maggie Muszka for administrative assistance with the REALL Program, and Drs Carol Farver, Ronan Factora, Johanna Goldfarb, and Daniel Shoskes for their assistance in conducting TTT modules.
References
- 1.Brown RS. House staff attitudes toward teaching. J Med Educ. 1970;45(3):156–159. doi: 10.1097/00001888-197003000-00005. [DOI] [PubMed] [Google Scholar]
- 2.Bing-you RG, Sproul MS. Medical students' perceptions of themselves and residents as teachers. Med Teach. 1992;14(2–3):133–138. doi: 10.3109/01421599209079479. [DOI] [PubMed] [Google Scholar]
- 3.Minor S, Poenaru D. The in-house education of clinical clerks in surgery and the role of housestaff. Am J Surg. 2002;184(5):471–475. doi: 10.1016/s0002-9610(02)01001-2. [DOI] [PubMed] [Google Scholar]
- 4.Wang WD, Yang PC, Chen CY, Lue BH, Yang PM. Using senior residents as standardized patients for evaluating basic clinical skills of medical students. J Formos Med Assoc. 2004;103(7):519–525. [PubMed] [Google Scholar]
- 5.Weisgerber M, Flores G, Pomeranz A, Greenbaum L, Hurlbut P, Bragg D. Student competence in fluid and electrolyte management: the impact of various teaching methods. Ambul Pediatr. 2007;7(3):220–225. doi: 10.1016/j.ambp.2007.01.005. [DOI] [PubMed] [Google Scholar]
- 6.Morrison EH, Hollingshead J, Hubbell FA, Hitchcock MA, Rucker L, Prislin MD. Reach out and teach someone: generalist residents' need for teaching skills. Fam Med. 2002;34(6):445–450. [PubMed] [Google Scholar]
- 7.Maker VK, Donnelly MB. Surgical residents peer evaluations—what we learned. J Surg Educ. 2008;65(1):8–16. doi: 10.1016/j.jsurg.2007.10.002. [DOI] [PubMed] [Google Scholar]
- 8.Weiss V, Needlman R. To teach is to learn twice: resident teachers learn more. Arch Pediatr Adolesc Med. 1998;152:190–192. doi: 10.1001/archpedi.152.2.190. [DOI] [PubMed] [Google Scholar]
- 9.Button JH, Bruel BM, Francsico GE. Assessment of musculoskeletal examination skills: physiatry residents as evaluators and models. Am J Phys Med Rehabil. 2007;86(11):926–934. doi: 10.1097/PHM.0b013e318152027b. [DOI] [PubMed] [Google Scholar]
- 10.Morrison EH, Friedland JA, Boker J, et al. Residents-as-teachers training in U.S. residency programs and offices of graduate medical education. Acad Med. 2001;76(suppl 10):S1–S4. doi: 10.1097/00001888-200110001-00002. [DOI] [PubMed] [Google Scholar]
- 11.Donovan A. Radiology residents as teachers: current status of teaching skills training in United States residency programs. Acad Radiol. 2010;17(7):928–933. doi: 10.1016/j.acra.2010.03.008. [DOI] [PubMed] [Google Scholar]
- 12.Skeff KM, Stratos GA, Berman J, Bergen MR. Improving clinical teaching: evaluation of a national dissemination program. Arch Intern Med. 1992;152(6):1156–1161. doi: 10.1001/archinte.152.6.1156. [DOI] [PubMed] [Google Scholar]
- 13.American Academy of Communications in Healthcare. Courses. http://www.aachonline.org/?courses. Accessed August 3, 2011. [Google Scholar]
- 14.Wamsley MA, Julian KA, Wipf JE. A literature review of resident-as-teacher curricula: do teaching courses make a difference. J Gen Intern Med. 2004;19(5 pt 2):574–581. doi: 10.1111/j.1525-1497.2004.30116.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Post RE, Quattlebaum RG, Benich JJ. Residents-as teachers curricula: a critical review. Acad Med. 2009;84(3):374–380. doi: 10.1097/ACM.0b013e3181971ffe. [DOI] [PubMed] [Google Scholar]
- 16.Litzelman DK, Stratos GA, Skeff KM. The effect of a clinical teaching retreat on residents' teaching skills. Acad Med. 1994;69(5):433–434. doi: 10.1097/00001888-199405000-00060. [DOI] [PubMed] [Google Scholar]
- 17.Roberts KB, DeWitt TC, Goldberg RL, Scheiner AP. A program to develop residents as teachers. Arch Pediatr Adolesc Med. 1994;148(4):405–410. doi: 10.1001/archpedi.1994.02170040071012. [DOI] [PubMed] [Google Scholar]
- 18.Edwards JC, Kissling GE, Plauche WC, Marier RL. Evaluation of a teaching skills improvement programme for residents. Med Educ. 1988;22(6):514–517. doi: 10.1111/j.1365-2923.1988.tb00796.x. [DOI] [PubMed] [Google Scholar]