Abstract
Background
We conducted a national survey of US pediatric program directors to explore the current status, content, and teaching methods of Resident-as-Teacher (RAT) curricula. The purposes of the survey were to (1) determine the level and method of evaluation of such curricula, and (2) assess the need for a national curricular resource in this area.
Methods
A survey was sent to US pediatric program directors that asked questions regarding demographics, support, design, development, content, and evaluation of RAT curricula, as well as existing needs and desires for RAT curricular resources.
Results
Sixty-two percent of pediatric program directors completed our survey. Eighty-seven percent have a formal RAT curriculum, but more than 50% allocate 10 hours or less to it during residency. The primary teaching modalities are lectures and workshops. Content areas include feedback, in-patient teaching, communication skills, case-based teaching, role modeling, evaluation, leadership skills, 1-minute preceptors, teaching/learning styles, professionalism, and small-group teaching. Sixty-three percent of programs report evaluating their curricula, but only 27% perceive their program to be very/extremely effective. Nearly all respondents expressed interest in a national RAT curriculum, preferring web-based modules for dissemination.
Conclusions
Despite an Accreditation Council for Graduate Medical Education requirement for a RAT curriculum, some pediatrics programs still lack one, and some consider their program only moderately effective. A wealth of curricular material exists across programs, which could be shared nationally. Establishing a national RAT curriculum would offer programs resources to meet educational mandates and the ability to tailor programs to best fit their own program needs.
Background
Residents are a vital part of the teaching force in training programs and medical schools. The Accreditation Council of Graduate Medical Education (ACGME) mandates that programs provide resources for residents to develop skills for educating patients, families, peers, students, and other health care professionals in various settings and to use multiple modalities.1 The Liaison Committee on Medical Education (LCME) objectives require residents to be trained in, and evaluated on, their teaching skills.2
It has been estimated that residents spend up to 20% of their time teaching.3 Recognizing the importance of this teaching role, Resident-as-Teacher (RAT) curricula are increasingly being incorporated into residency programs. In a 2000 study,4 more than 50% of residency programs in the United States had some type of RAT curriculum. Some studies5 have also shown an increase in residents' confidence in their teaching skills and in student evaluations of their teaching after the residents completed a RAT program.
Despite the increasing implementation of RAT programs, little is known about the format, content, and timing of existing programs or about the elements of successful programs, program support, or outcome measures used.5 To address these issues, we developed a survey to (1) better define the structure and existing content of RAT curricula, and (2) assess the need for a national RAT curricular resource for pediatric programs. We explored the current status, content, and teaching methodologies used in RAT programs, as well as program support and educator expertise required to oversee these curricula, barriers to sustaining these curricula, and criteria for evaluating these curricula. Lastly, we sought to aggregate our results into a needs assessment to explore the need for a national pediatric RAT curricular resource.
Methods
Our study group designed a voluntary and confidential web-based survey, and in the spring of 2009, e-mailed it to program directors of US pediatric residency programs.
The survey was designed using the ACGME and LCME objectives as a framework, supported by the current RAT literature. The survey went through a collaborative consensus-building process and an iterative review of the survey by our study group and other subject matter experts, as well as pilot-testing by a group of pediatric program directors.
The survey consisted of 11 to 40 (depending on early answers) multiple choice and free-text questions seeking information about program demographics, content, delivery methods, cost, and assessments of RAT programs. Our survey program allowed us to identify those who had not completed the survey without identifying the answers of those who had. We sent up to 3 reminder surveys to nonresponders.
The Institutional Review Board at the University of Chicago reviewed the study and declared it exempt.
Results
Demographics
A total of 172 program directors were surveyed, 113 (66%) responded, and 106 (62%) completed the survey. Ninety-eight of the 113 programs responding (87%) indicated they had an existing RAT curriculum. Programs reported having 14 to 128 (mean, 44; median, 38) residents, similar to the average across all US programs (41.4).6 Ninety-three respondents (82%) self-identified as university-based programs (having a primary affiliation with a US medical school), compared with 92% of the programs listed on FRIEDA Online6 (American Medical Association, Chicago, IL).
RAT Curricula Support
Eighty percent of residency programs (77 of 96) with RAT curricula have faculty designated to develop the curriculum. figure 1 illustrates the spectrum of faculty-designee roles. Programs indicated that they have continuity clinic leaders, chief residents, vice chairs for education, internal medicine faculty, an associate dean, and a chairman designated to design, develop, and implement the curriculum. Twenty-two percent of the 77 programs (n = 17) with faculty members designated to work on the RAT curriculum indicated that those faculty members have protected time or salary support for curriculum development. Nineteen percent of the programs (15 of 73) indicated that their RAT curricula were at least partially funded institutionally, with none being externally funded. Components of that support included faculty time, food, curricular materials, and technical support. Overall, 67% of program respondents (61 of 90) reported being moderately to extremely supported by their institution's leadership, whereas 6% (5/90) felt their program was not supported at all.
FIGURE 1.
Current Position of Resident-as-Teacher (RAT) Leadership in Residency Program
RAT Faculty
Approximately 24% of faculty members (17 of 72) involved in RAT curricula hold advanced degrees in education (Masters, PhD, or EdD), and 29% of faculty (21 of 72) have participated in a formal teaching program without a degree (eg, Stanford, Harvard Macy). Six percent (4 of 72) indicated that their faculty have participated in an accredited academic pediatric fellowship. Seventy-six percent (55 of 72) have attended educational and/or leadership workshops. Several programs have hired consultants/outside resources to assist with RAT curricula: 5% for assessment (4 of 75), 4% for curriculum design (3 of 75), 7% for curriculum development (5 of 75), 20% for teaching (15 of 75), and 7% for curricular evaluation (5 of 75).
Needs Assessment for RAT Development
figure 2 illustrates how programs developed their curricula. Two programs indicated using the American Academy of Pediatrics RAT program7 or collaborating in joint RAT curricula with another program.
FIGURE 2.
Method of Curricular Development for Resident-as-Teacher (RAT)
RAT Curriculum Implementation Practices
For 80% of responding programs, RAT curriculum participation is required, whereas for 2%, participation is optional. Seventeen percent offer curricula with optional components to complement the requirements. Most programs offer their RAT curricula to trainees in all 3 years of training. Programs use a variety of delivery methods for their RAT curricula (figure 3). figure 4 indicates the amount of time allotted per trainee for RAT curriculum. Eighty percent of the programs report that the time allotted is protected educational time.
FIGURE 3.
Formats Used for Delivering Resident-as-Teacher (RAT) Curriculum
FIGURE 4.
Total Time Dedicated to RAT Curriculum (for 3 Years)
RAT Curriculum Content
Eighty-nine percent of RAT curricula include skills practice: 65% use role play, 64% use direct observation, 12% use objective, standardized learner experiences. Common components of RAT curricula include skills development, teaching methods, and curriculum design. The table lists specific topics within each common component.
TABLE.
Existing Content of Resident-as-Teacher (RAT) Curricula (% of Respondents)
TABLE.
Continued
RAT Curriculum Evaluation
Sixty-three percent of respondents indicated that their programs evaluate their RAT curricula. figure 5 illustrates the means of evaluation. Components of those evaluations include participant satisfaction, evaluation of resident teaching and leadership proficiency, and resident attitudes toward teaching. Programs also assess efficacy with student, peer, and faculty evaluations: 27% of the programs consider their curricula to be very effective/extremely effective; 53%, moderately effective; and 18%, slightly effective/not effective; and 62% of programs report modifying their curricula based on their evaluation results.
FIGURE 5.
Evaluation Tools for Resident-as-Teacher (RAT) Curriculum
Respondents noted various reasons for curricula effectiveness. Those explanations included stratifying teaching objectives by level of training; residents being able to practice skills; prioritizing the RAT curriculum; rewarding individual/collective proficiency; being learner-centered, evolving curricula; demonstrating relevant techniques for immediate application; providing effective feedback; and reinforcing the RAT topics in various venues. Programs also indicated they see the effectiveness demonstrated by improvements in resident teaching skills; improvement in tests of resident attitudes toward teaching; results of peer evaluations and self-evaluations; and in observation of resident teaching. Popularity of the teaching rotations was also cited as an indicator of RAT curriculum effectiveness.
Possible reasons for ineffectiveness included clinical duty distractions, time constraints, competing clinical priorities, resident lack of motivation, lack of regular reinforcement, lack of faculty mentorship, lack of consistent curricula, difficulty evaluating teaching skills, and lack of support from faculty or chair.
Programs Without RAT Curriculum
Programs have indicated various reasons for not having a RAT curriculum including lack of resources, knowledge, and expertise; lack of resident buy-in; preference for informal, direct faculty feedback; and the program having lower priority compared with other curricular requirements.
National Core RAT Curriculum Options
Programs indicated that the following would be most helpful in creating/building their RAT curricula: resource website (80%), train-the-trainer workshops (60%), and electronic handbooks (56%). Other needs noted were additional time, faculty participation, and buy-in.
Ninety-five percent of programs expressed some level of interest in a national core RAT resource to aid programs with RAT curricula. Respondents would prefer such a resource to be optional, simple, and not time consuming and to include guidance/training for implementation. Those not in favor indicated there are already too many residency mandates, and smaller programs may struggle with implementation.
Most respondents (88%) prefer web-based delivery of national RAT resources. Others recommend national workshops (64%), expert site visits (23%), or accessing a paper manual or handbook (36%).
Discussion
In 2000, 80% of pediatric programs in the United States had a RAT curriculum.4 Our 2009 survey showed that number still is not 100%, despite accreditation requirements. Of those curricula that do exist, their effectiveness, their outcomes, and how they are perceived have not been well documented. Our group specifically did not assess programs outside of the United States to narrow the scope of the needs assessment.
Leadership and support are keys to the success of any curricular program. Most responding programs have designated faculty members responsible for the curriculum, but we did not assess whether that additional responsibility placed an undue burden on faculty. Most curricula are led by physicians with some workshop training but not necessarily advanced educational training/degrees. Even fewer programs have medical education specialists (nonphysician) involved in the RAT leadership. This lack of access to educational training or specialists can disadvantage programs with fewer resources. Our study demonstrates a lack of funding in most programs. Despite this, most programs are moderately/extremely satisfied with the support they receive. Although this suggests the RAT leadership can be satisfied with nonfinancial support, it cannot be interpreted to mean funding is not important.
The process of creating and delivering RAT curricula is highly variable. Most programs have developed their curricula based on site-specific issues, such as resident needs, faculty opinions, and considerations of medical educators. Although some programs reported using models from the literature, most programs did not incorporate experiences of others. Not applying principles from programs that work is counter to what Glassick8 advocates as critical to evaluating scholarship. There are well-described models that can provide a foundation for a RAT curriculum and that warrant consideration by those planning such programs.9,10 Although programs might not be interested in publishing results, evidence of curricular efficacy can demonstrate a positive return on investment to participants, leadership, and other programs.
The data suggest that there is a core set of topics that form the foundation of most curricula. There are also additional topics that are rarely/not reported, which might be highly beneficial to programs (eg, simulations, goals and objectives, time management, self-assessment, and leadership and change). Not including such valuable topics could indicate a lack of faculty comfort and experience in the topic areas, time constraints, or lower prioritization. Based on the variability among programs, residents in different programs might not experience training in a number of important educational topics, perhaps pointing to the need for national resources that would include value-added topics.
Most programs rely on lectures for curriculum delivery. We did not assess the features or methods used during lectures. If faculty deliver lectures in a way in which learners are passive, that would not be ideal for maximum learning. In fact, using only didactic techniques to instruct on effective instructional techniques does not provide a role model for ideal methods. Respondents also reported that workshops are frequently used, but there was no assessment to determine how effective those were in residents' learning. Methods such as online information and training or networks and discussion groups are rarely cited in carrying out these curricula, excluding many potentially active and valuable methods for instruction.
Our results indicate limited curricular evaluation, demonstrating a lack of the scholarship rigor that Glassick8 promoted. Resident satisfaction with RAT curriculum and increased confidence in teaching postintervention represent important aspects of evaluation but only within the context of measuring the teaching performance and learner satisfaction. This confirms findings that most RAT curriculum evaluations met the first and second levels of Kirkpatrick's educational outcomes framework and do not extend to an assessment of behavior (level 3) or outcomes (level 4).11,12 The call for outcomes-based evaluations and measures by the ACGME underscores the importance of improving and increasing the evaluation of curricular outcomes in addition to including longitudinal and multicentered evaluations of teaching skills and effectiveness.
Some programs still do not have a RAT curriculum, which is concerning, given the mandate to residents to become effective teachers. It is possible that the development of a national RAT curricular resource would be of benefit to these programs.
Finally, nearly all respondents expressed interest in development of, and access to, a national RAT curricular resource as long as it was not mandated and was easy to access. Programs are also interested in introducing/improving their RAT curricula contingent on it not resulting in the addition of another layer of work or requirements on an overburdened faculty. One could envision a train-the-trainer program online in areas of feedback, problem learner, orienting a learner, and skills teaching.
Conclusion
Residents play a major role in educating peers and more junior learners. During the past decade, the prevalence of some form of RAT curricula in pediatric residency programs seems to have remained high, providing educational training to residents to assist them in teaching and training our future physicians. Training residents as educators is now a recommendation, rather than an option, of the accrediting councils for medical education.
Despite advances, the development and facilitation of teaching curricula for residents continue to be teacher-centered and program-specific, and some programs report that they do not evaluate their curricula. Those that do evaluate their programs often rely on self-reported satisfaction data rather than on outcome measures that could assess whether the program succeeds in improving resident teaching skills.
Our findings suggest that online, extensive, and easily accessible, national RAT curricular resources would be useful for, and welcomed by, pediatric programs as they seek to improve and expand their curricula. Such resources could be presented as a warehouse of topic-related training modules that target different levels and types of residency training and that accommodate different educational venues and timeframes. In addition to curricula, there may be a need to offer accompanying resident assessment and outcomes-based curriculum evaluation tools. Such resources could also serve as a platform for increased networking and collaboration to exchange ideas, experiences, and information. Considering current restrictions on resident duty hours, faculty who are busy in a revenue-driven environment, and an ever-expanding and demanding curriculum, the development of national curricular resources for RAT programs could only be valued-added in this context.
Footnotes
H. Barrett Fromme, MD, MHPE, is Assistant Professor in the Department of Pediatrics at the University of Chicago; Shari A. Whicker, MEd, is Education Administrator for the Office of Pediatric Education at Duke University Medical Center; Steve Paik, MD, EdM, is Assistant Professor in the Department of Pediatrics at New York University School of Medicine; Lyuba Konopasek, MD, is Associate Professor in the Department of Pediatrics at Weill Cornell Medical College; Jennifer L. Koestler, MD, is Associate Professor of Pediatrics and Medicine and Associate Dean for Medical Education at New York Medical College; Beverly Wood, MD, MSEd, PhD, is Professor of Radiology at Loma Linda Medical Center and Professor Emerita of Radiology and Pediatrics at the University of Southern California, Keck School of Medicine; and Larrie Greenberg, MD, is Clinical Professor of Pediatrics and Internal Consultant at George Washington University School of Medicine.
Funding: The authors report no external funding source.
References
- 1.Accreditation Council for Graduate Medical Education. 2007 program requirements for graduate medical education in pediatrics. Available at: http://www.acgme.org/acWebsite/downloads/RRC_progReq/320_pediatrics_07012007.pdf. Accessed September 1, 2010. [DOI] [PubMed] [Google Scholar]
- 2.Liaison Committee on Medical Education. 2008 function and structure of a medical school: standards for accreditation of medical education programs leading to the MD degree. Available at: http://www.lcme.org/functions2008jun.pdf. Accessed September 1, 2010. [Google Scholar]
- 3.Greenberg LW, Goldberg RM, Jewett LS. Teaching in the clinical setting: factors influencing residents' perceptions, confidence and behaviour. Med Educ. 1984;18(5):360–365. doi: 10.1111/j.1365-2923.1984.tb01283.x. [DOI] [PubMed] [Google Scholar]
- 4.Morrison EH, Friedland JA, Boker J, Rucker L, Hollingshead J, Murata P. Residents-as-teachers training in U.S. residency programs and offices of graduate medical education. Acad Med. 2001;76(10 suppl):S1–S4. doi: 10.1097/00001888-200110001-00002. [DOI] [PubMed] [Google Scholar]
- 5.Wamsley MA, Julian KA, Wipf JE. A literature review of “resident-as-teacher” curricula: do teaching courses make a difference. J Gen Intern Med. 2004;19(5, pt 2):574–581. doi: 10.1111/j.1525-1497.2004.30116.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.American Medical Association. FRIEDA online specialty training search: pediatrics. Available at: https://freida.ama-assn.org/Freida/user/specStatisticsSearch.do?method=viewDetail&spcCd=320&pageNumber=2. Accessed December 27, 2010. [Google Scholar]
- 7.Johnson CE, Bachur R, Priebe C, Barnes-Ruth A, Lovejoy FH, Hafler JP. Developing residents as teachers: process and content. Pediatrics. 1996;97(5, pt 1):907–916. [PubMed] [Google Scholar]
- 8.Glassick CE. Boyer's expanded definitions of scholarship, the standards for assessing scholarship, and the elusiveness of the scholarship of teaching. Acad Med. 2000;75(9):877–880. doi: 10.1097/00001888-200009000-00007. [DOI] [PubMed] [Google Scholar]
- 9.Gaba ND, Blatt B, Macri CJ, Greenberg L. Improving teaching skills in obstetrics and gynecology residents: evaluation of a residents-as-teachers program. Am J Obstet Gynecol. 2007;196(1):87e1–87e7. doi: 10.1016/j.ajog.2006.08.037. [DOI] [PubMed] [Google Scholar]
- 10.Morrison EH, Rucker L, Boker J, et al. A pilot randomized, controlled trial of longitudinal resident-as-teachers curriculum. Acad Med. 2003;78(7):722–729. doi: 10.1097/00001888-200307000-00016. [DOI] [PubMed] [Google Scholar]
- 11.Hill AG, Yu TC, Barrow M, Hattie J. A systematic review of resident-as-teacher programmes. Med Educ. 2009;43(12):1129–1140. doi: 10.1111/j.1365-2923.2009.03523.x. [DOI] [PubMed] [Google Scholar]
- 12.Kirkpatrick DL. Evaluating Training Programs: The Four Levels. San Francisco, CA: Berrett-Koehler Publishers; 1994. [Google Scholar]