A Literature Review of “Resident-as-Teacher” Curricula

Abstract

OBJECTIVES

To examine the evaluation methods of resident teaching courses and to estimate the effectiveness of these teaching courses.

DESIGN

We searched the literature from 1975 to May 2003 using the PubMed MESH terms internship and residency and teaching; 1,436 articles were identified and 77 contained information regarding teaching courses. Fourteen articles contained information regarding outcomes of resident teaching courses and were selected for intensive review.

MAIN RESULTS

Five uncontrolled pre-post studies used resident self-reported teaching skills/behaviors as outcome measures; all reported some improvement in self-reported skills. Three uncontrolled pre-post studies examined live or videotaped resident teaching encounters and all revealed improvement in some teaching skills. One uncontrolled trial and three nonrandomized controlled trials used learner evaluations of resident teaching behaviors as outcomes and all revealed an improvement in ratings of residents after course participation. Four randomized controlled trials of resident teaching curricula are included in this review. One study did not show any quantitative benefit of a resident teaching course on performance on an objective structured teaching evaluation. Two studies assessing resident teaching evaluations before and after course participation showed conflicting results. One study noted improvements in resident teaching skills assessed through videotape analysis.

CONCLUSIONS

Resident teaching courses improve resident self-assessed teaching behaviors and teaching confidence. Teaching courses are linked to improved student evaluations. Further studies must be completed to elucidate the best format, length, timing, and content of resident teaching courses and to determine whether they have an effect on learner performance.

Studies estimate that residents spend up to 20% of their time on teaching activities—regardless of their future career plans.¹ Residents recognize they have responsibility for teaching medical students, interns, and fellow residents. Surveys demonstrate that residents enjoy teaching and consider it vital to their own education.² Studies also show that residents conduct more teaching at the bedside compared to attendings.³ Additionally, one survey found that medical students estimated one third of their knowledge could be directly attributed to house staff teaching.⁴ Another study found that residents assigned teaching activities increased their knowledge acquisition of assigned teaching subjects.⁵

Despite their significant teaching responsibilities, many residents receive no formal instruction on how to teach effectively. In a 2001 survey, 55% of residency directors indicated their programs offered residents formal instruction in teaching skills.⁶ Although “resident as-teacher” curricula are becoming more common in residency programs, surveyed program directors still express the need for more resident instruction in teaching.⁶

Although there are many published studies describing resident teaching courses, most have not been effectively evaluated. To synthesize the current knowledge regarding resident teaching courses, we reviewed literature published from 1975 through May 2003. We sought to answer the following questions:

• What methods have been used in evaluating the effectiveness of resident teaching courses?

• What is the estimate of the effectiveness of resident teaching courses?

METHODS

We searched the literature using the PubMed MESH terms internship and residency and teaching. The search was restricted to English language studies published between 1975 and May 2003. We identified 1,436 articles. We limited our search to studies published after 1975—when resident teaching courses first appeared.⁷

Two of the authors (MAW and KAJ) evaluated each citation for its relevance to resident teaching courses. One or both of these authors selected 77 articles for further review. Both authors reviewed abstracts of these 77 articles for information about outcomes of resident-as-teacher programs. Studies employing resident self-report of teaching skills or objective criteria (learner evaluations of residents, objective structured teaching evaluations, or videotape review) as outcome measures were included. Articles that were descriptive in nature, reported only resident satisfaction, or involved fellows/faculty were excluded. One or both of the authors (MAW and KAJ) identified 30 articles and read them in their entirety for data on outcomes as described above. The bibliographies of these 30 articles were cross-referenced; 3 additional articles were identified for further review. Fourteen articles met the inclusion criteria described above and are presented in this paper for further analysis. These articles were selected through consensus between both authors (MAW and KAJ) who independently reviewed and abstracted data from each article. Subsequently, the authors compared assessments and came to a consensus regarding the interpretation of the data.

RESULTS

The 14 articles selected for detailed analysis are summarized in Tables 1–3. The studies involved residents from different disciplines and postgraduate years. The teaching programs studied ranged from a 1-hour lecture to a 13-hour teaching course. Eleven studies described the teaching methodology utilized in their curriculum, including large and small group discussion,^7–15 lecture,^{7–9,11,14,16} role play,^{7–9,11,12,14–17} reflection on videotaped vignettes,^{7,10,11,13–15} standardized students,¹⁷ and review of videotaped teaching in a small group setting or with a consultant.^8,10,11,16 Participants in the studies ranged from 6 to 446. Evaluation designs consisted of pre-post cohort studies, post-post cohort studies, nonrandomized control trials, randomized controlled trials, and a crossover study. Where possible, effect size has been calculated from available data.

Table 1.

Summaries of Uncontrolled Studies Evaluating Residents-as-Teacher Curriculum Published 1975 Through May 2003

Author (Year)	Design	Participants (N)	Specialties	Teaching Intervention	Teaching Methods*	Outcome Measures	Results†
Lawson and Harvill7 (1980)	Pre-post	PGY-? (20,‡ 11§)	Family medicine, internal medicine	13 weekly 1-hour sessions. Content: objectives and planning, delivery methods, discussion/questioning, demonstration techniques/AV, and lecturing	Lecture, small group discussion, reflection on videotaped skills, practice with peers	Independent review of videotaped teaching	Mean change in overall teaching score from 2.85 to 3.50 (1 to 4 scale; P < .001, ES 16%). Significant improvements in delivery (P < .001), organization (P < .001), explanation (P < .001), and use of AV (P < .005).
Edwards et al.18 (1986)	Pre-post-post	PGY-1 (61,‡ 18§)	Multiple	1/2-day course		Resident self-ratings on teaching skills inventory	Self-ratings on teaching skills inventory improved after the course (6 and 18 months) in 6 of 8 categories and on the summary score (P < .001, ES 5%).
Edwards et al.19 (1988)	Pre-post	PGY-1 (145)	Multiple	1/2-day course		Resident self-ratings on teaching skills inventory	Self-ratings on teaching skills inventory improved after the course (P < .01).
Bing-You20 (1990)	Pre-post (resident self-report of teaching skills) Post-post (videotape analysis)	PGY-1/2 (26)	Internal medicine	8-hour resident-managed course		Resident self-report of teaching skills and videotape analysis at end of workshop and 2 to 11 months later (average 6.3 months)	Both interns and residents rated themselves as more effective in knowledge after the workshop (P < .05). Interns felt more effective in technical skill and perceived improved teaching organization (P < .05). Videotape analysis revealed a decline in intern and resident organizational skills between post 1 and post 2 (P < .05).
Litzelman et al.11 (1994)	Pre-post	PGY-1 (72)	Internal medicine	Weekend retreat. Content: learning climate, communication of goals, understanding/retention, feedback	Lecture, small group discussion, reflection on videotaped vignettes, role play	Resident self-report of teaching skills and student evaluation of resident teaching	Residents reported improvement in teaching skills after retreat (P < .01). Students’ evaluation of participants improved after the intervention in overall teaching skills, control of session, understanding/retention, and evaluation (P < .05).
Roberts et al.12 (1994)	Pre-post	PGY-1/2/3 (?)	Pediatrics	Two 4-hour sessions and a 1-hour course at 6 months. Content: clinical precepting skills preparation/delivery of a brief presentation	Small group discussion, reflection on videotaped vignettes, role play	Resident self-report on teaching skills inventory	Increased scores on teaching skills inventory. No P values reported. Effect size of changes in aggregate scores for teaching inventory was 2% for PGY-1s, 7% for PGY-2s, and 8% for PGY-3s.
Barth et al.16 (1997)	Crossover study	PGY-4/5 (6)	General surgery	1st intervention: lecture on communication effectiveness with self-review of videotaped teaching session; 2nd intervention: critical review of own videotaped teaching session with a consultant	Lecture, reflection on videotaped skills	Evaluation of videotaped teaching performance	Self-review of videotaped teaching session did not improve communication effectiveness. Review of videotaped teaching session with trained consultant improved communication effectiveness scores (P = .002, ES 11%).
White et al.14 (1997)	Pre-post	PGY-2/3 (21)	Pediatrics	1/2-day course on the OMP	Lecture, small group discussion, role play, reflection on videotaped vignette	Observation of resident-student teaching encounters prior to and after the course.	Resident teaching in all 5 areas assessed showed improvement. No P values reported.

^*Teaching methods described when known.

^†When sufficient data were available, effect size (ES) has been calculated. ES = % change between pre- and post-ratings. Calculated as (mean post-rating − mean pre-rating) × 100/maximum scale rating.

^‡Number of resident participants.

^§Number of participants who completed outcome evaluation measures.

PGY, postgraduate year; OMP, 1-minute preceptor.

Table 3.

Summary of Randomized Controlled Studies Evaluating Residents-as-Teachers Curriculum Published 1975 Through May 2003

Author (Year)	Participants (N)	Specialties	Teaching Intervention	Teaching Methods*	Outcome Measures	Results†
Jewett et al.10 (1982)	PGY-1/2/3 (53)	Pediatrics	Two 1/2-day workshops, two 1-hour follow-up consultation sessions. Content: developing learners’ clinical problem-solving skills, teaching and learning styles, impact of interpersonal behavior	Small group discussion, review of videotaped teaching encounter, reflection on videotaped vignette	Pre-post self-assessment of teaching attitudes; evaluation of clinical teaching skills by faculty, students, and peers on a monthly basis during course (6 months)	Significant improvement in confidence as teacher (P < .05) and in perceived positive feedback on clinical teaching (P < .01). Fifty-two percent of program participants were rated as “effective” teachers compared with 27% of controls. This difference was not statistically significant (no P values reported).
Edwards et al.8 (1988)	PGY-1 (22‡/18§)	Internal medicine, OB-Gyn, family medicine	1/2-day course Content: communicating objectives, motivating, directing attention, teaching procedures, feedback, evaluation	Small group discussion, interactive lecture, role play, review of videotaped teaching encounter	Evaluation of videotaped teaching performance during course and 6 months after course	During instruction, the experimental group had significantly higher ratings on ddx, expanding the case and overall teaching quality compared with controls (P < .017; ES 11%). At 6 months, the only significant difference was in communicating objectives (P < .017).
Dunnington and DaRosa17 (1998)	PGY (?)	General surgery	10.5-hour teaching course. Utilized reminders of course content at 2 and 4 weeks after course. Content: adult learning principles, teaching microskills, bedside teaching, feedback, psychomotor skills, teaching in the clinic/OR	Role play, standardized students	Objective structured teaching evaluation (OSTE) 6 to 7 months after course	Intervention group with few statistically significant differences on the OSTE. These differences were generally limited to 1 or 2 items on an individual OSTE station and were not consistent across the two study sites.
Furney et al.9 (2001)	PGY-1/2/3 (57)	Internal medicine	1-hour session on OMP	Lecture, small group discussion, role play	Resident self-report of use of OMP behaviors, learner ratings of resident performance of OMP behaviors.	Intervention group with statistically significant improvement in learner ratings of OMP behaviors in at least 1 item in all domains except “teach general rules” (P < .05, ES 3%–7%). Intervention group self-reported statistically significant (P < .03) improvement in all OMP behaviors except “teach general rules.”

^*Teaching methods described when known.

^†When sufficient data were available, effect size (ES) has been calculated. ES = % change between pre- and post-ratings. Calculated as (mean post-rating − mean pre-rating) × 100/maximum scale rating.

^‡Number of resident participants.

^§Number of participants who completed outcome evaluation measures.

PGY, postgraduate year; OMP, 1-minute preceptor.

Uncontrolled Studies

The majority of studies were uncontrolled and employed a pre-post design (see Table 1). Five of these studies utilized resident self-report of teaching skills before and after participation in a teaching skills course and all reported statistically significant increases in self-reported skills after course participation.^{11,12,18–20} Litzelman et al. also examined student evaluations of resident participants in the quarter prior to and after attendance at a teaching retreat and found ratings improved in the areas of overall teaching skills, control of session, understanding/retention, and evaluation.¹¹

Four uncontrolled studies employed direct observation of resident teaching performance as an outcome measure. Lawson utilized videotape analysis of a resident teaching session and found statistically significant improvements in videotaped performance after workshop participation.⁷ Bing-You utilized videotape analysis of resident teaching immediately after a teaching workshop (post 1) and an average of 6.3 months (post 2) after the course.²⁰ Although 7 of 11 interns demonstrated higher score in delivery and evaluation on post 2 evaluations compared to post 1, there was no pre-course evaluation. Thus, one cannot conclude that skills were obtained as a result of the course, and skills were more likely obtained as the result of practice. There was a significant decline in organizational skills for both interns and residents; this suggests that some effects of a teaching course are not long lasting; skills may deteriorate if not reinforced. Barth et al. looked at the effect of a lecture on communication and self-review of a videotaped teaching session compared to a 1-hour individual feedback session with a teaching consultant.¹⁶ Communication skills of residents improved significantly after individual feedback, but not after lecture and videotape self-review. This study suggests that individualized feedback on teaching may enhance the effectiveness of teaching programs and improve resident communication skills.

Nonrandomized Controlled Studies

Three nonrandomized controlled studies are summarized in Table 2. All examined learner ratings of workshop participants compared to controls and found resident participants had significantly higher ratings on learner evaluations in some areas. In the largest of these studies, Wipf et al. looked at all intern and student evaluations (3,946) of 446 residents in the 3 years prior to and after implementation of a 6-hour teaching course.¹⁵ Every subsequent year after course implementation, there was a significant improvement in the mean score for each item on the teaching evaluation form. Mean ratings on 9 items before the course ranged from 5.05 to 5.45 and improved to 5.4 to 5.7 on a 6-point scale. Although mean ratings changed from “very good” to “excellent” for all but one item, the effect size was small (2% to 9% change).

Table 2.

Summaries of Nonrandomized Controlled Studies Evaluating Residents-as-Teacher Curriculum Published 1975 Through May 2003

Author (Year)	Study Design	Participants (N)	Specialties	Teaching Intervention	Teaching Methods*	Outcome Measures	Results†
Edwards et al.19 (1988)	Nonrandomized control	PGY-1 (145)	Multiple	1/2-day course		Student evaluation of resident teaching skills	Students rated workshop participants more highly on knowledge, organization, demonstration of skills, and overall teaching effectiveness than control group (P < .05; ES 11%).
Spickard et al.13 (1996)	Nonrandomized control	PGY-2/3 (44)	Internal medicine	3-hour skills workshop. Content: learning climate and feedback	Small group discussion, reflection on videotaped vignettes	Resident self-report of attitudes toward teaching and teaching skills; student evaluation of resident teaching skills	Participants reported increased confidence in giving feedback (P = .01) and overall teaching skills (P = .001) compared to controls. Self-reported teaching skills in the areas of learning climate (P = .002) and feedback (P = .01) improved in participants, but not controls. Students rated participants more highly on feedback (P = .03; ES 9%) and learning climate (P = .007; ES 7%) after the course.
Wipf et al.15 (1999)	Pre-post study utilizing historical controls	PGY-2/3 (446)	Internal medicine	6-hour course. Content: managing work rounds, practice skills, feedback/evaluation, student teaching	Large group discussion, small group discussion, reflection on videotaped vignettes, role play	Intern and student evaluations	Learner evaluations of residents were higher in the 3 years after course implementation compared with scores prior to course (P < .001; ES 2% to 9% for each evaluation item).

^*Teaching methods described when known.

^†When sufficient data were available, effect size (ES) has been calculated. ES = % change between pre- and post-ratings. Calculated as (mean post-rating − mean pre-rating) × 100/maximum scale rating.

Spickard et al. also examined teaching evaluations for workshop participants and found improvement in the areas of feedback and learning climate.¹³ On a 9-point scale, mean ratings increased from 6.51 to 7.29 (9% change) and 6.67 to 7.34 (7% change) for feedback and learning climate, respectively. These data suggest that resident-as-teacher programs improve resident teaching evaluations and increase resident confidence in their teaching abilities.

Randomized Controlled Trials

Four randomized controlled trials are summarized in Table 3. Edwards et al. demonstrated that during instruction, participating residents had improvement in overall mean teaching score from 3.86 to 4.49 per 6-point scale (11% change) on a videotape analysis of their teaching skills.⁸ These improvements showed attenuation with time; at a 6-month evaluation, the only difference between experimental and control groups was in communicating objectives. Jewett et al. found that residents completing a teaching course reported increased confidence in their teaching skills and more positive feedback from others compared to controls.¹⁰ Although there were differences in faculty, peer, and student ratings of participants’ clinical teaching skills during the 6-month course (52% of experimental group were rated “effective” compared with 27% of the control group), these differences were not statistically significant. Furney et al. looked at the impact of a 1-hour curriculum on the 1-minute preceptor (OMP).⁹ Both student evaluations of residents and resident self-evaluation of OMP skills showed significant improvement 2 weeks after the intervention (3% to 7% change). Finally, Dunnington and DaRosa examined scores on an objective structured teaching evaluation (OSTE) performed 6 months after a teaching course for surgical residents and found minimal differences between intervention and control groups.¹⁷ In this study, there was no OSTE performed immediately after the teaching course. From these 4 studies, it appears that there may be an effect of resident teaching courses on specific teaching skills, but it is questionable whether this effect is long lasting.

DISCUSSION

This review underscores the fact that the current literature regarding resident teaching courses is limited and constrained by studies with diverse experimental designs and outcome measures. The teaching methodology for the interventions varied widely. Many studies used resident self-assessment of teaching skills as an outcome measure. Although resident self-confidence is important, self-assessment may not represent actual skills obtained. As residents advance in their training, they gain teaching experience and may report increased comfort with teaching. Thus, uncontrolled studies that utilize self-assessed teaching skills will be confounded by additional resident teaching experience. Most of the studies had a small number of participants, making it difficult to obtain adequate power for effective analysis.

There is evidence that resident teaching courses improve learner evaluations of participants. The three nonrandomized controlled studies that measured student ratings demonstrated statistically significant improvement in resident evaluations after interventions varying from 3 to 6 hours. However, when impact is assessed by effect size, the percent change in evaluations after the intervention is small (2% to 11%), in part due to the concentration of evaluation scores in the upper half of the scale. Yet in the largest controlled study, these differences resulted in residents being perceived as “excellent” rather than “very good” teachers.¹⁵ The 4 randomized controlled studies varied in teaching intervention, content, and outcome measures; 3 showed improvement in at least 1 teaching outcome.^8–10 In the negative study that showed similar OSTE performance between course participants and controls at 6 months, there was no OSTE obtained at baseline or immediately after the course.¹⁷ Thus, it is unclear whether skills were not acquired or if they were acquired and diminished over time. There are data from 4 small studies examining the lasting effects of resident teaching courses^8,12,18,20; 3 suggested a decline in teaching skills over time.^8,12,20

This review is the first to systematically summarize the current literature regarding resident-as-teacher curricula. This review has several limitations. First, we recognize the potential effect of publication bias, in that positive studies are more likely to be published, and we reviewed only published studies. Additionally, only studies with “quantitative” results were selected for review. Although we attempted to comprehensively review the literature, our search included one large database in the English language and bibliography cross-reference; studies may have been missed. Despite potential limitations, our review does have its strengths. We synthesized data from a multitude of studies with a variety of outcomes. In doing so, we are able to identify the value of a resident teaching course and define important areas for future research.

Residents are vital to teaching in academic centers. There is evidence that teaching courses improve resident self-assessed teaching behaviors, self-confidence as a teacher, and result in higher learner evaluations of residents. Given these demonstrated positive effects, one might argue that all residency programs should require resident teaching instruction. However, many obstacles to course implementation exist. Teaching courses can be time intensive for residents and faculty, and it is challenging to provide residents with uninterrupted educational time. Many academic faculty are not given supported time to create and teach these courses. Competing curricular requirements for training may result in fewer time-intensive curricula. All of these barriers may contribute to the fact that nearly half of all residency programs do not provide formal instruction in teaching.⁶

Some questions remain regarding resident-as-teacher courses. First, it is not clear what parts of the teaching courses are the most effective; only 1 study evaluated differences between 2 teaching interventions.¹⁶ Additionally, an initial objective to compare course length, format, and timing could not be achieved due to the widely differing study designs and insufficient detail about course content. Future research should be completed to elucidate the best structure of resident teaching courses, as well as optimal level of trainee and teaching methods. Research also is needed to assess the lasting effect of resident teaching courses and the impact of brief follow-up sessions to reinforce skills. Randomized studies with immediate post assessment as well as 6-month to 1-year post assessments with reinforcement curricula would be helpful. Future studies should examine objective measures such as students’ acquisition of skills as a result of resident teaching. Although formal study may be challenging, we faculty who conduct resident-as-teacher courses have found intangible benefits such as enhanced intern and resident morale and increased program emphasis on teaching excellence.