Feedback Based on Observation of Work Rounds Improves Residents' Self-Reported Teaching Skills

Abstract

Introduction

Residents provide a significant amount of bedside teaching to medical students and more junior colleagues, but often do not receive feedback that is tailored to this aspect of their professional performance.

Objective

To assess residents' self-reported improvement in teaching skills after feedback based on direct observation of work rounds.

Method

The authors initiated a program of direct observation of residents' teaching during work rounds during the academic year 2007–2008. Eleven interested faculty volunteers, including chief residents, observed teaching on work rounds by 18 second-year residents in internal medicine during 35 total encounters. Within 24 hours, the faculty observers provided individualized feedback to the resident teachers regarding the quantity and quality of their teaching based on the data collected with the Teaching on Work Rounds observation form. At the end of the year, a survey was conducted to assess the residents' receptivity to this program.

Results

Each observation averaged 92 minutes per observer, for 81.5 recorded hours of observations. Eighty percent of the residents felt that they were better teachers because of the feedback they received, and 87% subsequently reported having made conscious changes in their teaching during work rounds.

Discussion

A direct observation program of residents' teaching on work rounds improved residents' interest in teaching while motivating them to make conscious changes in their teaching based on the individualized feedback they received.

Introduction

Clerkship students receive a significant proportion of their total teaching during resident-led work rounds.^1,2 In addition, residents feel that teaching is important to their own education.^3,4 These data suggest that teaching residents to teach, especially during work rounds, may improve the education of undergraduate and graduate trainees.

To our knowledge, 7 studies to date have explored direct observation of resident teaching during work rounds,^5–11 with 2 studies (29%) assessing how residents spent their time during work rounds.^5,6 Norgaard et al⁷ demonstrated the validity of a checklist of tasks that should be completed during work rounds. Tremonti and Biddle⁸ compared work rounds and attending rounds, finding that the location and teaching behaviors differed. Ashton et al⁹ showed that a resident's management style on work rounds can affect patient outcomes, with a data-gathering style being most efficacious. Arseneau¹⁰ examined the question-asking behaviors of residents during work rounds, and an observation study by Wilkerson et al¹¹ evaluated residents on their organization, leadership, and enthusiasm for teaching. This showed that residents asked an average of 1.3 questions, with most clarifying patient data or confirming completion of previously assigned tasks. Only 22% of resident responses to questions encouraged learners to problem-solve.¹¹ None of these studies used observations to provide feedback with the aim of improving residents' teaching performance.

We evaluated resident teaching on work rounds because these rounds are the most important daily venue for resident teaching at our institution. Attendings are not present, which allows for maximal resident autonomy. Direct observation of teaching during work rounds enables an evaluation of what residents actually do in the challenging hospital environment where they practice.¹² Context specificity is important because an individual's performance on a particular problem or in a particular situation is only weakly predictive of the same individual's performance on a different problem or in a different situation,¹³ suggesting that feedback given during work rounds would have the greatest effect on subsequent teaching performance.

We hypothesized that giving specific, formative feedback based on direct observation of work rounds would be feasible to accomplish with minimal interruption to the daily work flow, be appreciated by residents, and lead to self-assessed changes in teaching skills and behavior.

Methods

Setting

Our institution is an urban, academic, tertiary-care hospital with 72 internal medicine residents. Teams are divided into subspecialty services, managed by an attending physician trained in the corresponding discipline, and are composed of 1 resident, 1 or 2 interns, a subintern, 1 or 2 third-year medical students, and occasionally, a pharmacist. Second-year (postgraduate year [PGY]-2) residents run most of the ward services and are the subjects for our observation. Rounds start at 7:00 am and typically end by 9:00 am, so that team members can attend morning report.

Instrument

To enhance the specificity of formative feedback regarding resident teaching techniques, the authors created the Teaching on Work Rounds (ToWR) observation form.¹⁴ It assessed 6 teaching dimensions (organization, learning climate, teaching techniques, leadership of work rounds, providing effective feedback, and fostering life-long learning), which are used in the assessment of clinical teaching in the validated Stanford Faculty Development Program No. 26.^15,16 Drawing on the literature, expert opinion, and the authors' personal teaching experience, we created 31 questions that characterized good clinical teaching specific to work rounds. The authors reviewed a video reenactment of resident teaching using this form and discussed responses explicitly to standardize future observations. Before making the first observation, each observer met with 1 of the authors for 15 minutes to review this form. Observers remained silent and unobtrusive during the observation, and residents were not allowed to see the forms before or after the intervention.

Data Collection and Analysis

Eleven faculty volunteers were selected as observers. Observers were chosen on the basis of their known interest and experience in clinical teaching. Throughout the 2007–2008 academic year, 18 of 24 PGY-2 residents (75%) were observed twice teaching during work rounds, followed each time by feedback based on first-hand data that had been collected.

We did not initially seek Institutional Review Board approval because the study was originally conceived as an educational initiative only, not as a research project. Approval for the initial period was not received. Approval was sought and granted for subsequent years' research, and the committee expressly commented that we would have been granted an exception for the first year, if it had been sought prospectively.

Feedback

Within 24 hours after each observation, faculty observers met with the resident for 10 to 30 minutes to offer verbal feedback with a written summary for later reference. Observers were instructed to model their feedback on previously described recommendations,^16,17 with the specific reinforcing and corrective feedback based on behaviors directly observed during the observation period.

Survey

At the end of the academic year, all PGY-2 participants were asked to evaluate the observation experience (table). The 7 questions for this were developed by the authors and were not pilot-tested. Residents had the option to include their names and were assured their survey would remain confidential.

TABLE.

Postobservation Self-Reflection on Being Observed Teaching During Work Rounds (WR)

Results

Each observation averaged 92 minutes per observer, for 81.5 recorded hours of observations. Fifteen of the 18 PGY-2 residents (83%) completed the postobservation survey at the end of the academic year. Resident responses were most positive about the value of the observation and feedback on their interest in teaching during work rounds and on whether they made changes in their teaching during work rounds, with 13 of 15 residents (87%) agreeing or strongly agreeing to these questions; residents were also positive on the value of the feedback in improving their teaching, with 12 of 15 residents (80%) agreeing or strongly agreeing to that question. Resident responses were split on whether they were self-conscious about being observed and on whether the observation affected their behavior. One resident reported that “overall, it's a good exercise whether or not the feedback helps because it forces you to focus on teaching. Without the observation sessions, I probably would have lost that focus over time, so it serves as a reminder that we need to at least try to teach a little every day.” All results for the follow-up survey are shown in the table.

Discussion

Most of our observed residents believed that they had changed their teaching behaviors and had become better teachers after they were observed and had received feedback, suggesting this program was perceived by the participants as beneficial. Of the 6 respondents (40%) who answered “Undecided” or “Disagreed” to the first survey statement (“I liked being observed on work rounds”), all were self-conscious, one-half felt they were not their “normal self,” and 5 (5/6, 83%; 5/15, 33%) disagreed that “every resident should be observed.” Yet, 4 of these 6 respondents (67%) still found that the process stimulated their interest, 5 (83%) made conscious changes in their teaching, and 4 (67%) felt they were better teachers.

This study has several limitations. Its small sample size and location in an urban, tertiary, academic medical center with a moderate-sized internal medicine residency program limit generalizability. Also, because our intervention included observation and feedback, it is unclear which one had the larger effect on residents' self-assessed improvements. An observer's presence could not be hidden, so it is possible observation alone, and not the specific feedback, led to our observed outcomes. However, although residents knew the purpose of the study was to assess clinical teaching on work rounds, the observation day was not announced, and residents could not prepare ahead of time. Our survey instrument is not validated, and it is possible that residents interpreted the questions differently. Finally, because teaching is considered important in internal medicine and the observers were respected teachers, social desirability bias may have affected the responses.

Several questions emerge from this study and provide a starting point for future research. Studies should explore whether direct observation and feedback of resident teaching during work rounds lead to improvements in clinical teaching as perceived by medical students and more junior residents. Do residents' higher performance on teaching assessments correlate with better learning by medical students and junior residents, as assessed by clerkship grade, in-training, or Shelf Exam scores (National Board of Medical Examiners, Philadelphia, PA)? What is the minimal duration of observation that would allow for accurate, complete assessment of teaching skills, as well as for meaningful feedback? How long do these benefits last, and is there a need for recurrent observations for reinforcement? Finally, can improving residents' teaching skills lead to better clinical skills and ultimately improved patient outcomes?

Conclusions

We demonstrated that feedback based on direct observation of resident teaching during work rounds is feasible, is accepted and even appreciated by residents, and changes self-reported teaching behaviors. Given the recent Accreditation Council for Graduate Medical Education focus on observation of competence, our intervention provides a potential strategy for residency programs seeking a means to assess resident teaching skills.