Oral Versus Written Feedback in Medical Clinic

Abstract

OBJECTIVE

To determine whether residents perceived oral, face-to-face feedback about their continuity clinic performance as better than a similar, written version.

DESIGN

Single-blind, randomized controlled trial.

SETTING

Two university-based, internal medicine residency clinics.

PARTICIPANTS

All 68 internal medicine and combined program (medicine-pediatrics, medicine-psychiatry, medicine-neurology, and preliminary year) residents and their clinic preceptors.

MEASUREMENTS AND MAIN RESULTS

Residents at each program were separately randomized to oral or written feedback sessions with their clinic preceptors. The oral and written sessions followed similar, structured formats. Both groups were later sent questionnaires about aspects of the clinic. Sixty-five (96%) of the residents completed the questionnaire. Eight of the 19 questions dealt with aspects of feedback. A feedback scale was developed from the survey responses to those eight questions (α= .86). There were no significant differences in the responses to individual questions or in scale means (p >.20) between the two feedback groups. When each university was analyzed separately, one had a higher scale mean (3.10 vs 3.57, p = .047), but within each university, there were no differences between the oral and written feedback groups (p >.20).

CONCLUSIONS

No differences were observed between the oral and written feedback groups. In attempting to provide better feedback to their residents, medical educators may better apply their efforts to other aspects, such as the frequency of their feedback, rather than the form of its delivery.

Feedback consists of information provided to learners for the purpose of reinforcing appropriate and correcting inappropriate efforts. It is an essential, but often neglected part of the educational process. Surveys reveal that students consider receiving constructive feedback to be a crucial component of the learning environment.¹ Although they identify certain aspects of feedback as highly important, these are frequently lacking in students' educational experiences.² In his observations of excellent teachers of clinical medicine, Irby emphasizes that his subjects universally devoted considerable time and effort to providing quality feedback, and he lists it as a crucial component in quality teaching.^{3, 4} Clinical educators vary greatly, however, in the style and structure of their feedback sessions,⁵ and it is unclear which methods are preferable.

Despite agreement on the importance of giving feedback, little information is available regarding practical methods of providing feedback to residents in ambulatory settings. One recent study demonstrated that written feedback could improve residents' teaching skills.⁶ Another found that oral feedback improved the documentation in clinic charts, but not other aspects of clinic performance.⁷

Previous to this study, the internal medicine residency programs at West Virginia University and Texas A&M University routinely provided feedback regarding continuity clinic performance to each resident at 6-month intervals. These sessions were held in the fall and spring. The private, face-to-face sessions required significant amounts of faculty time, and several preceptors had queried whether such sessions were worth the effort. This study was undertaken to determine whether there is a demonstrable advantage to face-to-face, oral feedback over a similar, written version.

Several authors have examined the feedback process to determine which aspects are important. In his classic article on the subject, Ende describes correctly delivered feedback as timely, specific, objective, and nonevaluative. It should address actions and decisions, remediable behavior, and common goals.⁸ One study examined the content elements of feedback that medical students desire and how these change as the learners mature, but the authors did not identify preferable methods of delivery.⁹ A study involving computer-based instruction found that several types of feedback were equally effective for medical students.¹⁰ Our hypothesis for this study was that a written form of feedback would be equal in efficacy to an oral one for internal medicine residents in continuity clinic.

METHODS

During the study interval (July 1993 through June 1994), all internal medicine residents at both West Virginia University (n= 39) and Texas A&M University (n= 29), had weekly continuity clinics in on-campus, ambulatory centers. The residents at West Virginia University were from categorical internal medicine, medicine-pediatrics, medicine-psychiatry, medicine-neurology, and preliminary programs, while those from Texas A&M University were either categorical internal medicine or preliminary year residents.

The organization of the continuity clinics in both programs was such that a team of residents was assigned to a given clinic session with a limited number of general internal medicine faculty members serving as preceptors. Before the study, the preceptors had provided scheduled, one-on-one feedback to each resident on the team. These sessions included reviewing the resident's continuity clinic performance (Evaluation Form), reviewing the number of patients seen (Data Form), and deriving a plan of action to address the resident's needs (Action Form). The preceptors were informed about the design of the study and its goals, and all the preceptors agreed to participate in the study.

For the study, residents in each program were separately randomized to either oral or written feedback about their performance in continuity clinic. Preceptors, but not residents, were aware of the randomizing process. Although resident teams were not a unit of randomization, each preceptor was involved in both types of feedback. Each preceptor received instruction on the structure of the feedback sessions and on the principles of giving quality feedback. For both written and oral feedback, the same Evaluation, Data, and Action Forms were used. The intervention feedback session was provided midway through the academic year.

Each resident randomized to receive written feedback was sent an individualized packet that included (1) a description of the procedure for assessing performance in continuity clinic, (2) the Evaluation Form completed by the preceptor for the most recent period, (3) the Data Form, (4) the Action Form completed by the preceptor, and (5) instructions for arranging a meeting with the preceptor to review the evaluation if the resident desired to do so. This form of feedback was new at both institutions, so these residents served as the experimental group. All written evaluations were completed before the oral feedback sessions.

A meeting with the preceptor was arranged for each resident randomized to oral feedback. Preceptors followed an outline for the oral feedback meetings: (1) state that the purpose of the meeting is to provide feedback on performance in continuity clinic, to assist in identifying areas of strength and areas needing improvement, and to help determine means to enhance clinical skills; (2) provide data via the Data Form; (3) provide formative evaluation (review the completed Evaluation Form, discuss strengths and weaknesses, and exchange feedback on these perceptions); (4) solicit feedback from the resident on perceived needs regarding specific ambulatory medicine content areas or clinical experiences; and (5) with the resident, jointly complete the Action Form. This format was a more structured version of the usual feedback process, and the residents receiving it served as the control group.

One month after receiving their feedback, the residents were asked to complete confidential surveys regarding their ambulatory experiences. The surveys consisted of 19 statements with 7-point Likert responses (1 = strongly agree to 7 = strongly disagree). The residents did not know that quality of feedback was the central issue being tested, and questions about other aspects of the clinic were included in the survey. The nonfeedback questions addressed the quality of teaching in clinic, residents' workloads, patient mix, medical problems seen in clinic, and how much the residents enjoyed their clinic experiences.

All analyses were performed using Statistical Analysis System (SAS) software. Comparisons between categorical variables were made using Mantel-Haenszel χ² Test, and Student's t test was used for comparing means of continuous variables. The responses to survey questions dealing with feedback issues were combined into a scale, using the factor analysis methodology described by De Vellis.¹¹ Criteria for scale development included primary loading>.40, secondary loading <.30, and Cronbach's α>.70. The power of the survey was 80% to detect a difference (p < .05) in the scale means of .65. All p values are two-tailed.

RESULTS

All the residents randomized to receive oral feedback (n= 35) completed their sessions. None of the residents randomized to receive a written feedback (n= 33) requested an oral session. Completed questionnaires were returned by 65 (96%) of the residents. The faculty members uniformly agreed that providing written feedback required much less time than the oral sessions, stating that the range of time per resident was 2 to 5 minutes for the written feedback versus 10 to 15 minutes for the oral sessions.

The residents' perceptions of the equality of the feedback they received in clinic were assessed by eight questions (Table 1) on the survey. There were no significant differences between the two feedback groups on responses to individual questions concerning their clinic feedback (all p>.20). These responses were combined into the feedback scale, for which Cronbach's α was .86. The scale's mean was 3.36 for all residents. The feedback scale mean for the residents receiving oral feedback was 3.37, and the mean for those receiving written feedback was virtually identical at 3.35 (p= .95).

Table 1.

Feedback Questions

Differences were observed between the residency programs. The feedback scale mean for West Virginia University residents was 3.57, while that for the Texas A&M residents was 3.10 (p= .047). The more favorable evaluations by Texas A&M University residents were also observed for other aspects of the clinic. The Texas A&M University residents were more likely than the West Virginia University residents to agree that they enjoyed seeing clinic patients, they did outside reading about their clinic patients, and clinic did not interfere with ward duties (all p < .01). When analyzed separately by university, there were no differences between residents receiving oral feedback and those receiving written feedback in their survey responses to feedback issues (p's >.20).

We examined the two different feedback groups to determine whether their perception of clinic differed in ways other than with regard to feedback (i.e., the nonfeedback questions). No differences were found in their responses to questions about workload, quality of patient mix, or other aspects of clinic in the survey (all p>.10). The mean response score was 3.42 for nonfeedback questions (3.39 for the oral feedback group vs 3.45 for the written feedback group).

DISCUSSION

Our study did not show a difference in the effectiveness of two styles of feedback, and there are several possible explanations for this result. Perhaps the intervention was too weak, and other, less formal forms of feedback exerted a stronger effect on the residents. We have only studied formal feedback sessions, but feedback could be occurring on a more continuous basis during clinic interactions. Increasing the frequency or the length of the sessions may have increased the magnitude of effect. Alternatively, the residents may not value feedback from their clinic attending physicians and may dismiss both the oral and written forms. This explanation seems unlikely because the responses to study questions indicate that the residents value their clinic feedback as much as other aspects of ambulatory care. Furthermore, experts on teaching in ambulatory settings stress the importance of providing feedback.^{12, 13} We feel that the two formats of feedback are equally effective, but other aspects of feedback, such as those discussed above, may simply be more important than the format of its delivery. Earlier studies have indicated that written or oral feedback can be effective in changing the behavior of medical students and medical educators.^14–16

There are some limitations in the study. First, our measure of feedback efficacy was the residents' perception of effect. Other studies have shown that instructors and learners do not always perceive feedback similarly.^{2, 3} We did not measure behavioral changes based on the feedback, and the feedback could have been effective in ways not perceived by the residents. Our survey instrument has not been previously validated or examined for reliability in other groups. However, it is similar in design to others used to measure residents' attitudes about educational experiences,^{17, 18} and our feedback questions were specifically designed to address aspects of feedback that other authors had shown to be important.^{3, 4, 6} Lastly, our residents may be atypical in their responses to feedback, and residents in other programs may react differently.

Performing the study at two universities strengthened it from several standpoints. The residencies have identical clinic structures, but are geographically distant. There was little possibility of contamination between residents at the two sites. Within each clinic site, residents had essentially the same type of clinic experience (same patient pool, one site) with the exception of the intervention being studied. Because there were no significant differences in the residents' perceptions of feedback quality between sites, preceptor characteristics (such as being able to give feedback) seem an unlikely explanation for the lack of differences between the two feedback session formats. The power of the survey was sufficient to show differences between other aspects of the residency programs, which supports our ability to measure differences in resident opinions with our instrument, if they exist.

If residents perceive that different formats have equal efficacy, program directors might want to use the feedback structure that is most feasible locally. Oral feedback sessions are very time-intensive compared with written feedback, so the implication of these results is that residency programs should channel their feedback efforts into different areas. Increasing the frequency of feedback or the level of detail would be options. Methods to evaluate behavioral changes resulting from the feedback would be another direction to explore. Face-to-face sessions may have value in other areas, such as encouraging mentoring or camaraderie, but this form of interaction does not appear to enhance the residents' perception of the feedback process.

There is little argument about the importance of providing feedback to learners. As the outpatient component of residency education increases, providing adequate feedback in this setting will become increasingly important. Preceptors, many of whom already feel overextended providing care to patients and teaching, will be hard pressed to do so. Determining the most effective and efficient methods of providing feedback in ambulatory settings merits further investigation.