Faculty Development Seminars Based on the One-Minute Preceptor Improve Feedback in the Ambulatory Setting

Abstract

OBJECTIVE

While several models of medical student instruction in the ambulatory setting exist, few have been formally studied. We wished to assess the impact of a faculty development workshop based on the One-Minute Preceptor model on the amount and quality of feedback in the outpatient setting.

DESIGN

Ambulatory teaching behaviors were studied during consecutive outpatient precepting sessions before and after 3 faculty development workshops. Student–teacher interactions were assessed using audiotapes of teaching encounters coded through qualitative techniques, and surveys of teacher, learner, and patient satisfaction.

SETTING

Ambulatory internal medicine clinic in a tertiary care medical center.

PATIENTS/PARTICIPANTS

Nine board-certified internist faculty preceptors and 44 third-year medical students.

INTERVENTIONS

Three 90-minute faculty development seminars based on the One-Minute Preceptor teaching model.

MEASUREMENTS AND MAIN RESULTS

Ninety-four encounters with 18,577 utterances were recorded, half before and half after the seminars. After the workshops, the proportion of utterances that contained feedback increased from 17% to 22% (P = .09) and was more likely to be specific (9% vs 15%; P = .02). After the workshops, teachers reported that the learning encounters were more successful (P = .03) and that they were better at letting the students reach their own Conclusions (P = .001), at evaluating the learners (P = .03), and at creating plans for post-encounter learning (P = .02). The workshops had no effect on the duration of the student–teacher encounter or on student or patient satisfaction with the encounters.

CONCLUSIONS

Brief, interactive, faculty development workshops based on the One-Minute Preceptor model of clinical teaching resulted in modest improvements in the quality of feedback delivered in the ambulatory setting.

Changes in medicine over the past 2 decades have moved a large proportion of patient care to the ambulatory setting. As a result, many educational experiences that were traditionally obtained on the inpatient wards have now shifted to outpatient clinics.

Inpatient teaching rounds usually occur at scheduled times for a specified duration, and are often centered on a preset agenda. In contrast, ambulatory teaching is often described as brief, episodic, and chaotic. The average duration of an ambulatory teaching encounter is only 4 to 10 minutes.¹ In those few minutes, teachers have to diagnose and manage the patient's problems as well as evaluate the student's strengths and weaknesses and make decisions about what to teach. Since diagnosing and managing the patient's problem occurs first, a large portion of the student–teacher encounter revolves around this issue. In a preliminary study of 50 ambulatory teaching encounters, 75% of the time was spent presenting the case and discussing patient management issues; only 30 seconds were spent providing feedback to the learner on how to improve.²

There has been a proliferation of teaching models designed to meet challenges in ambulatory teaching, many of which contain suggestions on improving feedback.^1,3–5 However, few of these models have been formally tested. As a result, leaders in medical education have declared that developing instruments to discover the most effective techniques for ambulatory teaching should be a research priority.⁶

One increasingly popular model for teaching in the ambulatory setting is the Five-Step Microskills model outlined by Neher et al., frequently nicknamed the One-Minute Preceptor technique.^6,7 In this model, the teacher first evaluates the learner by obtaining a diagnostic or management commitment and then probes for supporting evidence to back up the commitment. The preceptor teaches a limited number of general rules about the clinical case, followed by specifically reinforcing what the learner did right and correcting mistakes. Therefore, teachers create a milieu in which they are able to assess the knowledge, skills, and attitudes of the learner, succinctly fill any gaps in learner capabilities, and provide feedback to ensure that the lesson taught is appropriately assimilated. One prior study reported that residents on inpatient rotations randomized to a faculty development group learning the One-Minute Preceptor model improved their ability to probe learners for their diagnostic rationale and the amount of feedback they delivered to their interns and students. Improvement was measured through student and intern satisfaction questionnaires rating the residents on their teaching skills.⁸ We decided to determine if a faculty development program based on the One-Minute Preceptor model of clinical teaching could improve the amount and quality of feedback in the ambulatory setting.

METHODS

We studied the effect of an ambulatory faculty development workshop using a pre–post study design of teaching encounters between third-year medical students and their faculty preceptors. The study was conducted in an outpatient internal medicine clinic ambulatory medicine rotation at a tertiary care hospital (Walter Reed Army Medical Center). All internal medicine staff physicians were asked to participate. Our institution's human use committee approved this protocol, and informed consent was obtained from the preceptors.

Consecutive nonbedside ambulatory teaching encounters of faculty participants were audiotaped over a baseline period of 12 weeks. Typically, during our ambulatory medicine clerkship, preceptors will supervise 1 or 2 students for 1 or 2 clinical half days weekly. Therefore, while all encounters of a given preceptor were consecutively taped, the encounters did not occur on consecutive days of the week. Preceptors audiotaped their own encounters using a small, handheld tape recorder. These tapes were coded using the Teacher–Learner Interactive Assessment System (TeLIAS), a qualitative system of coding teacher and learner utterances. Coding of typed transcripts of the teaching encounters was divided between 2 investigators (SMS, JLJ) blind to the identity of the participants. To assess inter-rater reliability, samples of 10% of the audiotapes were double-coded. Furthermore, to assess intra-rater reliability, each investigator randomly re-coded 10% of the audiotapes 2 months after the entire coding process was completed. Half of the re-codes were selected from tapes obtained before the intervention, and half after the intervention. Spearman correlation coefficients to assess both inter and intra-rater reliability showed high levels of reliability (0.83 inter-rater reliability, 0.95 intra-rater reliability).

TeLIAS was developed to create a reliable and valid system for coding the structure and content of teaching and learning behaviors in ambulatory teaching encounters.² The unit of analysis in TeLIAS is an utterance, defined as the smallest discernible speech segment to which a classification may be assigned. Each teacher or learner utterance is potentially coded into 2 levels, a concrete typology of the utterance and an abstract “intent” of the utterance. All utterances are given a single concrete code; the codes are comprehensive and mutually exclusive. Concrete codes describe the nature of the utterance and are listed in Table 1. An utterance can broadly be coded concretely as a question, a statement, a transitional word (an utterance that changes the subject of discussion) or a back check (a filler utterance indicating that the speaker is listening). Questions can be further subclassified as clarifying, recall, analytic, or rhetorical and are further categorized as open or closed. We defined a closed-ended question as a question of narrow scope for which a single or very limited possibility of answers existed. An open question allowed a variety of possible or appropriate responses. For example, “Can you name some causes of right upper quadrant pain?” is a closed-ended question, whereas “Why do you think the patient has abdominal pain?” is an open-ended question. Statement codes can be summative, informative, non-integrative, thinking out loud, directive, or repeating something just stated by the partner. A summative statement integrates information and analysis, an informative statement conveys medical facts or patient data, and a directive statement conveys an order or instruction. To capture the more abstract meanings of utterances, secondary codes were created. Many utterances do not have a more abstract meaning or intent; other utterances may have more than one. Secondary codes are given in Table 2. Secondary codes were created after a review of the educational literature and were designed to classify teaching utterances by intent of speaker. While all utterances are coded into one concrete category, none, one, or more than one abstract code may be used. For example, if the preceptor asks the learner, “Why do you think the patient has anemia?” the utterance would be coded concretely as an open-ended analytic question and secondarily as a “Probes for understanding” utterance.

Table 1.

Classification of 18,577 Concrete Teacher and Learner Utterances in 94 Ambulatory Teaching Encounters

	Pre-seminar (N = 47)		Post-seminar (N = 47)
Utterance	Utterances	Total, %	Utterances	Total, %
Teacher
Question codes	713	17.9	657	18.2
Clarifying	280	7.0	253	7.0
Recall	119	3.0	92	2.5
Analytic	279	7.0	280	7.8
Rhetorical	35	0.9	32	0.9
Statement codes	2,139	53.7	2,223	61.6*
Summative	911	22.9	886	24.5
Informative	801	20.1	824	22.9†
Patient fact	139	3.5	146	4.1
Medical fact	662	16.6	678	18.8†
Nonintegrative	107	2.7	151	4.2
Thinking out loud	20	0.5	39	1.1
Directive	246	6.2	278	7.7
Repeats learner	52	1.3	45	1.2
Transitional word	118	3.0	145	4.0
Back check	1,016	25.5	584	16.2*
Total teacher utterances	3,986	100	3,609	100
Learner
Question codes	140	2.7	125	2.2
Clarifying	71	1.4	66	1.1
Recall	24	0.5	22	0.4
Analytic	33	0.6	26	0.5
Rhetorical	12	0.2	11	0.2
Statement codes	4,469	85.5	4,928	85.6
Summative	1,058	20.2	1,186	20.6
Informative	3,212	61.4	3,480	60.5†
Patient fact	2,946	56.3	3,126	54.3†
Medical fact	266	5.1	354	6.2†
Nonintegrative	107	2.0	151	2.6
Thinking out loud	31	0.6	44	0.8
Directive	15	0.3	16	0.3
Repeats teacher	46	0.9	51	0.9
Transitional word	76	1.5	100	1.66
Back check	539	10.3	608	10.6
Total learner utterances	5,227	100	5,755	100
Total utterances	9,213		9,364

^*P < .01.

^†P < .05.

Table 2.

Frequency of Abstract Codes Capturing the Intent of 18,577 Concrete Utterances

	Pre-seminar (N = 47)		Post-seminar (N = 47)
Utterance	Utterances	Total, %	Utterances	Total, %
Teacher-specific
Total feedback	676	17.0	701	19.4
Minimal	622	15.6	590	16.3*
Specific	50	1.3	106	2.9*
Behavioral	4	0.1	4	0.1
Interactive	0	0	1	0
Negative‡	61	1.5	101	2.8†
Total microskills	615	15.4	818	22.7*
Probes for commitment	60	1.5	64	1.8
Probes for evidence/Understanding	94	2.4	111	3.0
Teaches general rules	388	9.7	510	14.0
Reinforces what's right	50	1.3	106	2.9*
Corrects mistakes	23	0.6	27	0.7
Learning climate	21	0.5	38	1.1*
Communication of goals	24	0.6	62	1.7†
Promotes self-directed learning	18	0.5	36	1.0†
Inadequate wait time	52	1.3	46	1.3
Dominates discussion	8	0.2	3	0.1
Learner-specific
Makes commitment to diagnosis	88	1.7	105	1.8
Makes commitment to management	156	3.0	166	2.9
Expresses understanding of concept	10	0.2	18	0.3
Expresses uncertainty	110	2.1	101	1.8
Agrees with teacher	413	7.9	337	5.9

^*P < .05.

^†P < .01.

^‡All other types of feedback classified as positive or negative.

The 5 specific microskills of the One-Minute Preceptor model are included among the abstract coding categories of the TeLIAS system. We defined feedback as an utterance meant to relay information on a student's performance, or an utterance meant to elicit student self-evaluation. We defined several categories of feedback: minimal, specific, interactive, and behavioral and subclassified these as either positive or negative (Table 3). Some of our feedback categories overlap with One-Minute Preceptor microskills. For example, the microskill “corrects mistakes” is also negative feedback under our system. However, minimal negative feedback, because it does not provide an explanation of why the student is wrong, is not considered to be demonstrative of the One-Minute Preceptor microskill “corrects mistakes.”

Table 3.

Definition of Feedback Types

Feedback Type	Definition
Minimal feedback	Feedback indicating approval or disapproval of learner statement without explanation.
	“Good job,”“I agree,”“I'm not so sure about that.”
Specific feedback	Feedback specifically indicating why a statement the student made or non-behavioral action the student took was correct or incorrect.
	“I agree that the murmur you heard was likely mitral valve prolapse,”“I'd use 50 mg rather than 100 mg to start the patient on atenolol.”
Interactive feedback	Feedback eliciting the learner's self-evaluation of his/her performance with an interactive discussion based on the self-evaluation.
	“How do you think you did with this patient?”
Behavioral feedback	Feedback related to student behavior and attitudes in interpersonal relationships between the student, the patient, and members of the medical team.
	“You did a good job reassuring the patient about her underlying concern – that she didn't have cancer.”
Negative feedback	Any of above categories indicating disapproval of a student's action.

In our previous work, we found that a very small amount of time in each ambulatory encounter was spent providing feedback, but this amount varied widely by individual teacher. By using teachers as their own controls with a pre–post study design, one can minimize the impact on results of this variability in teaching behaviors, and discern smaller differences in effect. Since no previous studies have looked at the effect size on teaching behaviors one might expect with faculty development seminars, this approach would help ensure that small but important differences were not missed.

In addition to the audiotapes, preceptors and students filled out anonymous written surveys. The surveys consisted of statements the respondents were asked to rate on a 7-point Likert scale, with a score of 1 indicating strong disagreement and 7 indicating strong agreement. The questions covered 6 aspects of the teacher–learner encounter traditionally described in the literature—learning climate, control of session, understanding and retention, evaluation, feedback, and self-directed learning (Table 4). These aspects of the teacher–learner encounter have been widely promulgated as an integral part of the Stanford Faculty Development Program.^9,10 Patients also completed a questionnaire assessing their satisfaction with the encounter using the Rand-9 satisfaction survey.¹¹ Surveys were completed by the student and patient immediately after the encounter and placed in a lockbox in the clinic, and were collected by the investigators.

Table 4.

Perceptions of Teachers and Learners Regarding Ambulatory Encounters

	Teacher		Student
Survey Question*	Pre-seminar (N = 47) ±SD	Post-seminar (N = 47) ±SD	Pre-seminar (N = 46) ±SD	Post-seminar (N = 44) ±SD
Learning climate
Was the setting/tone of the encounter conducive to learning?	6.3 ± 0.6	6.3 ± 0.8	6.7 ± 0.6	6.5 ± 0.6
Control of session
Was time management efficient?	5.6 ± 1.1	5.6 ± 1.2	6.0 ± 1.4	6.0 ± 5.7
Evaluation
Was there sufficient opportunity to evaluate student competence during this encounter?	5.8 ± 0.9	6.2 ± 0.7†	6.5 ± 0.6	6.5 ± 0.7
Was bedside teaching appropriate for this case provided?	5.9 ± 0.1	5.7 ± 1.4	6.7 ± 0.5	6.5 ± 0.6
Understanding and retention
Did the teacher help the student reach his/her own Conclusions rather than telling the student what to do?	5.8 ± 0.9	6.3 ± 0.7‡	6.7 ± 0.5	6.5 ± 0.5
Feedback
Was the amount of feedback appropriate?	5.8 ± 0.8	6.1 ± 0.9	6.3 ± 0.8	6.3 ± 0.7
Was the feedback concrete and linked to specific student actions?	5.4 ± 1.1	6.0 ± 0.8†	6.3 ± 0.8	6.3 ± 0.7
Self-directed learning
Did the teacher help develop a plan for post-encounter learning?	4.5 ± 2.0	5.7 ± 1.6†	6.3 ± 0.8	6.4 ± 0.9

^*Agreement with statement on 1–7 Likert Scale (1 = strongly disagree, 7 = strongly agree).

^†P < .05.

^‡P < .01.

After the baseline period, study participants attended a set of three 90-minute faculty development seminars scheduled 1 week apart. The seminars consisted of a 30-minute mini-lecture and a 5-minute videotape of a staged student–teacher encounter designed to serve as a trigger tape for an interactive discussion. The seminars concluded with role-plays of teacher and student in small groups of two. The teachers played the roles of the students in these sessions, with a standardized script and a brief introduction from the faculty facilitators. Each role-play breakout group had a faculty facilitator. The mini-lectures were based on the One-Minute Preceptor model of clinical teaching. Each participant was also given a tape recording from one of their baseline teaching sessions for self-reflection, and a laminated pocket card describing the 5 microskills. After the intervention, audiotapes and surveys were again collected on consecutive ambulatory encounters for 12 weeks.

Analysis of the proportion of responses in each category was done using χ². Likert responses were analyzed using analysis of variance. Because of concern about the potential clustering of the results due to a differential effect on individual attendings, analyses of data were done using regression modeling (linear for continuous and logistic for categorical data) with the Huber/White/sandwich estimator of variance.¹² This method of analysis adjusts for observations that are clustered and is based on the assumption that the observations are independent between the clusters (here the individual attending), but not within.

RESULTS

Nine of thirteen eligible faculty preceptors volunteered to participate in the study. All preceptors were board-certified in internal medicine and had attended Stanford Faculty Development Program seminars between 1 and 8 years prior to this study. Seven preceptors were male, and two were female. All third-year medical students rotating on the service during the study period (N = 44) participated in a total of 100 teaching encounters. No patient participated in the study more than once. Preceptors averaged 5 encounters in each phase, with the total number of encounters contributed ranging from 6 to 15 overall. All teachers completed their post-encounter surveys, and 95% of student and 85% of patient surveys were collected.

Forty-seven of fifty audiotapes collected before and after the faculty development seminars were of acceptable technical quality. The average length of the student–teacher encounter was 15.2 minutes (before: 14.6 minutes; after: 15.8 minutes; P = .33). Total utterances (N = 18,577) were coded, averaging 198 utterances per teaching encounter. There was no difference in the mean number of utterances in each encounter before and after the workshop (before: 196; after: 199; P = .95).

Concrete Teaching and Learning Behaviors

Concrete teaching and learning behaviors are detailed in Table 1. Teachers listened to students more often after the seminars, with a small but significant decrease from 43% to 39% in total utterances in the student–teacher interaction (P < .01). The most common teacher utterances were summative statements, (before: 23%; after: 25%; P = .08), questions (before: 18%; after: 18%; P = .72), back check (before: 26%; after: 16%; P < .01), and medical facts (before: 17%; after: 19%; P = .01). A minority of the questions were open-ended (before: 21.9%; after: 28.4%; P < .01). The seminars, therefore, had modest but significant effects on increasing open-ended questions and teacher communication of medical facts, but reduced nondescript utterances purely spoken to indicate they were listening to the learner (back check). The quantity of more-sophisticated, synthetic summative information combining information with analytic reasoning did not improve.

Although students did the majority of speaking in teacher–student encounters, most of their utterances consisted of presenting the patient facts in the history and physical. However, the students showed small but significant changes, with a decrease in citing patient data (before: 56%; after: 54%; P = .03) and an increase in citing medical facts to display their knowledge base (before: 5%; after: 6%; P = .02) after the seminar series. The proportion of summative statements spoken by students (before: 20%; after: 21%; P = .63) was not significantly different after the seminar series.

Abstract Teaching and Learning Behaviors

Sixteen percent of teacher utterances contained 1 of the 5 One-Minute Preceptor microskills. (Table 2) Overall, the proportion of teaching statements reflecting use of the microskills increased significantly after the workshop from 16% to 21% (P = .03). Although all 5 microskills were used more frequently after the seminars, the only specific microskill to significantly increase was reinforcing correct information (P = .02).

Feedback was the most common abstract behavior coded both before and after the intervention. All 94 tapes contained feedback utterances, averaging 15 feedback statements per encounter. Eighteen percent of all teacher utterances were feedback to the learner. The majority of feedback (88%) was minimal, indicating simple approval or disapproval of learner statements, such as “I agree with your plan.” The quality of feedback improved after the intervention, with teachers providing over twice as much higher than minimal feedback (1.4% vs 3.0%; P = .03) after the faculty development seminars than at baseline. Most of this higher order feedback was specific, with preceptors stating the aspect of the student's statement they explicitly agreed with. Other types of higher feedback were rare, with only 8 behavioral statements and 1 interactive feedback statement noted during the entire study. After the intervention, teachers were also more willing to provide negative feedback. Negative feedback was nearly twice as common after the seminars than before (1.5% vs 2.8%; P = .007).

Preceptor, Learner, and Patient Perceptions of Encounter

A number of parameters improved from the preceptor's perspective after the faculty development seminars (Table 4). First, the preceptors felt that they had greater opportunity to evaluate the learners. They also felt that they were better able to assist the learners in arriving at their own Conclusions rather than having to outright tell the learner the correct answer. Although the amount of perceived feedback did not change among preceptors after the seminars, the sentiment that their feedback was linked to concrete learner behaviors did improve. Finally, teachers felt more confident that they provided plans for post-encounter learning. There was no change in teacher perceptions that their bedside teaching improved, that they were more efficient in time management, or that the learning climate of the ambulatory encounter changed.

Students had high levels of baseline agreement that learning climate, time management, opportunity for evaluation, bedside teaching, feedback, and post-encounter learning plans were appropriate. These high baseline levels of satisfaction did not change significantly after the seminars. Overall patient satisfaction was high at baseline, with a mean overall satisfaction component of the Rand-9 of 4.6 (out of 5 possible). Patient satisfaction did not change significantly after the seminars.

DISCUSSION

Our study represents the first time a semi-quantitative research tool with high inter-rater reliability has measured a subtle change in teaching behavior after a faculty development intervention. We found that a faculty development seminar based on the One-Minute Preceptor model modestly but significantly improved several aspects of ambulatory teaching. Most feedback before our intervention was low quality, providing learners no scaffold from which to improve. Feedback after the intervention was more likely to be specific. Teachers felt that they had greater opportunity to evaluate the students and were more likely to provide feedback linked to specific behaviors.

Providing feedback is considered to be an essential role for preceptors. Effective feedback is frequent, timely, linked to specific observed student behaviors, delivered with respect toward the learner, and generates a plan for self-improvement.^13–16 Despite the importance placed on effective feedback, few previous studies have examined the nature of feedback in outpatient teaching encounters, and even fewer have focused on interventions to improve the quality of feedback.

Feedback in ambulatory encounters has previously been found to vary from 3% to 16%,^17,18 and was rarely explicit or negative.¹⁹ One study of a 45-minute intervention using individual feedback of teaching observed on videotapes improved the amount of positive feedback from 8% to 28% of encounters, without changing specific feedback.²⁰ After 5 months, the number of encounters with positive feedback nearly returned to baseline. Another trial that audiotaped 376 preceptor–student encounters using a checklist found feedback in 24% of encounters. In that study, university preceptors and community preceptors were not significantly different in the amount of feedback delivered.²¹

Finally, Holmboe et al. found that a 20-minute didactic session on evaluation and feedback in the inpatient setting resulted in a trend toward increased quantities of specific written feedback at the end of the rotation.²² Three-by-five cards with reminder prompts about key elements of effective evaluation and feedback, and space to write notes were given at the end of the session. Preceptors could then use the notes when formulating their end-of-rotation feedback. However, each end-of-rotation written evaluation had a relatively low level of specific feedback on it, with an average of 2 specific statements in the control group of physicians compared with 3 in physicians who were randomized to the faculty development intervention. The 3″× 5″ card approach to stimulating feedback has been endorsed by other researchers, and has been associated with high rates of compliance in the ambulatory as well as inpatient settings.²³

Learner satisfaction surveys reflect similar themes of deficient quantities of feedback detected in the observational studies. Isaccson et al. surveyed 65 residents on their perceptions of feedback received. Only 8% of residents reported being satisfied with feedback received from their preceptors. While 83% of residents reported receiving positive feedback at least some of the time, only 20% reported receiving any corrective feedback at all.²⁴ Gil et al. reported similar negative perceptions among medical students.²⁵ Despite the perceived lack of feedback, learners consider feedback important. Several studies have shown that preceptors who give in-depth feedback have been listed as admired or respected role models.^26–28

Our study demonstrated an improvement in feedback, but the absolute increases were small. The average teaching encounter in our study was 14 minutes. In those 14 minutes, teachers had to diagnose and manage the patient's problems (a major focus, given the early stage of the learners), while diagnosing the learner as well. How much feedback is an appropriate target? In our study, preceptors spent about 2 minutes on feedback, mostly on minimal feedback, such as “I agree.” It is not clear whether minimal feedback has low educational value and reflects an attempt by teachers to promote a safe learning climate rather than to communicate specific information to the learner. Since most existing education literature suggests most feedback is minimal and most learners are not satisfied with the feedback they receive, we surmise that learners desire more insight in how to specifically improve to achieve the next step toward clinical independence. This specific question awaits clarification from further research. We feel that ambulatory teaching faculty development workshops based on the One-Minute Preceptor model modestly improved the quality of feedback, providing better conceptual scaffolding from which learners could improve.

One prior trial studied the effect of the One-Minute Preceptor model in resident physicians teaching in the inpatient setting, as measured by student and intern satisfaction and resident self-evaluation.⁸ The resident teachers undergoing training in Five-Step Microskills techniques felt better able to evaluate and provide feedback to their learners, which was similar to our results. Learners had significantly greater perception that they had a larger part in clinical decision making, more feedback, and greater stimulus to self-directed learning when One-Minute Preceptor techniques were used. This differs from our results, in which no significantly different perception in learners was demonstrated. It is possible that the teachers and learners in the inpatient setting may have had greater length and continuity of interaction, allowing greater time for learners to form perceptions. None of the feedback on our transcripts was preceded by a statement from the teacher to the learner identifying that feedback was forthcoming. Experts in medical education suggest that effective feedback should be labeled as such.¹³ This may provide a clue as to why our learners did not perceive that they were receiving feedback.

Our study differs from prior literature examining feedback in the outpatient setting in several respects. First, most prior studies demonstrated the presence or absence of feedback indirectly with surveys or directly using videotapes or monitors observing the preceptor–student encounter with a predetermined checklist.^15,16,18,19 In some cases, these studies were not blinded, allowing for the possibility of observer bias. Our semiquantitative technique using explicit feedback definitions provided a rigorous tool with which to quantify the amount and nature of verbal feedback delivered in the context of the teacher–learner interaction.

Second, all of our teaching encounters contained some form of feedback. This differed from other prior studies that found a much lower proportion of feedback per encounter in the range of 3.4% to 28%.^3,18,20,21 This difference may be due to the increased time for reflection our coders had in considering whether to classify a statement as feedback in the context of a written transcript. In contrast, observers in prior studies often had to make real-time coding decisions. Another reason may be that these studies did not contain an explicit definition of what was considered feedback, and it was possible that they did not consider minimal feedback to be feedback at all. Finally, nearly 90% of feedback delivered was positive in our study. This confirms the finding of the prior feedback literature that negative feedback is not commonplace.^17–20,29

We were able to demonstrate a doubling of more-sophisticated, non-minimal feedback statements after the faculty development seminars. Most of the increase was due to a greater number of specific feedback statements. The amount of negative feedback as a percentage of total feedback also increased a small but significant amount after the faculty development seminars. We attributed this improvement to the One-Minute Preceptor model we selected for the seminars. Two of the 5 microskills are focused on providing specific positive and negative feedback to the learner. This provides a plausible explanation of why the amount of feedback increased and why the type of feedback that increased was of the specific variety.

The amount of behavioral and interactive feedback remained negligible before and after the seminars. We wondered if our focus on the One-Minute Preceptor model may have been responsible. In the One-Minute Preceptor model, the learner is asked to make a commitment on a patient care issue that is prompted by the teacher. In effect, the teacher sets the agenda for the learning encounter. Furthermore, the teacher initiates the feedback process, which limits the opportunity for self-assessment. It is possible that a more learner-centered approach asking the learner what he or she wanted to focus on might have led to more behavioral and interactive feedback. Therefore, although the One-Minute Preceptor seems useful in many outpatient settings, having a versatile approach to the student–teacher encounter by using other techniques such as student self-assessment and role-play, when appropriate, seems prudent.

LIMITATIONS

Our study has several limitations. First, all participating faculty members had completed the Stanford Faculty Development program, which may have made them more receptive to our ambulatory faculty development seminars. Second, our study was conducted using internist preceptors in an ambulatory clinic in a tertiary care medical center that may have different characteristics than community-based teaching sites. Third, since we used third-year medical students in our encounters, our results may not be generalizable to learners at different levels. Fourth, audiotaping ambulatory encounters had the potential to miss nonverbal feedback. It is also possible that we missed teaching behaviors by not recording at the bedside. However, our previous research has found that over 95% of teacher communication at the bedside is spent verifying the history and exam, with very little time devoted to teaching or feedback.² Fifth, our results are limited to the short-term effects of a faculty development workshop based on the One-Minute Preceptor model of teaching. Since a previous study found that the effects of their faculty development program diminished over time,²⁰ how long the improvements we report linger is uncertain. Further research is needed to determine how frequently faculty development is needed to sustain the improvements demonstrated in this study. Finally, although coders were blind to the participants' identities, tapes were coded as they were collected. Therefore it is possible the coders could have guessed which phase the audiotapes represented. To assess for the possibility of bias, we re-coded a random and blinded 10% sample from each phase. The very high intra-rater reliability we found (Spearman's ρ = 0.95) would suggest no change in coding behavior based on inadvertent unblinding of the study phase.

Since our faculty development seminars were focused on teaching the One-Minute Preceptor model, it may not be surprising that the number of utterances containing microskills increased significantly after the seminars. However, the absolute increase in the number of microskills coded was a modest 5% of total teacher utterances. There are several reasons why a larger increase may have not occurred. First, it is possible that the student–preceptor interaction is different from the resident–preceptor setting in which the microskills model was first proposed. Students may be less amenable to preceptors probing for commitment, supporting evidence, and understanding if they are not able to formulate a differential diagnosis or management plan. It is also possible that we used an overly strict definition of “reinforcing what's right” and “correcting mistakes.” Feedback had to be interactive, specific, or behavioral in nature and linked in close juxtaposition to a student utterance to qualify for this classification in our methods.

CONCLUSION

We found that a faculty development workshop based on the One-Minute Preceptor model of ambulatory teaching increased use of the 5 microskills in the model, especially preceptor feedback. Feedback after the seminars was more likely to be linked to specific student behaviors. Teachers also reported themselves more confident in a number of important domains of teaching, including evaluation, providing concrete feedback, allowing the learner to reach his/her own Conclusions, and providing a plan for post-encounter learning, without increasing the encounter duration. Both learner and patient satisfaction remained high before and after the seminars. Our findings also confirm the usefulness of our qualitative coding system in assessing changes in teaching behavior in the context of an established model of ambulatory instruction.