Integrating Teaching Skills and Clinical Content in a Faculty Development Workshop

Abstract

Incorporating clinical content into medical education faculty development programs has been proposed as a strategy to consolidate faculty continuing medical education time and enhance learning. We developed a faculty development program for ambulatory internal medicine preceptors that integrated primary care genetics with ambulatory precepting. The instructional strategies addressed both areas simultaneously and included facilitated discussions, mini-lectures, trigger tapes, and role plays. To evaluate the program, we conducted a pre-post trial. Skills were measured by retrospective pre-post self-reported ratings and behaviors by self-reported implementation of commitment to change (CTC) statements. Participants' (N = 26) ambulatory precepting and primary care genetics skill ratings improved after the intervention. They listed an average of 2.4 clinical teaching CTC statements and 2.0 clinical practice CTC statements. By 3 months after the workshop, preceptors, as a group, fully implemented 32 (38%), partially implemented 35 (41%), and failed to implement 18 (21%) CTC statements. The most common barrier to clinical teaching change was insufficient skills (8 of 25; 32%) and to clinical practice change was lack of a suitable patient (15 of 25; 60%). Integrating clinical content with clinical teaching in a faculty development workshop is feasible, can improve clinical and teaching skills, and can facilitate behavior change.

Undergraduate and graduate medical education programs continue to expand ambulatory training, recognizing the discordance between the educational content of inpatient education and the practice of medicine. New training sites include university-based teaching practices and community offices.^1–3 Although these experiences offer unique opportunities,^4,5 they present several challenges for preceptors, including limited interaction time,⁶ barriers to accessing medical information,⁷ balancing learner autonomy with supervision, and, for community preceptors, the threat of diminished productivity.^8–10

Many institutions offer faculty development programs to help ambulatory preceptors overcome these challenges and improve their teaching skills.¹¹ However, these programs compete with clinical continuing medical education (CME) programs for preceptors' limited time. Adding clinical content to these faculty development workshops has been proposed as a strategy to consolidate preceptors' CME time and institutions' CME resources.^12,13 Furthermore, integrating the 2 domains may facilitate learning as participants apply the clinical content in clinical teaching scenarios.

In this article we describe the development and implementation of a faculty development program that integrated primary care genetics with office precepting skills. Additionally, in a prospective trial, we determined the impact of the program on participants' skills and behaviors in both areas.

METHODS

Subjects and Setting

We invited university-based and community-based general medicine faculty who precepted medical students or internal medicine residents in a variety of ambulatory settings, including community practices, hospital-based clinics, community health centers, and a Veterans' Administration hospital. We advertised the faculty development program through mailed brochures, e-mail, and telephone follow-up. Participation was voluntary.

Intervention (Faculty Development Workshop)

In developing the curriculum for the workshop, we were guided by a needs analysis, which included an internal review of current faculty development efforts and focus groups and a written survey of community-based preceptors. The deputy dean for medical education and the ambulatory internal medicine student clerkship director facilitated 6 focus groups of 4 to 5 community-based preceptors. We asked the preceptors to identify ways in which the medical school could facilitate their continued participation in the teaching program, including faculty development, subsidization of CME, greater integration into “university life,” and the impact of trainees on their office operations. The written survey was mailed to community-based preceptors in the third-year ambulatory internal medicine clerkship, the fourth-year primary care clerkship, and the primary care residency program ambulatory block rotation. The survey inquired about preceptors' level of interest (on a 3-point scale) in training in specific teaching skills, rewards for participation in the teaching program, and participation in faculty development workshops, including preferences for timing, frequency, and location. It also included an open-ended question about “barriers to effective precepting.” Thirty-nine of 105 (37%) preceptors returned the survey after 2 mailings.

The focus groups and survey revealed the following themes:

1. A strong desire to attend faculty development workshops that addressed the specific challenges of teaching in the community office

2. Experiencing time pressure as a formidable barrier to accommodating trainees (although preceptors expressed that they could not afford to reduce their patient volume, even though this might improve their effectiveness as teachers)

3. A high level of interest in improving clinical competence in specific areas

4. Difficulty finding time to attend faculty development workshops and other CME activities. (Within this constraint, preceptors expressed a preference for biannual frequency, weekday evening times, and a central location.)

The needs analysis reinforced our decision to integrate clinical content with training in clinical teaching within a single workshop, because this would allow preceptors to consolidate their limited CME time and satisfy their interest in improving their clinical competence. In developing the curriculum, we focused on the precepting encounter in the community office. In response to the preceptors' lament of time constraints, we sought to help them efficiently utilize the brief interaction time with a learner. Our choice of clinical content did not directly derive from the needs analysis, because the survey did not include questions about specific clinical topics. We chose genetics as our clinical content because of its rapidly increasing relevance to primary care providers, our access to local expertise, and our belief that this “hot topic” would attract participants. In particular, we covered genetic susceptibility to cancer, focusing on screening for mutations in the BRCA genes. We offered the same workshop on 2 evenings in locations 25 miles apart.

Table 1 lists the learning objectives. The workshop, entitled “Listening with Both Ears,” lasted 3 hours, following the format in Table 2. The same vignette of a trainee presenting a woman seeking BRCA screening and subsequently counseling her was used in all of the instructional strategies, including facilitated discussions, trigger tapes, mini-lectures, and role plays. We emphasized the use of probing questions to simultaneously identify the trainee's immediate learning needs and assess the patient's presenting problems, allowing the preceptor to customize his or her approach to take full advantage of the “teaching moment.” We also included other techniques designed to maximize precepting efficiency, including the “1-minute preceptor,”¹⁴ priming,¹⁵ and role-modeling.¹⁵ Participants received a bibliography of ambulatory teaching literature, reprints of 2 ambulatory teaching articles,^15,16 reprints of 2 genetics articles,^17,18 a genetics information sheet, copies of slides, and role play and observation checklists.

Table 1.

Learning Objectives for Faculty Development Workshop

Ambulatory Precepting	Primary Care Genetics
Utilize questions to assess learning needs, evaluate learner, and promote self-directed learning	Estimate a woman's risk of breast cancer in general and of having one of the BRCA mutations
“Diagnose” the patient and the learner	Interpret the results of BRCA testing
Practice ambulatory teaching strategies that target learning needs	Understand the effectiveness and risks of breast cancer risk reduction strategies
Emulate the “one minute preceptor”	Counsel a women regarding the potential advantages and disadvantages of BRCA screening
Allow a learner graduated autonomy while providing appropriate supervision	Appreciate the ethical, legal, and social issues surrounding genetic testing

Table 2.

Format of Faculty Development Workshop

1. 5 minutes	Introduction, objectives, and agenda
2. 5 minutes	Videotape demonstration of a case presentation by resident
3. 30 minutes	Facilitated discussion of genetics aspects of the case and mini-lecture to reinforce key points. (Facilitator with genetics expertise)
4. 5 minutes	Video trigger tape of precepting vignette A, which continues the case in #2 (ineffective preceptor). Includes case presentation after resident evaluated the patient, case discussion, and interaction between preceptor, resident, and patient in the examination room.
5. 30 minutes	Facilitated discussion of clinical teaching aspects of vignette followed by mini-lecture to reinforce key points. (Facilitator with clinical teaching expertise)
6. 30 minutes	Dinner
7. 30 minutes	Role play of the same vignette in breakout rooms. Participants are organized in groups of 3, one playing the preceptor, one playing the resident, and one observer using a checklist of precepting behaviors. A facilitator in each break-out room guides the participants.
8. 45 minutes	Working dessert (back in large group)
	Debrief role play (10 minutes)
	Video trigger tape of precepting vignette B, which continues the case in #2 (effective preceptor) and discussion (15 minutes)
	Evaluations and reflections (15 minutes)
	END
Total Time = 3 hours

Outcome Assessment

At the end of the workshop, the participants completed a written evaluation. This was voluntary for our curriculum evaluation purposes, but it was required for receiving CME credit. Participants rated their skills in ambulatory precepting and primary care genetics in a retrospective pre-post fashion.¹⁹ That is, they simultaneously completed a retrospective pre-rating (an assessment of their competence before the workshop but done after the workshop) and a post-rating (an assessment of their competence after the workshop) on a 5-point scale ²⁰ from “I do not do this” to “I am highly competent in doing this.” They also responded on a Likert scale to statements concerning the effectiveness and attractiveness of integrating clinical teaching with clinical content.

Finally, participants wrote “commitment to change” (CTC) statements.^21,22 They were asked to identify “up to 3 concrete, measurable changes that you will employ in your clinical teaching as a result of this workshop” and “up to 3 changes in your clinical practice…” Three months later, we mailed the participants a follow-up survey, which reproduced their CTC statements. For each CTC statement, they indicated whether they had implemented it fully, partially, or not at all. If they had not fully implemented a change, they identified from a list of options the primary barrier to implementation.

Data Analysis

We compared the pre- and post-skill rating scores with the Wilcoxon signed rank test for paired data. We described the overall rate of implementation of the CTC statements and the barriers identified for unfulfilled commitments. In addition, we compared the implementation rates for the 2 preceptor locations (university or community) and for the 2 types of commitments (clinical or teaching). We dichotomized the implementation variable for these comparisons, maintaining the “fully implemented” responses and collapsing “partially implemented” with “not at all implemented.” We collapsed the data in this way in order to relate the implementation rates to the perceived barriers, which were identified for all commitments not “fully implemented.” To test for statistical significance of these comparisons, we used logistic regression with hierarchical modeling to adjust for effects at the level of the preceptors, each of whom had the opportunity to “implement” multiple commitments of both types. Thus, we do not present simple bivariate statistics, because these do not account for between-preceptor differences.

Human Subjects

The human investigation committee granted this project an exemption from formal review.

RESULTS

Twenty-six preceptors attended the workshop. Twenty-four (92%) completed the evaluation at the conclusion of the workshop and, of these, 22 (92%) returned the 3-month follow up form. Fifty-nine percent worked in private practice or community health centers, and 41% worked in university-based teaching clinics. Twenty-three percent precepted only medical students, 32% precepted only residents, and 45% precepted both. Forty-two percent of preceptors were attracted to the workshop mainly by the office-precepting content, 4% by the genetics content, and 54% by both.

The preceptors' self-reported retrospective pre-post skill ratings are listed in Table 3. Significant improvements occurred in all skills. Notably, at baseline, the ambulatory precepting skills were higher than the primary care genetics skills.

Table 3.

Retrospective Pre-post Skill Ratings (N = 24)^*

	Before Workshop	After Workshop
Skill	Median (Range)	Median (Range)†
Identify trainee's learning needs	3 (2 to 5)	4 (3 to 5)
Set a limited teaching agenda	3 (2 to 5)	4 (3 to 5)
Utilize priming and modeling	3 (1 to 5)	3.5 (3 to 5)
Maintain trainee's sense of ownership	3 (1 to 4)	4 (2 to 5)
Identify women at high risk of having a BRCA	3 (1 to 5)	4 (2 to 5)
Interpret the results of BRCA testing	2 (1 to 5)	3 (1 to 5)
Explain the effectiveness and risks of risk reduction strategies for BRCA	2 (1 to 5)	3.5 (2 to 5)
Council a woman about genetic screening	2 (1 to 5)	4 (2 to 5)

^*Rating scale: 1 = I do not do this, 2 = generally not able to do this, 3 = somewhat able to do this, 4 = quite competent in doing this, 5 = I am highly competent at doing this.

^†P < 0.001 for pre-post comparison.

The preceptors listed a total of 95 CTC statements, including 52 (55%) clinical teaching and 43 (45%) clinical practice. Individual preceptors listed a mean of 2.4 (range 1 to 3) clinical teaching CTC statements and 2.0 (range 0 to 3) clinical practice CTC statements. Most of the CTC statements related to the workshop's learning objectives. The most common teaching statement was “assess trainees' learning needs” and the most common clinical statement was “assess women's risk for breast cancer and BRCA.”

The mean time to follow up was 14 weeks (range 12 to 17). Of the 95 total commitments, preceptors, as a group, fully implemented 32 (38%), partially implemented 35 (41%), and failed to implement 18 (21%); 10 responses were left blank. The full implementation rates did not significantly differ between university-based (30%) and community-based (44%) preceptors (odds ratio [OR], 0.53; 95% confidence interval [CI], 0.16 to 1.8) or between clinical teaching (43%) and clinical practice (32%) commitments (OR, 1.6; 95% CI, 0.55 to 4.9). Individual preceptors fully or partially implemented an average of 79% (SD, 22%) of their own commitments. Nineteen of 22 (86%) preceptors implemented at least 1 change.

Table 4 lists the barriers identified for the changes that were not fully implemented. The most common barrier to implementing teaching changes was insufficient skills, although the distribution was fairly equal. In contrast, not encountering a suitable patient accounted for 60% of the barriers to clinical practice changes.

Table 4.

Barriers to Implementation of Changes

Barrier*	Clinical Teaching Changes, n (%)	Clinical Practice Changes, n (%)
“I have not encountered a suitable learner/patient yet”	4 (16)	15 (60)
“I do not have enough time during a patient visit”	4 (16)	2 (8)
“I need to improve my skills before I can implement this change”	8 (32)	3 (12)
“Systems or logistical barriers in practice prevented me”	4 (16)	2 (8)
“This change is not important enough to my clinical teaching/practice”	0	0
Other	5 (20)	3 (12)
Total	25	25

^*Barriers identified by 22 preceptors for the 53 “commitment to change” (CTC) statements that they failed to fully implement. The barrier question was left blank for 3 CTCs, leaving 50 listed in the table.

DISCUSSION

Guided by a needs analysis, we developed a faculty development model for integrating clinical content with training in clinical teaching and evaluated its impact on participants' skills (competence) and behaviors (performance). Our results demonstrate that this approach can improve participants' self-reported skills in both areas—in this case primary care genetics and ambulatory precepting. In addition, a majority of participants fully or partially implemented changes in their behavior areas within 3 months.

Combining clinical content and clinical teaching in faculty development workshops offers many advantages. On a practical level, this model allows faculty to consolidate their limited CME time as opposed to attending separate programs. In terms of educational effectiveness, it may synergistically enhance learning as participants' apply the clinical content in clinical teaching scenarios and vice versa. This approach also creates conditions appropriate to adult learners.²³ Faced with the task of precepting the trainee in the teaching exercises, the participants immediately need to know the clinical content. Furthermore, the trigger tapes and role plays simulate the actual preceptor-trainee-patient encounter—a moment in which clinical and teaching skills must be exercised simultaneously. Studies in cognitive psychology suggest that this encoding specificity should enhance recall in “real life” precepting scenarios.²⁴

Our intervention also included features that have demonstrated efficacy in improving physician performance, including multiple teaching strategies, interactive techniques (role play, peer discussion), and reinforcing methods (follow-up contact about change implementation).^25,26 In addition, the act of committing to change itself may have facilitated behavior change. In controlled studies, participants who wrote commitment statements were more likely to make practice changes after CME programs.^27–29 The potency of this technique may lie in the fact that participants commit to making the changes most important to their own professional responsibilities.

Preceptors in our program fully implemented only 32% of their clinical practice commitments. An analysis of the reported barriers suggests that the preceptors had limited opportunities to implement these changes, insofar as they did not see many women with a higher risk of breast cancer during the study period. Thus, this may reflect our choice of genetics for clinical content rather than the educational impact of the curriculum. It is possible that the implementation rate would have been higher for more commonly encountered clinical conditions.

Our study builds on prior work in faculty development. Three other investigators reported workshops that combined clinical content and educational process. Mandel et al. described a series of workshops for community-based obstetrics and gynecology preceptors.³⁰ Each workshop included a summary of student performance and ratings of the faculty, an exercise in clinical teaching or curriculum development, and a clinical update. This format lacked true integration of clinical teaching and clinical content, however, because the segments were presented sequentially. Evaluation was limited to participant satisfaction.

Nieman's goal was to “provide a framework within which preceptors can become effective and competent teachers, who have the ability to apply recent clinical information during the training of medical students.”¹² In one of the workshops, participants brainstormed about strategies to efficiently integrate an inexperienced medical student into a sample office schedule that included 2 patients with diabetes mellitus. This exercise was followed by an update on diabetic care, including developments in drug regimens and foot care relevant to the 2 patients appearing in the precepting segment. The workshop facilitators also offered specific training in how students can assist in diabetic foot care. Beyond participant satisfaction ratings, the evaluation of this program was limited to a brief follow-up phone survey of 30 preceptors. Most recalled the workshop's content but few subsequently consulted the written materials.

Swagerty et al.¹³ developed an integrated faculty development program for community-based preceptors in a medical student ambulatory clerkship. They sought to improve their geriatric medicine and clinical teaching skills, recognizing the faculty shortage and limited patient exposure at academic medical centers. The program involved nine 2-part evening sessions. The first part focused on a geriatrics clinical content area and the second on a clinical teaching skill. They described an extensive needs analysis and recruitment strategy for this program, but provided little detail of the actual instructional strategies used in the workshops. Thus, it is difficult to determine the level of clinical content/clinical teaching integration. Their evaluation was limited to participant retention, attendance, and satisfaction with the curriculum, all of which were high. In addition, 88% of the participants indicated that they would change “some aspect” of their clinical practice or teaching in geriatrics.

Our work extends these efforts by achieving a higher level of clinical teaching/clinical content integration. We addressed ambulatory precepting and primary care genetics simultaneously through each phase of the workshop. Furthermore, we performed a more rigorous evaluation, including documentation of changes in skills and behaviors.

Our evaluation of skills was limited to participant self-evaluation, which lacks the objectivity of more rigorous measures, such as direct observation of teaching or evaluation by trainees. However, we sought to enhance the validity of self-evaluation by employing a retrospective pre-post strategy. By simultaneously assessing their pre and post skills, participants can judge their abilities against a constant internal standard. Traditional prospective pre-to-post self-evaluation, in contrast, remains vulnerable to response-shift bias, in which participants' internal standard changes as the as a result of the educational intervention itself.³¹ The intervention may increase their understanding of the dimensions being rated or their insight into their abilities within these dimensions. In several studies comparing the 2 methods, subjects' retrospective pre-post self-ratings were more consistent with independent assessments.^31–34

Retrospective pre-post evaluation remains susceptible to its own biases. Seeking social desirability, subjects might provide lower retrospective pre-ratings to ensure a positive effect of an educational intervention. However, in a faculty development study, participants' retrospective pre-post and traditional pre-to-post ratings differed only for skills and attitudes addressed by the workshops and not for items unrelated to the intervention.¹⁹ If participants were trying to please, one might expect the same effect on all items. Recall bias also might influence subjects' retrospective pre-ratings. Nonetheless, retrospective pre-ratings in 2 studies did not change when reassessed 1 and 2 weeks after an educational intervention, arguing against decay over a short period of time.^19,35

Although investigators frequently measure skills in their evaluation of faculty development programs,³⁶ few attempt to document behavior change, probably because to do so can be methodologically challenging, time consuming, and expensive. We chose a more practical “commitment to change” method. This strategy has been widely employed to evaluate the impact of clinical continuing medical education programs on physicians' practice behaviors ^{22,27–29,37–39} but, to our knowledge, has not been applied in faculty development workshops such as ours. In studies evaluated with this strategy, subjects, as a group, implemented 51% to 67% of total changes, ^22,37–39 and 47% to 93% of individual subjects implemented 1 or more of their own changes.^27,28,39 Participants in our program implemented changes with similar frequencies. This type of outcome assessment is limited by its reliance on self-reported behavioral change. Nonetheless, 1 study confirmed the validity of this methodology by comparing self-reported prescribing behavior to written prescription logs.⁴⁰

Educators seeking to adapt this curriculum locally⁴¹ will find that the model can accommodate other combinations of clinical teaching and clinical content areas. Nonetheless, several limitations should temper the interpretation of our results. Our evaluation strategy did not include a control group, although it is unlikely, given the short study period, that the changes in skills and behaviors are attributable to another concurrent “intervention.” In addition, our outcome measures relied on self-reports rather than more objective metrics. However, as above, we used previously tested evaluation strategies with established validity. And finally, although our participants represented a diverse spectrum of ambulatory teachers, we studied faculty at a single institution who participated in a single program.

Effective preceptors must draw upon their clinical and pedagogical skills simultaneously. Our faculty development model integrated training in these areas to prepare preceptors for this challenging role. Participants consolidated their CME time, improved their self-reported skills, and made specific changes in their clinical practice and teaching behaviors. By eliminating an artificial educational boundary, we removed “the fatal disconnection of subjects which kills the vitality of our modern curriculum.”⁴²“There is only one subject-matter for education,” Alfred North Whitehead continued in 1929, “and that is Life in all its manifestations.”