Use of Fellow as Clinical Teacher (FACT) Curriculum for Teaching During Consultation: Effect on Subspecialty Fellow Teaching Skills

Abstract

Background

Subspecialty consultation in inpatient care is increasing. Teaching by subspecialty fellows in a consultation setting may be an important source of work-based learning for students and residents. However, teaching and evaluation of learners in this context may be challenging due to personal and systems-based barriers.

Objective

We developed and evaluated a framework designed to overcome barriers to teaching and to improve fellow teaching skills during inpatient consultation.

Methods

The PARTNER (Partner with resident, Assess the learner, Reinforce positives, Teaching objectives, New knowledge, Execute recommendations, Review) framework was delivered to rheumatology and pulmonary and critical care medicine fellows at 3 academic medical centers as part of a 2-session Fellow as Clinical Teacher (FACT) curriculum. Fellows' teaching skills were evaluated using an objective structured teaching exercise (OSTE) pre- and postcurriculum, and at the end of the academic year. Self-assessment surveys were used to evaluate fellows' self-perception of teaching skills.

Results

Twelve of 16 eligible fellows (75%) participated in the program and completed 73 OSTE cases. Teaching skills measured by OSTEs and self-assessment surveys improved after administration of the FACT curriculum. There was no significant skill decay at the end-of-year evaluation. The curriculum was rated highly, and 73% (8 of 11) of fellows stated they would teach more frequently as a result of the intervention.

Conclusions

The FACT curriculum was practical and feasible, and significantly improved fellows' teaching skills teaching during inpatient consultation.

What is known and gap

Teaching by fellows in the context of requested consultation could be a source of work-based learning for residents.

What is new

Evaluation of the impact of a 2-session Fellow as Clinical Teacher (FACT) curriculum using an objective structured teaching exercise.

Limitations

Small sample; assessment limited to teaching exercise, not on impact of actual clinical teaching.

Bottom line

The FACT curriculum was practical and feasible, and significantly improved fellows' teaching skills.

Introduction

Achieving an appropriate balance between service and education is a major challenge in residency education.¹ Because the role of consultation in inpatient care is increasing,^2,3 expanding work-based learning opportunities in this setting may be an important mechanism for maximizing resident education in the inpatient setting.⁴ In academic medical centers, consultation-related communication generally occurs between trainees. Therefore, addressing the effectiveness of resident-fellow teaching interactions is an important step in improving resident work-based learning.⁵ Teaching during consultation also may have a broader impact on patient care through increased collaboration and relationship building.^6,7

Centered on a consult question, the resident-fellow interaction creates a unique learning opportunity that engages many adult learning principles.⁸ However, fellows and residents face challenges to effective teaching and learning in the hospital setting, including time constraints, dissimilar team schedules and priorities, lack of a personal relationship, and different expectations for teaching.^4,9,10 We previously conducted a pilot study of a curriculum focused on improving fellows' teaching skills.¹¹ Here we describe a framework for improving fellows' teaching during inpatient consultation.

Methods

PARTNER Framework

The PARTNER (Partner with resident, Assess the learner, Reinforce positives, Teaching objectives, New knowledge, Execute recommendations, Review) framework for teaching during consultation was developed by the investigators with input from content experts (Table 1). The framework combines elements from existing teaching models,^12,13 adjusted to the challenges facing the resident-fellow interaction.^4,10,11,14

Table 1.

The PARTNER Framework

Fellow as Clinical Teacher Curriculum

The PARTNER framework was included in the Fellow as Clinical Teacher (FACT) curriculum, which consists of two 60-minute sessions. The first focuses on identifying and overcoming barriers to teaching residents in the inpatient consult setting, with suggestions elicited from participants and supplemented by findings from the literature.^4,14,15 The second session is devoted to skills relevant to teaching during consultation. Utilizing video examples, a discussion format, and role play, fellows were introduced to adult learning principles and the PARTNER framework.⁸

Setting and Participants

The FACT curriculum was administered to first- and second-year fellows in Massachusetts General Hospital (MGH) and Brigham and Women's Hospital rheumatology fellowship programs and first-year fellows in the MGH/Beth Israel Deaconess Medical Center pulmonary and critical care medicine program. Sessions were administered between August and October 2014, separated by 2 to 6 weeks. All fellows were invited to participate in the study.

Outcome Measures and Procedures

The primary outcome measure was fellows' performance on an objective structured teaching exercise (OSTE),¹⁶ a simulated encounter in which a teacher interacts with a standardized learner acting in a scripted role. Three investigators acted in the role of the standardized learner (E.M.M., K.D., J.I.M.). Standardized learners had no supervisory responsibility for participants. Fellows were given a scenario describing a consultation they were asked to perform by an intern on the primary medical team, and were instructed to deliver their recommendations and engage with the intern in a teaching interaction lasting up to 7 minutes (scenarios are available as online supplemental material). Fellows received the clinical topic 1 day before the OSTE to ensure medical knowledge was not a barrier to effective teaching. The OSTE scripts were composed by the authors, pilot tested with fellows not participating in the study, and revised from their input.

Fellows completed OSTEs prior to participating in the FACT curriculum, 2 to 6 weeks after completion of the curriculum, and at the end of the academic year. Precurriculum and postcurriculum OSTEs consisted of 3 stations completed in the same order, with the same clinical case, but using a different standardized learner: (1) an intern who admitted the patient and had good medical knowledge; (2) an intern who did not admit the patient (cross-covering intern), but with excellent medical knowledge; and (3) an intern who admitted the patient and had poor medical knowledge. We varied the standardized learner rather than the teaching format or clinical case, because we wanted to assess fellows' ability to teach in the setting where learner assessment and tailoring teaching to the learner were critical to an effective interaction. We limited the end-of-year OSTE to a single station (an intern who admitted the patient and had good medical knowledge), because interim analysis demonstrated scores on the 3 OSTE stations were highly correlated. The clinical scenarios differed for the 3 OSTE administrations to minimize testing effect of prior OSTEs on performance. Fellows received feedback after completing the postcurriculum OSTE, but not prior to that, to limit the effect of feedback on future performance.

OSTEs were video recorded and independently rated by 3 investigators. The standardized learner and an observer rated each OSTE in real time. A reviewer blinded to the timing of the OSTEs rated them using the video recordings. The rating instrument was adapted from a previous OSTE scale with some evidence of validity (provided as online supplemental material).¹⁶ All investigators underwent two 90-minute rater training sessions. Interrater reliability between pairs of raters (reviewer 1 versus reviewer 2) was moderate to high as measured by the Spearman correlation coefficient (0.74, P < .001; for blinded reviewer versus reviewer 1 [0.46, P < .001]; and blinded reviewer versus reviewer 2 [0.38, P < .001]).

Secondary outcomes included fellows' self-assessment of teaching skills and attitudes toward teaching measured by precurriculum, postcurriculum, and end-of-year surveys, and satisfaction with the curriculum (provided as online supplemental material). Surveys were adapted from a previously published instrument,¹⁷ and were previously utilized in the pilot study of the curriculum.¹¹

The study was approved by the Partners Institutional Review Board.

Statistical Analysis

OSTE scores and survey responses were linked to individual subjects and compared using matched pairs analysis with the Wilcoxon signed rank test and the Student's t test where appropriate. The mean score of all 3 raters was used in the primary outcome analysis. Interrater reliability was measured using the Spearman correlation coefficient.

Results

OSTE Performance

Twelve of 16 eligible fellows (75%) participated in the study (8 first- and 4 second-year fellows), and completed a total of 73 cases (Figure). All 12 fellows completed the precurriculum OSTE, and either the postcurriculum OSTE or the end-of-year OSTE. Fellows' OSTE performance improved after participating in the FACT curriculum (Table 2). Most significant gains were noted in assessing residents' learning needs and giving feedback to learners. End-of-year OSTE scores were similar to postcurriculum performance, suggesting that there was no significant skill decay. OSTE scores for second-year fellows were higher at baseline for 9 of 11 items, compared to first-year fellows, but were not significantly different after completion of the curriculum or at the end of the year for most measures. The exceptions were “evaluating synthesis” (P = .018), “managing time” (P = .035), and “relaying recommendation” (P = .024), which were higher for second-year fellows at the end of the year.

Figure.

Fellow Objective Structured Teaching Exercise (OSTE) Participation

Table 2.

Fellow Performance on the Objective Structured Teaching Exercise

Self-Assessment Survey

Following completion of the FACT curriculum, fellows reported more confidence in their teaching skills during consultation on 4 of 5 survey items (Table 3). There was no decay in teaching skills confidence at the end of the year. Fellows rated the curriculum highly (80% [8 of 10] as very good or excellent on a 5-point Likert scale), and 73% (8 of 11) stated that they would teach more frequently during consultation after participating in the FACT curriculum.

Table 3.

Fellow Self-Assessment of Teaching Skills

Discussion

Our study demonstrated that the FACT curriculum, utilizing the PARTNER framework, improved fellows' teaching skills in the setting of inpatient consultation as evaluated by OSTEs. In addition, the curriculum was highly rated by participants, and fellows reported that they would be more likely to teach following this intervention.

Our intervention differs from previously described “fellow as teacher” curricula^18–22 because it focuses specifically on skills important for teaching during inpatient consultation. Teaching in this setting differs from teaching experiences fellows may have had previously and carries several challenges. The hospital environment can make it difficult for residents and fellows to have in-person interactions; residents and fellows may have inaccurate perceptions of each others' goals and expectations; fellows often do not know the residents, making learner assessment challenging; and these teaching encounters must be conducted efficiently due to time constraints facing trainees on the wards.⁴ The FACT curriculum attempts to address these challenges. The first session focuses on overcoming barriers to setting up an in-person teaching interaction, and the second introduces the PARTNER framework, which is a structured approach for creating a learning contract, developing a positive learning environment, facilitating rapid learner assessment, and relaying recommendations in a time-efficient manner.

Improvement in fellows' skills, confidence, their likelihood of teaching, and the high ratings of the curriculum suggest that the FACT curriculum and the PARTNER framework may be effective. The greatest gains in OSTE scores were seen in aspects of teaching that are particularly challenging during consultations, such as assessing the learner and providing feedback. Fellows' self-assessment also improved, except their ability to assess the learner. One possible explanation for this finding is that learners may overestimate their skills prior to training programs.²³ This study builds on our pilot study by addressing the immediate impact of training, as well as retention of skills. The OSTEs included multiple stations, simulating learners of different ability and engagement, thereby increasing study validity.

The FACT curriculum is adaptable to other settings. It is time efficient, encompassing two 1-hour sessions, which were inserted without replacing other didactics. While we have not conducted faculty training previously, we anticipate that the curriculum could be administered by core faculty with approximately 1 hour of preparation, using a detailed faculty guide. Our materials and documents are transferrable to other specialties with an inpatient component. While some of the challenges addressed in this curriculum may be present in the outpatient setting as well, the resident-fellow interaction in this domain has not been explored and may be different.

Our study has several limitations. The sample size was small. However, the large effect size for this study and for our pilot¹¹ suggests that the intervention is effective. Utilizing investigators as OSTE raters may have introduced bias. We attempted to minimize bias by using 3 raters, including a blinded rater. The study lacked a control group, raising the possibility that the improvement in teaching skills may have been due to maturation effects. However, there were no significant differences between first- and second-year fellows on postcurriculum OSTEs, and our pilot study did not reveal differences in teaching skills between first- and second-year fellows at any time point. Finally, the study was not designed to assess the impact of the curriculum on resident-fellow interactions in actual inpatient settings or the effect on patient outcomes. These areas should be the focus of future study.

Conclusion

The FACT curriculum improved fellows' teaching skills as measured by OSTEs and was well received by the learners. This focused and time-efficient program is easily integrated into training programs and generalizable to any specialty that provides inpatient consultation.