Understanding Clinical Teachers: Looking Beyond Assessment Scores

Good clinical teachers have a profound impact on physicians’ careers. Effective teachers influence medical students’ decisions to enter Internal Medicine.^1,2 Our own teachers left lasting impressions: we still hear their encouraging words when interacting with learners. Indeed, doctor means teacher,³ and physicians promise to pass their knowledge of medicine to the next generation.⁴

Clinical teachers are often regarded as the best doctors, but they are less commonly the most productive researchers; few can be a jack-of-all-trades. Therefore, the standard measure of clinical teachers’ achievements is learner’s assessments, not publications.^5,6 Surveys of US department and promotion committee chairpersons have revealed that learners’ assessments are among the most important criteria for advancing clinician educators.^5,6 Consequently, learner-of-teacher assessments must be trustworthy.

In earlier issues of JGIM, we summarized the validity of published clinical teaching assessments,^7,8 most of which involved internal medicine faculty. Among the 14 identified domains of teaching, interpersonal skills and clinical teaching were most common. Internal consistency was reported more often than interrater and test-retest reliability. Remarkably, only a minority of teaching assessments were based on educational frameworks or previous literature. We thus encouraged researchers to expand upon previous work when developing teaching assessments and to report measures of reliability.

We also characterized published teaching assessments according to Messick, who said that construct validity is based on content, internal structure, response process, criterion, and consequences evidence.⁹ Among published assessments, content and internal structure evidence was common, whereas other sources of validity evidence were generally neglected.⁸ We concluded that future research should increase the range of validity evidence to include response process, such as the impact of learners’ characteristics on their assessments of teachers, and criterion measures, such as associations between assessment scores and outcomes like teaching awards.

In this issue, Fluit et al. present the largest systematic review of clinical teaching assessments to date.¹⁰ A full text review of 155 abstracts yielded 32 instruments, which were analyzed for elements of good clinical teaching, Canadian Medical Educational Directives (CanMEDS) competencies, and validity evidence. The most common domains were teaching strategies, supporter role, role modelling, and feedback. Only one-third of the items reflected the CanMEDS competencies. Content validity was usually based on published literature and expert input, and less often on previous questionnaires or theories. Internal structure validity was typically determined by factor analysis and internal consistency. As in previous research,^7,8 other sources of validity evidence were scarce.

Fluit et al. concluded that no published instrument covered all relevant aspects of teaching. Underrepresented features were teachers’ roles as managers, communicators, and health advocates. Additionally, most instruments were limited to inpatient settings, internists, and measures of internal structure validity. Hence, the authors called for more comprehensive assessment methods, echoed requests for broader validity evidence,⁸ and expressed apprehension about using deficient instruments to make decisions regarding faculty promotion and tenure.

Fluit et al. raise vital concerns regarding the quality of existing instruments for measuring teaching effectiveness. Clinical teaching is complex, so it is unlikely that a single instrument could successfully measure teaching behaviors across diverse educational settings. Experts have therefore argued for multi-source assessments that incorporate peer and learner ratings, faculty development, educational service, and awards.^5,11,12 Moreover, we must distinguish between using teaching assessments for administrative decisions versus research. The purpose of research, which traditionally incorporates both quantitative and qualitative methods, is to reveal new knowledge about clinical teachers. For this reason, assessment scores will always serve to inform our understanding of clinical teachers.

What has research already demonstrated regarding the qualities of good clinical teachers? A systematic review of clinical teaching research found that interpersonal relationships between teachers and learners are important.¹³ Successful teaching behaviors include demonstrating enthusiasm,¹⁴ promoting dialogue,¹⁵ and spending extra time with learners.¹⁶ Clinical teachers should also know basic skills like asking questions, diagnosing learners, utilizing teaching frameworks, and giving meaningful feedback.¹⁵ Finally, master teachers seem to prefer facilitating discussions over transmitting knowledge,^17–19 and they promote learner growth by balancing challenge and support.²⁰

Despite the abundance of research on clinical teaching, a limitation of many previous studies has been the use of heterogeneous samples of teachers. To refine our understanding of clinical teachers, future research should focus on the very best (or worst) teachers, as determined by reputation,¹⁶ teaching awards, and assessment scores. Understandably, investigating narrowly defined populations of teachers may require multi-institutional study designs to generate adequate sample sizes. Additionally, we’ve observed that learners’ written comments provide tremendous insights about clinical teachers’ competence. While learners’ comments can be viewed as overly subjective, these comments are probably no more biased than assessment scores. Notwithstanding the rich qualitative research on clinical teachers, using learners’ written testimonials to separate average from excellent faculty members remains challenging. Ultimately, addressing all these limitations in future research will increase knowledge of clinical teaching, improve assessment methods, and strengthen our ability to reward academic faculty members.