An Association Between Paying Physician-teachers for their Teaching Efforts and an Improved Educational Experience for Learners

Abstract

Background

Medical schools often rely on faculty volunteerism to address clinical teaching needs for students. Increasing time pressures on physicians has made it difficult to secure commitments for clinical instruction. In the 2005–2006 academic year, the Johns Hopkins University School of Medicine (JHUSOM) launched the Colleges Program, recruiting 24 salary-supported physician-faculty to serve as advisors to students as well as teachers of the second year course, ‘clinical skills’. We hypothesized that compensating physician educators would have a measurable positive impact on the students’ experiences in this course.

Materials and Methods

Students’ assessments of paid colleges faculty (CF) preceptors from the 2005–2006 year were compared to those of volunteer preceptors from the two prior years (2003–2005 academic years) along six different teaching parameters linked to the course’s objectives. Multivariable regression analysis was used to identify the factors independently associated with higher preceptor scores.

Results

Fifty-eight preceptors taught clinical skills over the 3-year study period. The overall response rate for preceptor evaluations by medical learners was 77% (277/359). CF, more likely than volunteer preceptors to have a full-time academic appointment (100 vs 63%, p < .01), have an additional advanced degree (48 vs 15%, p < .01) and prior faculty development training (52 vs 17%, p < .01). Scores for all six evaluation domains were higher for CF compared to those from the two previous years combined (all p < .001). In the fully adjusted regression model, only CF status was independently associated with high preceptor evaluation scores (Odds Ratio 4.3, 95% CI 1.01–18.20).

Conclusions

Salary support for teaching efforts in the time-intensive CS course coupled with the prestige of being appointed to the CF was associated with higher student evaluations.

INTRODUCTION

Traditionally, medical schools have relied upon physician faculty members to donate their time to teach the medical students.¹ It was expected that physicians would give back to the profession and, in some cases, the institutions that trained them by educating the next generation of physicians (http://classics.mit.edu/Hippocrates/hippooath.html). Over the last decade, economic pressures have forced physicians to devote more time to revenue-generating activities with an emphasis on productivity—thus limiting the amount of time they are willing and able to volunteer for teaching responsibilities.^1,2,3 This change has made it difficult for medical schools to recruit adequate numbers of faculty to fulfill their educational missions.⁴

The financing of medical education is challenging, and institutions are attempting to identify ways to reward faculty for their teaching efforts.^2,5,6 These incentives have included monetary compensations to the ‘best’ teachers, complimentary continuing medical education courses, and establishing teaching academies to acknowledge the commitment to and evidence of educational excellence.^7,8 Whereas such incentives may have variable effectiveness in enlisting physicians to participate in educational programs, it is unclear whether these ‘recruited’ physicians are able to genuinely commit to the curriculum in the way that is required to bring about optimal teaching and learning. If educational outcomes were improved by these incentives, the mechanism of action might merely be that offering payment for teaching boosts the potential pool of teachers such that the most talented educators can be selected from a larger cohort of candidates.

In 2005, the Johns Hopkins University School of Medicine (JHUSOM) developed an advisory system and learning community named “The Colleges Program.” Faculty in this program perform two main functions: (1) serving as longitudinal advisors for medical students and (2) acting as clinical preceptors for the ‘clinical skills’ course. In return for this effort and commitment, faculty receive 19% salary support per year. We speculated that funding educators may have a positive impact on students’ educational experience in this course. To test this hypothesis, we compared the students’ ratings of the preceptors in the ‘clinical skills’ course for the 2005–2006 academic year with prior years when the preceptors received no salary support for their time.

MATERIALS AND METHODS

Clinical Skills Course and Colleges Program

At our institution, the ‘clinical skills’ course is taught in 3 quarters during the second year of the medical school curriculum. Four to five students are paired with a preceptor. The groups meet weekly for 4 hours. During this time, students receive didactic lectures about specific aspects of the history and physical exam. They then spend several weeks reviewing the history and physical exam with their preceptor before practicing on volunteer patients in the hospital. Students are required to write up their encounters in the traditional history and physical format and turn this in to their preceptor. They also present their patients orally. Students receive guidance and feedback on their clinical skills, their write-ups, and oral presentations.

Until 2005, ‘clinical skills’ preceptors have exclusively consisted of faculty who volunteered their time to teach the course. In return for their efforts, they were given free registration to the Medicine Review Course offered at JHUSOM each year. For the 2005–2006 academic year, the JHUSOM ‘Colleges Program’ provided salary support to the faculty teaching the clinical skills course. The selection process for the Colleges faculty included a written application, letters of support, and interviews with a selection committee.

During the initial year of the Colleges Program, no other substantive changes were made to the existing ‘clinical skills’ course. Further, because the 2005–2006 academic year was the first year of the Colleges, the second year students did not yet have an established relationship with their ‘clinical skills’ preceptors from their first year at JHUSOM.

The operating budget for the ‘Colleges Program’, which has assumed several major roles in the medical school curriculum including longitudinal student advising and the ‘clinical skills’ course is approximately $ 1.1 million annually.

Evaluation Method

At the end of the clinical skills course, students are asked to complete an evaluation of their preceptor and the course. Students are asked to rate their preceptors in the following 6 areas: (1) teaching history-taking, (2) teaching the physical exam, (3) helping them to establish rapport with patients, (4) feedback on write-ups, (5) feedback on oral presentations, and (6) the overall quality of the preceptor experience. Assessments were made using 10-point Likert scales (1 = poor, 10 = excellent). Students were given these evaluations at the time that they took their final clinical skills exam to encourage full participation. In addition, students who took the course in 2005–2006 were offered bonus points for submission of their evaluations. The students’ evaluations did not contain any identifiable information, and students were assured that data would only be shown to faculty members in aggregate.

Statistical Analysis

Descriptive statistics were used to summarize student ratings of their preceptors in each of the 6 questions on the evaluation. Evaluations from years 2003–2004 and 2004–2005 were combined and compared to evaluations from the year 2005–2006. The evaluation scores were skewed toward higher values and not normally distributed, therefore we used the Wilcoxon rank sum test to compare median scores for each question between the previous 2 years and the most recent year’s evaluations.

Because our hypothesis related to the effect of compensating preceptors for their time on the quality of teaching, we repeated our analyses excluding those faculty who had taught as volunteer preceptors in 2003–2004 or 2004–2005 and had also been selected as Colleges faculty in 2005–2006 (N = 7). We also compared the median scores for these 7 faculty preceptors in 2003–2004 and 2004–2005 vs 2005–2006 to determine if receiving compensation improved their evaluation scores.

Multivariable Analysis

Multivariable regression analysis was used to identify factors that were independently associated with higher preceptor evaluation scores. To create our outcome variable, we created a summary score for each preceptor that was equal to the average of all scores in the 6 evaluation areas. We then calculated measures of central tendency and dispersion for the mean scores of all preceptors. We designated all scores equal to or greater than the mean as “high evaluation scores.” The multiple logistic regression model was created in a user defined fashion using independent variables that we hypothesized might be associated with receiving higher evaluation scores for teaching clinical skills. These variables included: rank (as a proxy for years of teaching experience),^9,10 department/division,¹¹ previous experience teaching the ‘clinical skills’ course,¹² having received intensive training through the Johns Hopkins Faculty Development in Teaching Skills Program,^13–15 affiliation with the School of Medicine (full-time versus part-time faculty appointment), and Colleges faculty status. The information regarding the variables was obtained by utilizing the Johns Hopkins faculty directory, historical faculty rosters for the clinical skills course, and historical faculty rosters for the Johns Hopkins Faculty Development in Teaching Skills Program.

We performed a similar analysis with the mean summary evaluation score as a continuous variable using multiple linear regression. The findings in these 2 regressions were similar (data not shown). Data were analyzed using STATA 8.0 (STATA Corp., College Station, Tex). The study was approved by the Johns Hopkins Institutional Review Board.

RESULTS

Table 1 demonstrates the demographic characteristics of the 58 faculty preceptors who have taught in the ‘clinical skills’ course over the last 3 years. Colleges faculty were more likely than preceptors from the previous 2 years to have a full-time affiliation with the School of Medicine (100 vs 63%, p < .01), have an advanced degree (48 vs 15%, p < .01), and have been a participant in the Johns Hopkins University Faculty Development Program in Teaching Skills (52 vs 17%, p < .01).

Table 1.

Characteristics of Clinical Skills Preceptors, College Faculty Compared to Noncollege Faculty*

Characteristic (n, %)	College faculty (N = 23)	Noncollege faculty (N = 35)	P value
Female sex	8 (31)	11 (35)	0.79
Division of general internal medicine†	13 (57)	14 (39)	0.18
Rank
Fellow, instructor, assistant	17 (74)	29 (83)	0.41
Associate, full	6 (26)	6 (17)
SOM affiliation (full-time faculty)	23 (100)	22 (63)	0.004
Advanced degree (MD + PhD, MBA,MS or MPH)	11 (48)	5 (15)	0.005
Participant in FDP‡	12 (52)	6 (17)	0.005
Prior ‘clinical skills’ teaching experience	7 (30)	17 (49)	0.17

*College faculty represents preceptors from the 2005–2006 clinical skills course, and noncollege faculty is comprised of faculty from the 2003–2005 clinical skills courses and excludes faculty who would become College faculty.

†Whereas general internists made up a significant proportion of the teachers, three or fewer faculty preceptors were from each of the following specialties: pulmonary, gastroenterology, cardiology, endocrine, renal, geriatrics, oncology, psychiatry, emergency medicine, neurology, surgery, pediatrics, and anesthesia.

‡Faculty preceptor had been a participant in the Johns Hopkins Faculty Development Program in Teaching Skills.

Comparison of Evaluation Scores by Year

The response rates for preceptor evaluations were as follows: 2003–2004—69%, 2004–2005—72%, and 2005–2006—91%. Figure 1 shows a comparison of the median scores and interquartile ranges for each area of evaluation between the years 2005–2006 and the 2 previous years combined. For all questions, ratings for the most recent year were significantly higher (all p < .01). This finding did not change (all differences remained significant) when Colleges faculty (n = 7) were excluded from the analysis (all p < .01).

Figure 1.

Median evaluation score and interquartile range for clinical skills preceptors; 2003–2005 preceptors compared to 2005–2006 preceptors. Wilcoxon rank sum test for all comparisons, p < 0.01

The median scores for each question were then compared among these 7 Colleges faculty who had taught in both periods (without and then with funding for their time). No significant differences in the evaluation scores were noted between the years when these preceptors had been compensated for their teaching as Colleges faculty and the prior years when they were volunteer preceptors.

Multivariable Regression Analysis

Tables 2 and 3 show the unadjusted and adjusted odds of receiving a ‘high evaluation score’ for the 58 faculty preceptors who taught clinical skills during our study period. Only being a Colleges faculty was independently associated with a ‘high evaluation score’ (odds ratio 4.3, 95% CI 1.01–18.20).

Table 2.

Percentage of Preceptors Receiving A Composite High Score Evaluation by Characteristics Included in the Multivariate Model

Characteristic (n, %)	High score	P value
College faculty status
Yes	18 (78)	0.006
No	15 (42)
Rank
Associate or full professor	9 (75)	0.155
None, fellow, instructor, assistant professor	24 (52)
Department
General internal medicine	15 (56)	0.957
Other*	18 (56)
Clinical skills teaching experience
2 years or more	11 (46)	0.153
Less than 2 years	22 (65)
FDP participant†
Yes	11 (61)	0.664
No	22 (55)
SOM affiliation
PT	5 (38)	0.127
FT	28 (62)

*Three or fewer faculty preceptors were from each of the following specialties: pulmonary, gastroenterology, cardiology, endocrine, renal, geriatrics, oncology, psychiatry, emergency medicine, neurology, surgery, pediatrics, and anesthesia.

†Faculty preceptor had been a participant in the Johns Hopkins Faculty Development Program in Teaching Skills.

Table 3.

Factors Associated with High Preceptor Evaluation Scores

Characteristic	Adjusted OR (95% CI)
Colleges faculty	4.28 (1.01–18.20)
Rank*	2.07 (0.40–10.62)
General internal medicine†	0.87 (0.22–3.49)
‘Clinical skills’ teaching experience‡	1.69 (0.51–5.88)
FDP participant§	0.74 (0.14–3.77)
SOM Affiliation∥	1.24 (0.38–3.95)

*Rank of associate and full professor compared to no rank, fellow, instructor, and assistant professor

†General internal medicine compared to other Department of Medicine subspecialties, psychiatry, emergency medicine, neurology, surgery, pediatrics and anesthesia.

‡Two or more years previous clinical skills teaching experience compared to less than 2 years teaching experience

§Faculty preceptor had been a participant in the Johns Hopkins Faculty Development Program in Teaching Skills.

∥School of Medicine Affiliation, full-time compared to part-time

DISCUSSION

To our knowledge, this is the first study that has attempted to assess the impact of salary support to physician-teachers on a medical education outcome. The data shows that, in the initial year of the Colleges program, when faculty received financial support for teaching, preceptor evaluations were significantly higher for all parameters linked to the course’s objectives compared to the evaluations of the volunteer teachers from the previous 2 years. Of note, students rated the overall quality of the precepting as being significantly higher.

Medical student education has primarily been based on “fragile social contracts” between faculty and their institutions.¹⁶ In this model, academic physicians donate teaching time for personal fulfillment in knowing that they are “giving back” to the profession. The economic realities in medicine over the past decade have forced academic medical centers to demand increased productivity from their faculty.^17,18 In addition, physicians are now required to spend more time on documentation and utilization review.¹⁹ As a result, even physicians who continue to teach may be distracted and less committed than they have been in earlier times. Our data suggests that students may be able to perceive such variations in faculty teaching performance and, accordingly, the quality of the educational product.

For a small subgroup of physician-teachers who taught in this course both before and after salary support was available, their evaluations were not significantly different across these 2 periods. This finding suggests that the performance of individual physician teachers did not appear to improve by monetary compensation alone. As such, the overall improved performance by the ‘clinical skills’ faculty after the adoption of the Colleges program is largely attributable to the other teachers. The newly available salary support may have allowed physicians who were previously unable to volunteer 4 hours per week the opportunity to do so by “buying down” their other clinical, administrative, or research duties. The internal funding appears to have increased the pool of highly talented educators that became accessible. Further, the Colleges faculty may have felt more engaged and committed to this work because of: (1) the competitive selection process of the program, (2) the more clearly defined mission of the program, and (3) the financial compensation through salary support. Clinician educators are known to be most deeply motivated by helping others and far less so by extrinsic rewards.²⁰ The Colleges program, with its salary support for core faculty members, enabled specific educators to teach in this course and help learners grow professionally.^21–23

Several limitations of this study should be considered. First, we relied exclusively on medical students’ subjective assessments of the preceptors who taught them. It is also possible that students’ evaluations of this educational experience may have been augmented by knowing that their school has invested in their education. However, this seems unlikely given the lack of significant improvement in scores for the 7 faculty members who taught under both systems. Second, like all cross-sectional studies, our results describe associations between various factors and high preceptor evaluation scores, but causality cannot be determined. Third, the composite summary evaluation score for each preceptor was equal to the average of all evaluation scores. This method lacks established predictive validity, and there is no gold standard for assessing criterion-related validity. Nevertheless, the scale has face and content validity in that the 6 individual teaching aspects upon which learners assessed their physician—preceptors are fundamental elements of ‘clinical skills courses’ and are directly linked to the course objectives.^24–27 Fourth, the statistically significant improvement observed across all categories measured demonstrates that the overall educational experience for student has been enhanced; however, the scores were actually very high even before the Colleges system was put in place. Fifth, the offering of bonus points for completing the evaluations in 2005–2006 may have resulted in the higher response rate. Because students in each of the 3 years were assured that only collated evaluation results would be passed on to physician teachers, we do not believe that this incentive to complete the evaluation would have influenced responses. Lastly, because this study was conducted at a single institution, the results may not be generalizable.

We believe that this study represents a first step toward assessing the cost-effectiveness studies of medical education interventions and, more specifically, for paying educators to teach. Our study supports the supposition that providing monetary rewards, as well as the prestige of being appointed to the College Faculty, results in subjective improvements in precepting performance among ‘clinical skills’ course faculty. Whether this improvement leads to objective enhancement of medical students’ baseline clinical and communication skills requires further evaluation. The Colleges Program continues at our institution, and it is believed to be making substantive contributions to our learning community even before objective, confirmatory data are available. Whereas there are significant costs associated with such a program, remunerating and recognizing physicians for teaching these fundamental elements of medical practice, namely, physical examination and communication skills, clearly demonstrate the institution’s genuine commitment to clinical excellence and patient care.