Abstract
OBJECTIVE
Department of medicine chairs have a critical role in the promotion of clinician-educators. Our primary objective was to determine how chairs viewed: 1) the importance of specific areas of clinician-educator performance in promotion decisions; and 2) the importance and quality of information on available measures of performance. A secondary objective was to compare the views of department chairs with those of promotion and tenure committee chairs.
METHODS
In October 1997, a questionnaire was mailed to all department chairs in the United States and Canada asking them to rate the importance of 11 areas of clinician-educators' performance in evaluating them for promotion. We also asked them to rate 36 measures of performance. We compared their responses to a similar 1996 survey administered to promotion committee chairs.
RESULTS
One hundred fourteen of 139 department chairs (82%) responded to the survey. When considering a clinician-educator for promotion, department chairs view teaching skills and clinical skills as the most important areas of performance, as did the promotion committee chairs. Of the measures used to evaluate teaching performance, teaching awards were considered most important and rated as a high-quality measure. When evaluating a clinician-educator's clinical skills, peer and trainee evaluation were considered as the most important measures of performance, but these were rated low in quality. Patient satisfaction and objective outcome measures also were viewed as important measures that needed improvement. Promotion committee chairs placed more emphasis on productivity in publications and external grant support when compared to department chairs.
CONCLUSION
It is reassuring that both department chairs and promotion committee chairs value teaching skills and clinical skills as the most important areas of a clinician-educator's performance when evaluating for promotion. However, differences in opinion regarding the importance of several performance measures and the need for improved quality measures may represent barriers to the timely promotion of clinician-educators.
Keywords: promotion criteria, clinician-educators, academic advancement
The demand for clinician-educators in internal medicine and other primary care disciplines has increased because of the increased emphasis on teaching in primary care settings.1–6 However, medical schools have had difficulty recruiting and retaining experienced clinician-educators because promotion decisions often are based on productivity in research.7–12 Some medical schools have responded by developing academic tracks with promotion criteria that recognize and reward the diverse roles played by their faculty.6,11,13,14 Despite these changes, clinician-educators continue to express concerns about barriers to their academic advancement.5,15,16
A 1996 survey of medical school promotion and tenure committee chairs in the United States and Canada indicated that the many types of contributions made by clinician-educators are valued highly and considered frequently in promotion decisions.17 However, because key decisions regarding promotion are made at the departmental level before reaching the medical school's promotion committee, we believed it was important to determine whether department chairs share similar views.
Our specific objectives were: 1) to determine how department of medicine chairs viewed the importance of specific areas of a clinician-educator's performance in decisions regarding promotion to the level of Associate Professor; 2) to determine how they viewed the specific types of information that are typically available to measure a candidate's performance; and 3) to compare the views of department of medicine chairs with those of medical school promotion committee chairs.
METHODS
Study Population
The target group for this survey included department of medicine chairs at U.S. and Canadian medical schools whom we identified from lists provided by the American Association of Medical Colleges and the Association of Professors of Medicine. The same list of medical schools was used in the 1996 survey of promotion committee chairs (81% response rate). On the first page of the survey instrument, clinician-educators were defined as: “physicians whose primary responsibilities are patient care and education, and whose research represents only a minor portion of their academic contributions.”
Survey Content
The questionnaire was modeled after the instrument used in our previous survey of promotion committee chairs.16 The questions that were used to compare the responses of the department chairs in this survey with those of the promotion committee chairs were identical. In one section of the survey, chairs were asked to rate the importance of 11 major areas of a clinician-educator's performance using a 7-point scale (from 1, indicating minimally important to 7, indicating extremely important). These areas included teaching activities, curriculum development, clinical skills, role modeling and mentoring, research and other scholarly work, administrative abilities and reputation and personal qualities. In another section, chairs were asked to rate the importance of specific measures that can be used to assess each area of performance using the same 7-point scale. Department chairs also were asked to rate the quality of information that is usually available for each measure using a 5-point scale (from 1, indicating extremely poor quality to 5, indicating extremely high quality).
Survey Administration
After pilot testing, we mailed the questionnaire with a cover letter from the Society of General Internal Medicine's Task Force for the Clinician-Educator. The first mailing was in October 1997. We sent reminder postcards, facsimile reminders, and 2 repeat mailings to nonresponders.
Statistical Analysis
We conducted double data entry and reviewed each data set carefully to identify any errors. Frequency distributions for each question were examined to identify any irregularities in the data. Descriptive statistics were used to summarize responses to questions. We used analysis of variance with Bonferroni correction to determine the statistical significance of differences between the importance scores given to the 11 major areas of a clinician-educator's performance. Responses given by promotion committee chairs and department chairs from the same medical school were paired and importance scores compared using the Student paired t test. Paired data were available for 84 of the 139 medical schools in the United States and Canada (60%). Once responses were paired, we separated the data from any information that would specifically identify a department or school. The study protocol was approved by the Johns Hopkins Joint Committee on Clinical Investigation. Stata Statistical Software Release 5.0 (Stata Corp., College Station, Tex), and SPSS 7.5 Base for Windows 95 (SPSS Inc., Chicago, Ill) statistical packages were used for all analyses. Narrative comments were reviewed independently by 2 investigators to identify major themes. A third investigator reviewed comments and themes in order to resolve any discrepancies.
RESULTS
Characteristics of Responding Department Chairs and Their Departments
One hundred fourteen of 139 department chairs (82%) responded to the survey. Thirteen percent of responders were from Canada. Sixty-nine percent of departments reported having a clinician-educator track, 87% had a tenure track, and 64% reported that their clinician-educators were eligible for tenure.
Resources that were reported as being available to assist clinician-educators in achieving academic advancement included faculty development programs (68% of departments), financial support and time to attend professional meetings (66%), at least one-half day per week of protected time (52%), guidelines for compiling a teaching portfolio (51%), teaching effectiveness committees (18%), and graduate school courses in medical education (14%).
Sixty-nine percent of responding chairs provided clinician-educators with formal feedback related to their performance at least once a year, while this feedback was given every 2 to 3 years by 24% of departments, and less than once every 3 years by 4% of departments.
Importance of Specific Areas of Performance
Department chairs rated all of the areas of a clinician-educator's performance addressed in the survey as important in promotion decisions with mean importance ratings that were greater than the midpoint of the 1-to-7 scale (Table 1). Teaching skills and clinical skills were rated as the most important areas of performance. These areas also were reported as the most important by promotion committee chairs.17
Table 1.
Differences in the Importance of Specific Areas of a Clinician-Educator's Performance
| Mean Importance* | |||
|---|---|---|---|
| Performance Area | Department Chairs (N = 114) | Promotion Committee Chairs (N = 115) | Medical Schools with a Difference ≥2 Points Between the Two Chairs, % (N = 84) |
| Teaching skills | 6.23 | 6.38 | 18 |
| Overall clinical skills | 5.86 | 5.78 | 28 |
| Coordination of training programs | 5.72 | 5.20† | 39 |
| Other written scholarship | 5.52 | 5.26† | 28 |
| Design, implementation and evaluation of educational programs | 5.53 | 5.28 | 37 |
| Mentoring and role modeling | 5.31 | 5.63 | 43 |
| Personal qualities | 5.43 | 4.94† | 36 |
| Reputation | 5.31 | 5.49 | 37 |
| Research in medical education | 5.18 | 4.51† | 35 |
| Clinical research | 4.98 | 4.88 | 36 |
| Coordination of clinical practice | 4.93 | 4.72 | 48 |
Importance scale: 1 = minimally important, 7= extremely important.
P < .05 for the unpaired difference between the 114 department chairs and the 115 promotion committee chairs who responded.
Specific Measures Used to Assess Performance
Department chairs viewed teaching awards as the most important measure for assessing teaching activities (Table 2), with the highest mean importance rating (6.3 on the 1-to-7 scale). Teaching awards also had a high quality-of-information score (mean of 4.5 on the 1-to-5 quality scale).
Table 2.
Ratings of Importance and Quality of Information on Measures of Performance by Department Chairs and Promotion Committee Chairs
| Mean Importance* | |||
|---|---|---|---|
| Specific Measures of Evaluation | DC (N = 114) | PCC (N = 115) | Mean Quality, DC (N = 114) |
| Teaching activities | |||
| Teaching awards | 6.3 | 6.0 | 4.5 |
| Assessment by learners | 5.8 | 5.3† | 3.6 |
| Peer/colleague evaluation | 5.5 | 5.9† | 3.3 |
| Teaching portfolio | 5.5 | 5.4 | 3.4 |
| Curriculum development | |||
| Publication related to curriculum work | 5.6 | 5.2 | 4.3 |
| Presentation of curricula at a national meeting | 5.5 | 4.5† | 4.0 |
| Clinical skills | |||
| Peer evaluation | 5.5 | 5.8 | 3.2 |
| Trainee evaluation | 5.4 | 4.8† | 3.4 |
| Measurement of patient satisfaction | 4.9 | 2.7† | 2.7 |
| Objective process and outcome measures | 4.6 | 3.4† | 2.2 |
| Number of patients seen per month | 3.9 | 2.3† | 4.0 |
| Income generated from clinical practice | 3.3 | 1.5† | 4.1 |
| Role modeling and mentoring | |||
| Input from trainees | 5.9 | 4.9† | 3.7 |
| Input from outside institution | 4.6 | 5.4† | 2.9 |
| Research and other scholarly work | |||
| The journal in which the publication appeared | 5.5 | 5.9† | 4.6 |
| Number of peer-reviewed publications | 5.3 | 5.7 | 4.7 |
| Presentation of research work at a national meeting | 5.2 | 4.2† | 4.2 |
| External grant support | 5.0 | 6.0† | 4.6 |
| Impact of publication | 4.5 | 3.7† | 3.8 |
| Administrative abilities | |||
| Success in the administration of a training program | 5.6 | 4.5† | 4.0 |
| Success in the administration of clinical services/practice | 5.5 | 4.8† | 3.7 |
| Involvement in institutional committees | 5.1 | 5.2 | 4.2 |
| Personal qualities and reputation | |||
| Ethical conduct/behavior | 6.3 | 5.7† | 3.5 |
| Leadership qualities | 6.2 | 5.5† | 3.8 |
| Enthusiasm | 5.9 | 4.2† | 3.8 |
| Invited lectures or presentations | 5.4 | 5.6 | 4.1 |
DC, department chair; PCC, promotion committee chair
Importance scale: 1 = minimally important, 7= extremely important. Quality-of-information scale: 1 = extremely poor quality, 5 = extremely high quality.
P < .05 for the unpaired difference between the 114 department chairs and the 115 promotion committee chairs who responded.
Two measures used to assess performance in the development of curricula were rated of high importance (mean importance scores greater than 5), including publications related to curriculum work, and presentation of curricula at national meetings.
Department chairs view peer and trainee evaluation as the most important measures for assessing a clinician-educator's clinical skills, while the number of patients seen per month and income generated from clinical practice were considered least important. However, the quality-of-information scores for these less-important measures were higher than for any of the other measures assessing clinical performance.
Although the department chairs were most interested in input from trainees within the institution for the assessment of a clinician-educator's performance in mentoring and role modeling, the promotion committee chairs appeared to value more the opinions of others outside the institution.
With respect to assessing a clinician-educator's performance in research, the journal in which publications appeared, number of peer-reviewed publications, presentation of research work at a national meeting, and external grant support were considered most important. The quality of this objective information was rated very high.
Success in administration of training programs or clinical services was felt to be very important in the assessment of administrative abilities. Other factors related to a faculty member's reputation and personal qualities were considered also in promotion-related decisions.
Differences in Opinion Between Department Chairs and Promotion Committee Chairs
There was a statistically significant difference between promotion committee chairs and department of medicine chairs in their ratings of the importance of 4 of the 11 areas of performance: coordination of training programs, conducting education-related research, personal qualities, and other written scholarship (book chapters, review articles, editorials).
In comparing the importance scores assigned by the department chair and promotion committee chair from the same medical school (N = 84), the chairs' valuations of the 11 performance areas that relate to promotion differed by 2 points or greater at 18% to 48% of the medical schools (Table 1). Although one might hope that opinion leaders from a given school would appraise these factors similarly, statistically significant differences in importance scores between the 2 chairs for these variables that relate to the promotion of clinician-educators were noted in the following areas: coordination of a training program, conducting education related-research, and personal qualities (all P < .05).
There was a statistically significant difference between department chairs and promotion committee chairs in the importance ratings assigned to 19 of the 36 measures used to evaluate the performance of a clinician-educator (Table 2). Department chairs assigned higher importance scores than did promotion committee chairs for 16 of these 20 measures of performance. Of the measures used to assess clinical skills, there were significant differences in the importance scores assigned by the 2 chairs for 5 of the 6 measures, all of which were valued more by department chairs. Only 4 measures were valued significantly more by promotion committee chairs: input from outside home institution, peer evaluation of teaching, external grant support, and journal in which publications appeared.
Recommendations for Successful Achievement of Promotion
Expectations for Publication
The minimum number of peer-reviewed publications that department chairs expect from a candidate for promotion to the rank of Associate Professor was half as many for clinician-educators as compared to clinician-investigators (mean of 5 vs 10, P < .001). In addition, 35% of responders reported that there was no minimum number of publications expected of clinician-educators.
Specific Recommendations by Chairs
When department of medicine chairs were asked to “offer one specific recommendation to facilitate the timely promotion of a newly hired Assistant Professor in a clinician-educator track,” narrative comments were provided by 92 (81%) of the respondents. The comments were categorized into 8 main themes that can be summarized as follows: 1) document and track all activities, including objective measures of teaching or clinical success; 2) achieve a reputation for excellence; 3) publish all scholarly activity; 4) with the help of a mentor, set and meet goals and objectives; 5) develop an area of expertise or focus for your work; 6) be involved in research; 7) be actively involved in getting yourself promoted; and 8) develop curricula or other educational projects.
DISCUSSION
This national survey of department of medicine chairs solicited opinions about the major factors used to assess performance of clinician-educators in making promotion-related decisions, the quality of specific measures used to evaluate candidates, and the congruence of opinion between the department chairs and the promotion committees (by highlighting the discrepancies in opinion between department chairs and promotion committee chairs). Clinician-educators in medicine departments should be reassured that their many contributions are recognized and regarded as important in promotion decisions by both their department chairs and medical school promotion committee chairs. Because clinician-educators devote most of their time and efforts to patient care and teaching, it is encouraging that these areas of performance are rated of highest importance by both department chairs and medical school promotion committee chairs. If the contributions of clinician-educators are recognized and valued, what then is leading to their perception of barriers to promotion?
The results of our study demonstrate that significant discrepancies in opinion exist between the department chairs and promotion committee chairs at many institutions regarding the promotion of clinician-educators. Educational activities and research related to education appear to be valued more by department chairs than by promotion committee chairs. Medical school promotion committee chairs may undervalue research in medical education because of their perceptions about the rigor of the research methods, a relative absence of funding opportunities, or a less-direct link of this work to patient outcomes, or for other reasons. It is also noteworthy that the department chairs were significantly more appreciative than were the promotion committee chairs about the measures related to clinical productivity. This may very well be because of the department chairs' concern and preoccupation with budgets and the fiscal health of the department.
Our study also suggests that there is much room for improvement in the quality of the measures used to assess a clinician-educator's performance. Of particular interest are those specific measures considered by department chairs to be of high importance, but for which quality measures do not exist. For example, for 4 of the measures felt to be most important in assessing clinical skills, (peer evaluation, trainee evaluation, patient satisfaction, and outcome measures of clinical practice), the information available to department chairs is not believed to be of high quality. Measures of teaching skills, such as teaching portfolios, evaluation by peers, and evaluation by learners also were reported to be suboptimal. Given that teaching skills and clinical skills represent the 2 areas of performance considered most important in assessing a clinician-educator for promotion, it will be important to improve the quality of measures used in their assessment. Although the written comments of department chairs emphasized the need for documentation of teaching activities, it was surprising to learn that the overall importance score assigned to teaching portfolios was not higher. The expectations regarding the number of publications necessary for promotion to the rank of Associate Professor were significantly less for clinician-educators than for clinician-investigators. However, the department chairs' narrative comments indicate that publications provide important measures of academic achievement. The need to document and, whenever possible, publish all scholarly activity is especially important for clinician-educators. The development of special expertise in a focused area of clinical practice and teaching is highly desirable because it enhances the clinician-educator's ability to produce works of scholarship.
It is promising that many departments of medicine have several mechanisms in place to assist clinician-educators in achieving academic advancement, including faculty development programs, formal mentoring, financial support, and protected time. In addition, there appears to be genuine effort to give faculty formal feedback on how to achieve promotion. Still, only half of the responding departments reported protecting a minimal amount of time for their faculty's professional development, which is likely because it has become increasingly difficult to maintain, given the increased demand for clinical productivity.18 Future studies may wish to draw directly on the perspectives of clinician-educators themselves, as they attempt to rise in the system and attain promotion in a climate that appears to be more attuned to and rewarding of the accomplishments of the research faculty.
Several limitations should be considered. First, the data collection was by self-report and may not represent the actual practices of department chairs. Respondents were assured that all data would be kept anonymous and only reported in aggregate. We are hopeful that this resulted in truthful responses as opposed to those that would be considered socially desirable. A second limitation of this study may be that the elapsed time between the 2 surveys could have accounted for some of the differences seen between the department chairs and promotion committee chairs. We believe that a temporal bias is unlikely, given the relatively short difference in time frames (between the 2 studies) and the historically slow pace at which policy changes occur in academic medicine. A third limitation may be in the interpretation of quality-of-information scores. It is unclear whether low scores represent actual poor quality of information or the lack of availability of such data. Finally, we may not have captured all of the areas of a clinician-educator's performance considered important or all of the measures used to assess performance. However, this is unlikely to be a major limitation because we asked the chairs to add additional areas and very few were added.
We conclude that uniform promotion criteria, such as those developed by the Society of General Internal Medicine,19 need to be developed and applied both at the department level and the level of the promotion committee in order to facilitate the timely promotion of clinician-educators. Clinician-educators, with the support of their department chairs, need to take an active role in developing and improving the measures used to assess their performance, in terms of both the quantity and the accuracy of information. This is particularly relevant for the areas that clinician-educators are most passionate about, teaching and clinical skills.
Acknowledgments
The authors would like to thank the Society of General Internal Medicine's Task Force for the Clinician-Educator and Drs. Nicole Lurie and William Branch for their support of the survey.
Dr. Wright is an Arnold P. Gold Foundation Associate Professor of Medicine.
REFERENCES
- 1.Levinson W, Branch WT, Kroenke K. Clinician-educators in academic medical centers: a two-part challenge. Ann Intern Med. 1998;129:59–64. doi: 10.7326/0003-4819-129-1-199807010-00013. [DOI] [PubMed] [Google Scholar]
- 2.Osborne LM, Sostok M, Castellano PZ, Blount W, Branch WT. Recruiting and retaining clinician-educators: lessons learned from three programs. J Gen Intern Med. 1997;12(suppl 2):83–9. doi: 10.1046/j.1525-1497.12.s2.12.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Jacobs MB, Tower D. Enhancing the training of internal medicine residents at Stanford by establishing a model group practice and raising its clinician-educators' status. Acad Med. 1992;67:623–30. doi: 10.1097/00001888-199210000-00001. [DOI] [PubMed] [Google Scholar]
- 4.Friedman RH, Pozen JT, Rosencrans AL, Eisenberg JM, Gertman PM. General internal medicine units in academic medical centers: their emergence and functions. Ann Intern Med. 1982;96:233–8. doi: 10.7326/0003-4819-96-2-233. [DOI] [PubMed] [Google Scholar]
- 5.Cantor JC, Cohen AB, Barker DC, Shuster AL, Reynolds RC. Medical educator's view on medical education reform. JAMA. 1991;265:1002–6. [PubMed] [Google Scholar]
- 6.Levinson W, Rubenstein A. Mission critical—integrating clinician-educators into academic medical centers. N Engl J Med. 1999;341:840–4. doi: 10.1056/NEJM199909093411111. [DOI] [PubMed] [Google Scholar]
- 7.Batshaw ML, Plotnick LP, Petty BG, Woolf PK, Mellitis ED. Academic promotion at a medical school. Experience at Johns Hopkins University School of Medicine. N Engl J Med. 1988;318:741–7. doi: 10.1056/NEJM198803243181204. [DOI] [PubMed] [Google Scholar]
- 8.Barondes JA. The academic health center and the public agenda: whose three-legged stool? Ann Intern Med. 1991;115:962–7. doi: 10.7326/0003-4819-115-12-962. [DOI] [PubMed] [Google Scholar]
- 9.Maudsley RF. Faculty roles and academic promotion. Annals R Coll Physicians Surg Can. 1996;29:298–301. doi: 10.1097/00001888-199606000-00002. [DOI] [PubMed] [Google Scholar]
- 10.Carey RM, Wheby MS, Reynolds RE. Evaluating faculty clinical excellence in the academic health sciences center. Acad Med. 1993;68:813–7. doi: 10.1097/00001888-199311000-00001. [DOI] [PubMed] [Google Scholar]
- 11.Bickel J. The changing faces of promotion and tenure at U.S. Medical Schools. Acad Med. 1991;66:249–56. doi: 10.1097/00001888-199105000-00002. [DOI] [PubMed] [Google Scholar]
- 12.Stossel TP. Volume: papers and academic promotion. Ann Intern Med. 1991;106:146–8. doi: 10.7326/0003-4819-106-1-146. [DOI] [PubMed] [Google Scholar]
- 13.Jones RF, Gold JS. Faculty appointment and tenure policies in medical schools: a 1997 status report. Acad Med. 1998;73:211–9. doi: 10.1097/00001888-199802000-00023. [DOI] [PubMed] [Google Scholar]
- 14.Greer DS. Faculty rewards for the generalist clinician-teacher. J Gen Intern Med. 1990;5(suppl 1):53–8. doi: 10.1007/BF02600438. [DOI] [PubMed] [Google Scholar]
- 15.Baldwin CD, Levine HG, McCormick DP. Meeting the faculty development needs of generalist physicians in academia. Acad Med. 1995;70(suppl 1):97–103. doi: 10.1097/00001888-199501000-00032. [DOI] [PubMed] [Google Scholar]
- 16.Jones RF, Froom JD. Faculty and administration views of problems in faculty evaluation. Acad Med. 1994;69:476–83. doi: 10.1097/00001888-199406000-00015. [DOI] [PubMed] [Google Scholar]
- 17.Beasley BW, Wright SM, Cofrancesco J, Babbott SF, Thomas PA, Bass EB. Promotion criteria for clinician-educators in the United States and Canada. JAMA. 1997;278:723–28. [PubMed] [Google Scholar]
- 18.Sheffield JVL, Wipf JE, Buchwald D. Work activities of clinician-educators. J Gen Intern Med. 1998;13:406–9. doi: 10.1046/j.1525-1497.1998.00120.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Lubitz RM. Guidelines for promotion of clinician-teachers. J Gen Intern Med. 1997;12(suppl 2):71–7. doi: 10.1046/j.1525-1497.12.s2.10.x. [DOI] [PMC free article] [PubMed] [Google Scholar]