The Medical Educator Teaching Portfolio

Abstract

A revolution in thinking and redefinition of traditional scholarship by prioritising teaching skills and achievements has led to the birth of the Teaching Portfolio, designed to carry a comprehensive and dynamic record of the teaching activities of the faculty. A teaching portfolio documents the faculty’s teaching scholarship and effectiveness. It is a record of selected information on one’s teaching achievements, skills and strategies and dynamically represents the faculty’s growth, progress and teaching record. The author suggests a template for the Teaching Portfolio of a Medical Educator, which consists of four parts: (1) evaluation, (2) personal professional development, (3) learning processes and (4) an appendix.

In recent years, the three pillars of academic medicine, Patient Care, Research and Teaching, have undergone a revolution resulting in many changes, chief among these being a potential equalisation between the three. There is now new weighting for the educational component in faculty promotion. Until recently, research carried a disproportionately higher credit than teaching or patient care. A corollary product of this revolution has been the emergence of new faculty positions such as Clinician-Educators in various medical schools. These new positions were created to retain valuable clinical teachers, thereby propagating medical education.1 In 1997, the Association of American Medical Colleges (AAMC) conducted a survey that showed nearly 75% of medical schools in the US have separate promotion tracks for faculty whose primary responsibilities lie in teaching and in patient care.2 These new tracks have aided medical education resulting in better recognition of the value of medical education, more faculty dedicating their lives to careers in academia, and increased numbers of innovations in medical education.3

Evaluation of the performance of these faculty has become an area of much debate. In consideration for faculty appointment and promotion, the classic criteria of scholarship in original research and discovery is now significantly modified. Schools are now relying more on an expanded definition of scholarship defined by Boyer, namely the scholarship of Integration, Application, and Teaching in addition to the classical scholarship of discovery.4 Through this expanded view of scholarship, medical schools have developed new criteria for evaluating faculty that serve as clinician-educators.3,5

Medical Schools have realized that a typical curriculum vitae (CV) as based on classical guidelines is heavily weighted towards the traditional definition of scholarship. Most of the space in a typical CV is devoted to the faculty’s accomplishment in research and service to the institution with minimal devotion to teaching efforts and achievement. Therefore, the revolution, incorporating the concept of a teaching portfolios, was born. This has now become synonymous with the evaluation of clinician-educators. The small section on education in the traditional CV has now matured to a separate teaching portfolio. When Harvard Medical School created their clinician-educator track in the late 1980s, they required the faculty to develop a teaching portfolio before they could be considered for promotion.3 McHugh, a former chairman of the professional promotion committee at John’s Hopkins Medical School, writes, “The major hindrance to promotion is vagueness about the career achievement of the candidate”.6 He continues to state that “the most problematic” faculty on the clinician-educator track are those who have not created a document that can show their accomplishments. A teaching portfolio is such a document, a collection meant to convey the quality and quantity of a faculty’s teaching effort, effectiveness and achievements. It is an example of what the 1997 ACME-TRI report meant, when it called for “administrative mechanisms to document teaching accomplishments”.7

There is no consensus as to what such a portfolio should contain. There are several diverse ideas in the literature. We set forth in this paper suggestions to create a holistic Medical Educator Teaching Portfolio (METP) that includes not only materials necessary for evaluation for faculty promotion, but also those to aid the educator in improving his/her own teaching skills through reflection and continued learning. Further, we discuss the potential uses of a METP in appointment and promotion as well as concerns related to such portfolios.

CONTENTS OF A TEACHING PORTFOLIO

We are proposing a template for a Teaching Portfolio for medical educators based on components of several models created at various medical schools and institutions of higher learning. The contents of this METP are based on an extensive review of the literature, including a 1996 survey conducted by Beasley and colleagues regarding what criteria medical schools and their faculty promotions and tenure committees use in the evaluation of clinician-educators for promotion.1,3,5–11 We also address the concerns of the critics of such portfolios as voiced in the literature or at national forums. We propose that an METP be divided into four parts: (1) Evaluation, (2) Personal Professional Development, (3) Learning Processes and (4) Appendix [Table 1].

Table 1.

Contents of a Medical Educator’s Teaching Portfolio

PART I EVALUATION

1. Teaching Awards received List of courses taught (hours, # and types of students, # of years taught) Formal presentations (lectures, CME, reviews, grand rounds) Invited lectures/seminars & type of audience Objective criteria (e.g. student scores) Evaluation summaries (e.g. by peers, students)

2. Clinical Awards received Objective measures (# of hours, # of students, # of patients,) Clinical Service with house staff (# months in attending/consulting-duty) Clinical leadership roles Preceptorship and a list of students Certification/re-certification examination results CME credits accumulated Activities that develop the doctors’ clinical skills Letters of recommendation from colleagues Evaluation summaries (by patients, nurses, students, residents, referring doctors, peers) Evaluation by department chairperson

3. Administrative National / International peer-reviewed awards Education committee involvement/position (& comments by the committee) Office held in academic societies (regional, national & international) Study groups membership (e.g. NIH & other granting agencies) Director of a thesis or dissertation Member of thesis or advisory committee for MS or PhD student Directorship of courses (for medical students, residents, fellows) Chairmen duties (task forces, societies) Association with student organization or student-faculty committee Visiting professorships Peer reviewed committee letter Recruitment of talent (by schools) Assistant/associate dean of education position held Role model/mentor responsibilities (students, residents, fellows, junior faculty) Evaluations from mentees, junior faculty, etc.

4. Educational development

Curriculum and syllabus Number developed Publications related to a curriculum/syllabus Curriculum innovations Presentation of a curriculum at a national meeting Number of learners exposed to the curriculum/usage statistics Evaluation of a curriculum (by students, peers, editorial review) Course syllabi evaluations (by learners) Journal publications Grant support Teaching materials published (and reference) Number of peer reviewed publications (and reference) Publication impact (determined via: citation Index)

Enduring educational material Book chapters Review articles Teaching aids for students and residents Patient education materials Videotapes CD ROMs Computer software Web-based teaching materials Lecture handouts Examinations / Tests Evaluations and reviews of materials produced

Other Service to or position on Editorial Boards Presentation of an abstract/poster/exhibit at a national meeting Development of a workshop for a national meeting Reviewer of medical education grants/papers/exhibits Examiner for Peers (e.g. American Board of Radiology, Royal Colleges) Medical education journal club Oral presentations

PART II PERSONAL PROFESSIONAL DEVELOPMENT Philosophy of education Long-term goals/objectives to attain them Results/outcome of examination/tests of mentees & students Positions achieved by students, mentees & trainees Self-assessment (strengths/weaknesses)

PART III LEARNING PROCESS Areas of needed professional growth Record of queries & answers; problems & solutions Methods of attaining growth Record of past growth Record of achievements CME classes/activities

PART IV Theappendix (Attachments) Tests/examinations Curriculum Lecture notes/handouts Examples of students’ work Evaluations from various sources Journal articles Enduring educational material

PART 1. EVALUATION

The Evaluation part of the METP is suggested to be divided into four major categories in accordance with the responsibilities of clinician-educators: (i) teaching, (ii) clinical, (iii) administrative, and (iv) educational development. These form the main body of the METP. This portion should contain CV-like entries, summaries of data, and brief narrative accounts.12 This should include teaching responsibilities and experience, evaluations by one’s students, mentees, audience and peers, and details of preceptorship and directorship of a thesis or dissertation and service to student/faculty committees. Where applicable, these entries should be succinct, yet supported by reference to the Appendix section that should contain concrete examples of the individual’s achievement, such as curriculum/syllabus development, lecture notes, tests prepared, and evaluations7 described above. For instance, a summary of student evaluations of the faculty can be included within the body of the METP, and then referenced to copies of the actual evaluations in the appendix. The body of the METP therefore provides a succinct review of the objective data found in the appendix, allowing reviewers fast access to a variety of information. Together, these components can be used to evaluate clinician-educators for new jobs, promotion, merit pay, teaching awards, and funding for projects/grants.10 They also have a variety of other potential uses due to their much greater detail and clarity than obtainable from a CV.

PART 2. PERSONAL PROFESSIONAL DEVELOPMENT

In this second part of the METP, the faculty member gets the opportunity to document the complete circle of his or her education, philosophy, goals/objectives, lectures/conferences, tests/examinations and results/outcome. For example, this part should include a personal statement prepared by the faculty regarding his/her philosophy on medical education, goals/objectives, and strengths and weaknesses7— an overview of his/her interest in medical education and the goals for future involvement in medical education and its importance in his/her career plans. Here the faculty can expand reflection on how his or her philosophy of teaching correlates with the objective contents available in the METP. The faculty can show relationship of lectures and talks delivered to results of tests and examinations and to final outcome. If the faculty is a member of thesis or advisory committeeS for MS or PhD, the outcome and successes of the candidate are recorded here. Preceptorship outcomeS are another good example Part II records.

By striving for consistency across all components of the METP (philosophy-goals-objectives-lectures-tests-outcomes), clinician-educators can help ensure the entire spectrum of their teaching is congruent. The METP needs to be objective and concise, with ample evidence that the philosophy is matched by the outcomes.

PART 3. THE LEARNING PROCESS

This part should include components pertaining to areas of professional growth specifically needed by the faculty; methods of attaining growth and a record of past growth.13 By documenting queries and their answers, problems and their solutions, etc., the faculty can add structure to the account of their journey of continuing education. This section can serve as an assessment tool for what needs to be learned by the faculty, a planning tool for how to go about this learning and a historical record of what has been learned13 as well as the path to this achievement. Such a process forces physicians to become active participants in their continuing education, rather than passive lecture-goers. Mathers and colleagues found that portfolios are an effective and efficient means of pursuing continuing medical education.14 In future, computer and telecommunication technology might allow physicians to fulfil CME requirements through self-directed learning documented in online learning portfolios.15 METP is an effective way of documenting and continuously updating all this.

COLLECTING DATA

The process of creating an METP is one that is best undertaken with the assistance of a mentor.10 This is particularly true for junior faculty, but helpful for all. Such a relationship can help ensure that the resulting METP is as objective as possible. Furthermore, such a partnership would help foster standardization across the Institution. As an alternative to mentorship, workshops could be conducted to introduce faculty to the process of creating an METP.3

The very process of assembling an METP fosters improved teaching through reflection on the contents.10 Seldin writes that the very act of creating a Portfolio causes faculty to reevaluate their teaching activities and change them to increase their teaching effectiveness.16 It acts as a tool of self-improvement for the medical educator. Beecher and colleagues determined that 100% of faculty in their study “reflected” on education as a result of composing a portfolio.17 Ideally, educators would improve their teaching efficacy by developing their METP via critical writing on a regular basis.

Meticulous care should be taken in acquisition of data for all the three parts of METP. There needs to be an accurate record of lectures/conferences given, workshops delivered or attended and also evaluations by students and audience. Evidence of quality of teaching is extremely important. In addition, it should contain a record of role as preceptor. This includes evaluations by students, mentees, peers, course director, department heads etc., as well as test scores, national and local. Evidence of quality of teaching is also provided by the progress of the medical students, residents and other students that the faculty taught or was a preceptor to, as well as any junior faculty mentored. In addition, the positions achieved by mentees and trainees of the faculty should be recorded. Innovation in teaching is often reflected by international, national and local recognitions and awards received.

DISCUSSION

Despite the merits of an METP, there are many detractors of such an assessment tool. A recurring theme highlights the subjective, rather than objective, nature of portfolios filled with “selected details” as opposed to hard data.5,10,18 There is obvious truth in this statement; however, there are several ways to increase the objectivity of an METP. First, the aforementioned mentor would serve to oversee the process and help ensure that the resulting METP reflects the clinician-educator’s entire teaching efforts.10 Secondly, the appendix should be filled with hard evidence in the form of evaluations, examples of teaching material created, and the students’ accomplishment of stated objectives (e.g.. scores on national board examinations, achievements of physicians/scientists mentored by the faculty,7 results of public education, etc. The evidence in the appendix should support the individuals stated philosophy on education, from the goals to the outcome with inclusion of all the intermediary steps.

In 1999, Pitts and colleagues conducted a small-scale study to determine the reliability of the assessment of portfolios by general practitioners.19 They found the assessors to have moderate consistency within their own assessments, but the reliability between assessors to be less than adequate. The most commonly expressed hindrance to their assessment was the “individuality” of the portfolios rendering them hard to compare. Others have reported concern over the assessment of “non-standardised” material and the labour-intensive nature of such an endeavour.18

We believe these factors can be solved in part by standardisation of the METP across an institution and eventually across the country. Each institution must create their own METP template. Harvard Medical School reports that “standardization of criteria [for teaching portfolios] did much to objectify and enhance confidence in the [clinician-educator] track”.3 Furthermore, as the numbers of faculty evaluated via portfolios increase, the promotion committees’ ability to assess them accurately, reliably, and expeditiously assess them will improve.

Further, Beasley and colleagues voice concerns over the use of an “indirect measure” (i.e. METP) replacing the more first-hand method of direct observation.8 We agree with such a concern and we do not believe that the METP can take the place of direct observation of classroom teaching or video review, but rather that it should be used as an adjunct in a complete assessment of all of the clinician-educator’s activities.

Some educators have voiced concerns over the criteria for promotion of clinician-educators. The concern refers to the traditional requirement of “research experience” and discovery, peer-reviewed articles and national or international reputation.8,9,20 In spending time developing original articles and cultivating a national reputation, clinician-educators are liable to neglect their primary responsibilities, namely, patient care and teaching.20 Despite the development of clinician-educator tracks and discussion of the need to use an expanded definition of scholarship in their evaluation, many schools continue to require peer reviewed original publications in reputable journals.2,3,8,20 We wish to raise awareness of this concern and ask the promotion committees to revisit, reevaluate and reconsider the activities that are most important for this faculty. It is indeed already happening in a few centres in the USA and Canada. In Canada, the teaching portfolio is referred to as “Educators Dossier”.21

CONCLUSION

The portfolio described is meant to be a complete account of a teacher’s thoughts, activities, and successes —a dynamic collection that is never finished, but rather actively supplemented. In its fullest capacity, a METP can be used for evaluation, reflection, continued learning and, of paramount importance, improved teaching. Compiled with the aid of a mentor and with a well-referenced appendix, the METP becomes a more objective measure of an educator’s value. Such an assessment tool is not meant to be used alone, but rather in conjunction with other established methods including the standard CV and direct observation where possible. As portfolios become standardized, their value and ease of use increases. Each Institution must create its own template that the faculty can use in the creation of his/her METP. We offer in this article an example that can be used as a step toward developing a standardized template for medical institutions. Potentially it could be a stimulant or a future model for an improved universally acceptable medical educator’s teaching portfolio.