Recognition of Teaching Excellence

Abstract

The 2008-2009 Task Force for the Recognition of Teaching Excellence was charged by the AACP Council of Faculties Leadership to examine teaching excellence by collecting best practices from colleges and schools of pharmacy, evaluating the literature to identify evidence-based criteria for excellent teaching, and recommending appropriate means to acknowledge and reward teaching excellence. This report defines teaching excellence and discusses a variety of ways to assess it, including student, alumni, peer, and self-assessment. The task force identifies important considerations that colleges and schools must address when establishing teaching recognition programs including the purpose, criteria, number and mix of awards, frequency, type of award, and method of nominating and determining awardees. The report concludes with recommendations for the academy to consider when establishing and revising teaching award programs.

INTRODUCTION

The 2008-2009 AACP Council of Faculties leadership identified the need for greater emphasis on and recognition of teaching excellence in our schools and colleges. It also wanted to prompt thorough exploration of evidence-based criteria and processes that can be used to recognize excellent teaching. To this end, a Task Force for the Recognition of Teaching Excellence was created and charged to:

• Identify methods for faculty development and encouragement of the Scholarship of Assessment, Learning and Teaching (SALT);

• Identify and collect “Best Practices” of methods to assess teaching excellence;

• Recommend means by which teaching excellence can be acknowledged and rewarded by the college/school and university; and

• Share the task force findings and recommendations via a manuscript or resource tool kit.

This white paper serves not only as the final report of this task force, but also as a call to action for colleges and schools of pharmacy to enhance their faculty development efforts in the area of teaching, and review their teaching award criteria and practices with an aim toward greater emphasis, clarity, and evidence-based models. The white paper explores the concept of teaching excellence, provides an extensive background on teaching awards, addresses each of the charges, and concludes with recommendations and next steps for the academy to continue this important work.

DEFINING TEACHING EXCELLENCE

There is no one definition of teaching excellence. It varies depending on a host of variables, such as who is defining it, the learners (eg, students vs. colleagues), subject matter, methods used, and many other factors. When focusing on the higher education of students, however, there are common characteristics found in the literature about good teachers:

• (1) Positive student-faculty contact

• interacts with students – gets to know them, in and out of the classroom

• helps students learn outside of class, is accessible in and out of class

• promotes cooperation among students

• gives prompt feedback

• (2) Effective active learning

• encourages students to be self-directed, independent, lifelong learners

• engages students in disciplinary thinking

• encourages higher-order thinking

• (3) Achievable, yet high expectations

• acknowledges student expectations and what students can expect from the teacher

• creates a safe yet challenging learning environment

• emphasizes time on task

• (4) Respects diverse talents and ways of learning

• demonstrates respect for students and their individuality/differences

• is fair and flexible

• (5) Effective communication skills

• demonstrates passion, enthusiasm, charisma

• offers something substantive to say and knows how to say it

• raises provocative and significant questions instead of just providing answers

• commands student attention and maintains it

• inspires/motivates students

• is compassionate, caring

• (6) Commitment to teaching well

• engages in activities to continue to develop teaching skills

• invites and accepts feedback to improve

• tries new techniques to promote learning

This list is meant to represent a variety of criteria for determining teaching excellence. It is important, however, for schools of pharmacy to discuss what criteria are important to them when creating/revising policies regarding teaching awards.

TEACHING AWARDS

Given that a primary mission of colleges and universities is education, or helping students to learn, improved performance by educators should contribute to increased learning. In fact, it has been posited that “the most meaningful measure of teaching is student learning.”^1(p.57) Attempts at meaningful measurement of effects of teaching on learning are rare, however, probably because of challenges related to the effort and precision involved, perceived threats to careers, and potential for introducing conflict among peers rather than collaboration.¹ Evaluation of teaching can be done effectively, however, when sensitivity and professional judgment are applied to the process.

To what extent is attention to teaching quality relevant to the average college professor's job performance? As identified in a review of literature on teaching awards in higher education, there is a common assumption that teaching awards foster a commitment to teaching improvement.² However, this is not necessarily the case. In an examination of existing research on college teaching, Ward noted the institutional reward system as one of the factors having only minimal influence on teaching improvement efforts; intrinsic motivation, teaching consultation services, and a teaching-positive institutional climate may be more influential for teacher improvement.³

The concept of recognizing and rewarding excellent teaching on a campus would suggest another potential benefit: retention of exemplary instructors. Indeed, Kalis and Kirschenbaum concluded that teaching awards do in fact enhance retention and morale.⁴ Those outcomes, however, may not be automatic, at least not for all disciplines. Aucott and colleagues found that awards for teaching excellence do not translate necessarily into faculty retention.⁵ Paradoxically, in their study, teaching award winners within departments of medicine had decreased longevity (the “kiss of death”), which was a suggested perception at other research institutions as well.⁶

Batshaw found that former medical educators at one institution perceived that teaching had the least influence on promotion decisions among 6 major factors, the others being (in descending order of importance) publications, grant support, research quality, clinical practice, and administration.⁷ Regarding recognition from colleagues or managers, the teaching awards did not tend to enhance an instructor's position, and in fact may not have been acknowledged at all by those peers.⁸ Menges suggests, however, that the most important purpose of institutional teaching award programs is “to raise the esteem in which teaching is held.”⁹ One UK center investigated the impact of teaching awards (that included a financial reward) on their campus, and found that (according to previous winners) the awards “widened opportunities” by allowing for publications and presentations, and that they “increased self-esteem.”^{8 (p.445)}

Recognition of teaching excellence in individual colleges and schools of pharmacy has been underway for some time. In a 1991 survey, 85% of pharmacy colleges and schools responding indicated that they had at least 1 school-specific teaching award¹⁰; the percentage increased to 89% in 2003¹¹ and to 92% in 2008.⁴ In fact, teaching awards were far more common than those for either research or service among pharmacy faculties, and a number of colleges and schools granted multiple annual teaching awards.⁴ Student voting was the most common method of recipient selection.^4,11 While many awards were the exclusive result of student voting (41% of responses in the Kalis and Kirschenbaum study), student input was deemed a factor in all, even those selected by administration or a faculty committee.¹⁰ The criteria on which the awards were based ranged from none that were explicit (often thought of as “popularity contests”) to a list of 19 specific factors.^10,11 Although some commonalities in criteria for award recipients existed among colleges/schools, the ways in which “excellence” in pharmacy teaching was defined obviously varied among those wishing to celebrate it. This celebration and recognition of excellence are important for fostering a climate in our academy that values teaching.

ASSESSING TEACHING EXCELLENCE

There are 3 primary sources of evidence/data used to measure teaching excellence (1) students/learners and alumni; (2) peers, administrators, and/or instructional specialists; and (3) the teacher himself/herself in the form of self-assessment and reflection. A teaching portfolio can serve as the data repository about a faculty member's teaching. To this end, the task force examined in detail the use of these data sources and the teaching portfolio for measurement and documentation of teaching excellence.

Data From Students and Alumni

The use of student evaluations to assess teaching effectiveness is a common if not widespread practice in higher education. The intent is to provide both formative feedback to improve individual courses and summative feedback to assess teaching effectiveness, which may in turn be used as evidence for decisions regarding tenure, promotion, and annual merit raises. Obtaining student feedback is relatively easy and is supported by a significant body of evidence as being credible and valid.^12-17 Yet many faculty members have doubts and experience apprehension regarding the use of student evaluations. The major concerns identified include: (1) the reliability and validity of the evaluation instrument used, (2) the consistency by which the evaluation is administered, (3) the ability of students to effectively evaluate teaching, and (4) the use of student evaluations as the sole or primary measurement of teaching effectiveness.¹⁷ Certainly, student ratings of teaching are based on their perceptions of good teaching and are influenced by many factors such as any existing perceptions about the teaching in a particular class or institution as well as their prior knowledge and experience with teaching.¹⁸ Overall, however, the quality of the validity and reliability of student ratings of teaching effectiveness outweigh the concerns. Nevertheless, student ratings should not be the only measure of teaching effectiveness.

Student Evaluations.

The following guidelines for implementing a student course evaluation system are based on review of the literature and extensive experience of task force members:

• Use an identical evaluation instrument for every course or every faculty member in the college, unit, or program. This will aid the institution in establishing benchmarks, documenting change and progress, and making meaningful comparisons.

• Provide guidance to faculty members for interpreting the results of an evaluation and subsequently improving skills or correcting deficiencies.

• Ensure that administrators understand the limitations and proper use of evaluation instruments.

• Ensure that students understand the importance and use of the evaluation instrument in faculty development and improvement. Training in the use of the instrument may be necessary.

• Follow a prescribed routine for administering the evaluation (eg, administered 2-3 weeks before the end of the semester; the instrument and a set of instructions are administered by a third party; the faculty member is not present during the evaluation and is not allowed to see the results until final grades are submitted).

• Ensure student participation and responsibility by instilling the sense that the information provided is important and will be used to improve teaching and learning. Making teaching evaluations a requirement tends to alter students' responses and raises the issue of maintaining students' anonymity.

• Set departmental and programmatic goals once the faculty and administration have experience with the evaluation instrument and the process for administering it.

• Establish guidelines for use of the evaluation instrument in team-taught courses (eg, evaluations will be administered within a certain timeframe of the faculty member completing his/her instruction; only faculty members completing a set minimum number of contact hours will be evaluated; a random sampling of students will be asked to complete evaluations for each instructor to avoid survey fatigue).

• Student evaluations should only constitute a portion of the weight for evaluating teaching effectiveness; other sources should include peer review and teaching portfolios.

Examples of student evaluation systems are found in Appendix 1.

When gathering data from students about teaching effectiveness, processes used in addition to traditional course evaluations are student interviews or focus groups, and alumni ratings. Student focus groups can be used to gather feedback about individual courses or the program at large. They can be voluntary or a randomly selected sample of students can be asked to participate randomly sampled from students. Focus groups can help to clarify written findings and comments from course and program evaluations. Data from focus groups also can be formative, such as suggestions on how to best gather student feedback.

Gathering opinions from alumni can serve as another important component in evaluating teaching effectiveness. Alumni can offer unique perspectives about programmatic issues, such as how prepared they felt for their careers and how prepared are the new graduates with whom they work; recent graduates can comment on the adequacy of their training as they began new positions and met real-life challenges. Alumni can comment on curricular structure, teaching and assessment methodology, and relevancy of and deficits in content.

Alumni Evaluations.

Many issues encountered in alumni evaluation systems are similar to those in students evaluation systems. When establishing a process by which to gather data from alumni:

• Establish an accurate database of alumni addresses to ensure a high response rate and a representative sample.

• Include demographic information (practice setting, postgraduate training, geographic location) in the survey instrument to stratify data if necessary.

• Identify a specific cohort or postgraduation period (alumni that graduated 1 year and 5 years prior from the time of the survey).

• Use valid and reliable instruments that will gather data of most importance to the institution.

Alumni data are critical for individual faculty members, since it has been demonstrated that alumni feedback about certain courses and instructors may have changed over time. For example, they may now appreciate a professor who challenged them, while as students they did not.¹⁹ Faculty members should have access to the findings from all evaluations.

Limitations of student and alumni ratings.

A single source of evidence is not the best method by which to document effective teaching. While student evaluations are a critical piece of information, there are limitations to this source of input that can be linked to the students' ability to evaluate effective teaching and/or the process and tools used to collect the data. Students may not be equipped to judge the currency or comprehensiveness of course content, the degree to which the instructor provides a balanced view of the subject matter, or the validity of procedures for assessing student achievement. In addition, student rating systems may have several important limitations including: (1) poorly constructed instruments; (2) administrative procedures that have not been standardized, thereby making comparisons among faculty difficult; and (3) lack of technical, statistical, and pedagogical support, thereby making interpretation of results and improvement of skills problematic.²⁰ For alumni ratings, the greatest barrier comes from achieving high enough response rates from a representative population to ensure validity of the data. There also may be some question, depending on how long the students have been alumni, as to how well they remember the course experience. While there are limitations to both student and alumni data, the benefits of its use in measuring teaching effectiveness certainly outweigh the risks.

Data From Peers

If teaching is a form of scholarship that promotes a thoughtful, critical-thinking, evidence-based approach, then it requires public scrutiny to elevate it to the level of scholarly activity. Lee Shulman of Stanford University has proposed that teaching is community property rather than a solitary activity. Therefore, teaching should be openly discussed and evaluated. It requires collaboration among faculty members to nurture new educators and to provide continuous quality improvement.²¹

Peer review must incorporate information from multiple sources, including materials provided by the instructor that convey both the strengths of his or her teaching as well as efforts to improve. Peer reviewers provide feedback to the instructor, and administrators determine the value of the contribution to the institution and provide data that will give the instructor a basis for comparison to other faculty members. Quality peer evaluation should occur over the course of a career rather than be limited to 1 or 2 isolated points in time. Peer evaluation can be formative or summative. Formative evaluation is informal, ongoing, and provides rich detail to the instructor who is evaluated, while summative evaluation is more general in nature and provides a basis for comparison with other instructors.

Establishing a peer evaluation system.

The process of establishing a peer evaluation system must be valued and supported by the institution. The process developed must be systematic, sound in theory and practice, manageable to implement, and well understood by faculty members. Important elements in the process include classroom observation, observer training, experience with the forms used, and review of course materials before the observation. In establishing a peer evaluation system, several decisions need to be made including how many classroom observations are essential, how many observers are needed, what other teaching materials should be reviewed, and what expertise is required of the reviewers. Other questions to be answered include whether the dates of the observations should be announced in advance and whether the instructor should be permitted to choose the dates of the observation.

There are several types of peer observation forms available for use or adaptation. The forms generally fall into a few categories: a checklist, a rating scale, open-ended written analysis, or some combination of these. Checklists focus the attention of the reviewer on specific behaviors but provide little detailed feedback. A rating scale adds the element of rating the effectiveness of each behavior, but adequate descriptions/definitions for each scale point must be provided. Written analysis provides rich feedback, but because the assessor chooses which behaviors to rate, the evaluation may focus on a limited number of behaviors and criteria. The best approach probably is a combination of methods.

Examples of peer assessment programs.

The first pharmacy school to describe a peer evaluation of teaching process was Shenandoah University, which developed a pilot process in 2004.²² The process was both voluntary and formative in nature. Only 25% of the faculty members participated as either an evaluator or a faculty member under review. The evaluators received training and a guide for possible areas to evaluate; however, a specific form for evaluation was not provided. The pilot process was evaluated and suggestions for improvement were solicited from participating faculty members. The process was well-received by those who participated.

The University of Colorado peer assessment program was constructed around 3 elements: mastery teaching, clinical supervision, and cognitive coaching.²³ The process incorporated 8 steps. The instructor's materials were reviewed in advance by the assessors. A pre-observation conference clarified and answered any reviewer questions. The observation took place, followed by the assessor analysis of the observation. A post-observation conference was held to provide feedback to the instructor. Re-observation could be scheduled if there was conflicting information or disagreement among the assessors. A report was prepared by the assessors and agreed upon with the instructor and a copy was provided to the department chair.

In the Colorado process, faculty members were reviewed during years 1, 3, and 5 pre-tenure and every 5 years following promotion. All assessors had to attend a training session. Two assessors were assigned to each instructor. An evaluation of the process demonstrated that the faculty members believed the process was positive, fostered collegiality and creativity, and improved teaching. Concerns of the faculty members included a lack of appreciation for individual teaching styles, lack of content experts for all faculty members, the time commitment required for the process, and how the assessments might be used for summative teaching evaluation.

Northeastern School of Pharmacy's annual mandatory, formative process incorporated self- and student assessment with the peer review.²⁴ They developed a peer observation and evaluation tool called POET. The process was evaluated for content validity, external validity, and interrater reliability. The process included a pre-observation meeting between instructor and reviewer(s) to review objectives, handouts, learning strategies, and teaching pedagogy. The classroom observation used the POET form, a 39-item rubric. A post-observation meeting occurred within a week of the classroom observation. During the meeting, reviewers provided positive feedback and constructive criticism focused around 3 main areas. A post-assessment meeting took place within 2 weeks, during which the assessment was reviewed and student achievement of learning outcomes was discussed.

A pilot peer assessment process was developed at the South Carolina College of Pharmacy-MUSC campus based on a “Best Practices in Teaching” document compiled by the faculty of the Department of Clinical Pharmacy and Outcomes Sciences.²⁵ An evaluation form focusing on 7 domains and using a Likert scale was developed. Each faculty member was evaluated by 3 peers and then asked to serve as a peer evaluator for 3 colleagues. Presentation materials and sample examination questions were supplied to the evaluators in advance. Immediate written and verbal feedback was supplied to the evaluated faculty member and discussion time was made available if needed. The process was formative in nature. An evaluation of the pilot process revealed that both faculty members who were evaluated and those who participated as peer evaluators felt the process enabled them to review their teaching practices, learn from peers, and exchange teaching philosophies. The department voted to continue the peer assessment process as a requirement with minor modifications.

Limitations of peer assessment.

Peer observations provide only a snapshot of a faculty member's skills and teaching style rather than an extensive evaluation. Also, the observation may be affected by the teaching philosophy of the assessor and/or the reputation of the instructor who is being reviewed. Other considerations include the relative scientific discipline of the instructor versus that of the peer assessor, and whether the peer is junior or senior to the instructor who is evaluated. Observers require training because most of them do not have experience in the process. Finally, faculty members tend to be generous in their ratings of colleagues when the process is a summative assessment that may impact promotion and tenure decisions.²⁶

Self-assessment and Reflection

The third major source of data for evaluating teaching is the self-assessment. The use of self-reflection as a teacher development strategy is a powerful tool, but only when the process includes 3 important characteristics. First, the self-reflection must be a description of development over time, including failures. Using the artist portfolio model of “best works” documents has not been shown to be effective for educational growth.²⁷ Second, the reflection process must be guided.^28-30 This means that the teacher's development, with all of its successes, failures, surprises, and disappointments, is discussed openly and honestly with peers and/or students. Third, teacher self-reflection must be conducted in a non-punitive environment.^28-32 It should not be used for advancement/merit pay; this inhibits honest reflection and without honest reflection this exercise has limited benefit. This does not mean that teachers cannot be rewarded for participating in a self-reflection program, but the criteria for receiving a reward must be the teacher fully engaging in the self-reflective aspects of the process, and not just documenting positive course evaluations and successful outcomes (grade performance, for example). Examples of useful practices for teacher self-evaluation are found in Appendix 2.

Teaching Portfolios

One tool for documenting teaching excellence is a teaching portfolio. Teaching portfolios are highly personalized products that document accomplishments and performance and include materials from the faculty member and others, as well as products of teaching/learning that allow for public and peer review of teaching. Teaching portfolios can be used for many purposes: self-improvement, annual department review, teaching awards, promotion and tenure, and merit pay decisions. They can be quite comprehensive, including lists and descriptions of teaching activities; evaluative data; artifacts of teaching materials and student work; and teaching philosophies, goals, reflections, and self-assessments.³¹ A primary benefit for faculty members of compiling teaching data into a portfolio is establishing a venue to reflect on and respond to its contents, such as student evaluation data. They also provide a more complete picture of teaching excellence, and invite peer review of teaching that promotes teaching as a scholarly activity.³³

REWARDING TEACHING EXCELLENCE

Some of the areas that should be considered in the creation/revision of a teaching recognition program are discussed.

Purpose of the Recognition

Establishing the purpose of the recognition is important because other decisions will flow from the purpose. Recognition most often serves to reward good work, as well as indicate to the students, faculty members, and administration that good teaching (and by proxy, student learning) is important. Formal recognition also may contribute to meeting tenure requirements, posttenure requirements, or promotion for those not in tenure-track faculty positions. Determining the purpose of an award is important since the criteria by which candidates will be judged may differ. For example, criteria and processes used to gather data about teaching for the purpose of promotion and/or tenure may differ from those used for an annual teaching award. The focus of this section of the paper is primarily on the latter.

Criteria for Teaching Excellence

Previous sections of this paper have described numerous criteria that can be used to determine teaching excellence. These criteria may differ depending on who is making the determination – for example, what students may consider to be teaching excellence may differ from faculty members' opinions. The determining body should set forth its criteria at the outset of the data collection and selection process so that those involved in the selection process are basing their decisions on the same set of criteria. This is not to say, for example, that all members of a selection committee should be required to use exactly the same detailed criteria when selecting candidates/recipients for an award as doing do may stifle flexibility and individual opinion, and likely lead to the same winners year after year. It is also important to let faculty know the criteria by which the decision is being made – this strategy may make the award selection process seem more credible and less like a popularity contest. It would serve as a clarification of the expectations that need to be met to earn the award.

Determining Award Recipients

While determining the winner or recipient of an award relates directly to the criteria being used in the selection process, decisions must also be made about the process of selection. Are the awards for just one individual or for a team of teachers? Will there be a judgment of the quality of the data used to meet the award criteria? If so, how will quality be determined? If the selection is done by a committee, who serves on the committee? Is there an anonymous vote taken by members or is there discussion of individual merits followed by a consensus decision? These details need to be articulated prior to the decision-making process.

Nominations and Recognition

The nominating body for the award and the selection body may be different. While there were no evidence-based recommendations found in the literature, Draugalis noted a myriad of options: student-nominated and selected award, peer-nominated and selected award, administration-nominated and selected award, alumni-nominated and selected award, or any combination of these.¹¹ Many colleges and schools of pharmacy use selection committees that are comprised of representatives from some if not all of these groups. However, if only one or a few awards are given, a broad representation of criteria and decision makers should be used. Consider alumni, peers, and student selectors.

Alumni awards.

For an alumni-based award, it is essential to use a carefully designed rubric for selection of awardees to lend validity to the process and make it seem less like a popularity contest. If the award is not entirely alumni-selected, then an alumnus could participate in a broader panel to determine teaching awardees. The alumni perspective should be an important contribution of promoting and maintaining teaching excellence.

Faculty nomination and selection.

Another approach to identifying and awarding teaching excellence is to canvas faculty members for their choice of the strongest candidates. Eligibility for awards at most colleges and schools is broad, with the only limitation being that candidates/recipients must have a full-time faculty appointment. Some colleges and schools impose a restriction on winning the award in consecutive years. The most common period of ineligibility for previous winners found in the literature was 1 year.⁴

Typically, teaching excellence award guidelines are set and posted to the faculty for nominations annually. The nominees are notified by their college faculty advisory council or department chairs.^9,11 At least 1 college/school empanels a student representative from the fourth year of the pharmacy program to the selection committee.³³ Each nominee is asked to provide a packet that supports the nomination. Each packet is reviewed and discussed by a small cohort of peers, plus or minus a student, and a selection is made. The clear advantage of a faculty advisory committee is that teaching peers make the selection based on a set of criteria rather than popularity. Often, the “award” is a one-time monetary award but also may be a salary boost, a plaque, faculty development funds, or recognition at a ceremony, with or without a meal.⁴

Student-driven awards acknowledging faculty teaching excellence.

Teaching excellence awards also may be selected by students. A student body or subset may create their own selection criteria and process to determine who should be awarded. If the process is not defined for faculty members, however, then it may seem more like a popularity contest than one that is evidence-based.

Number and Types of Awards

Unfortunately a thorough search of the literature did not yield any evidence-based answers to the number and types of awards given by colleges and schools of pharmacy. This identifies a need for research, ie, to determine how many and what types of awards have the most impact on student learning and teacher/preceptor satisfaction and retention. Regardless of the lack of evidence, individual programs will need to determine what is best for them. It may partially depend on the resources available and the size of the faculties. Strasburger notes that there are considerably more good teachers than poor ones and muses that each excellent teacher should receive an award.³⁴ He goes on to say that when only one faculty member's excellence is acknowledged, that acknowledgement actually “diminishes--purely unintentionally, mind you--the equally outstanding work that many, many others in a given field are doing.” Kalis and Kirschenbaum found that most of the schools gave a single teaching excellence award but between 5%-20% of the schools offered from 2 to 10 awards.⁴ Menges and Gastel suggest that any teaching acknowledgement should be representative to the types of teaching that occur within the program, including lecture, seminar, clinic, and laboratory.^9,35

Awards could be based on type of teaching, with criteria varying slightly for each if necessary. Consider the different styles, with lecture, laboratory/small group, distance education, course design or coordination, precepting, and others. Longevity is another consideration. Are there awards specific to length of time with the college/school or institution? Would this aid in retention? Consider schemes like 0-5 years, 5-10 years, 10+ years, and so on. A junior faculty award (“rising star” award; this might be similar to the “new practitioner” awards that several professional associations offer) and a senior faculty award could be awarded. Consider using similar criteria for didactic faculty members and preceptors.

Academic year awards are given to teachers who primarily teach students in a particular year of the program (eg, first year pharmacy [P1] students), so an award could be offered for each academic year to teachers (P1, P2, P3, P4) as well as to preceptors (for those precepting introductory pharmacy practice experiences and those precepting advanced pharmacy practice experiences).

Awards could come from a variety of sources: 1 from each department, 1 from each campus (for schools with more than one), 1 for the entire school, 1 from various types of practice sites for preceptors, 1 given to faculty members in health sciences (recognizing each program or competing against other programs), and 1 at the university level (which could be difficult to attain – especially for large and broadly represented universities). Awards could be based on innovations-type criteria, similar to AACP's annual competitions.³⁶ Preceptors could also be included. Awards could be based on significant improvements in teaching/precepting over time, or demonstration of improved student learning over time.

Frequency of Awards

The literature was devoid of evidence as to the most effective frequency of awards. Published literature does report frequency used by schools, with the most common awards being given annually. Consideration could be given to provide some awards less or more frequently. Some also have criteria by which the same person cannot be awarded the same award in consecutive years, for example. It seems intuitive that a valid awards program would be designed to recognize more than one individual repeatedly.

Substance/Type of Award

Are faculty members given a plaque, a lump sum of money, salary increase, or support to attend professional conferences? Does the plaque hang prominently in the school? Are the awards announced at graduation or another grand event? Are winners nominated for a higher level award, such as the university-level? Does he or she earn the privilege of chairing next year's award selection committee? Is he or she asked to discuss good teaching in a faculty or resident teaching development program? There are once again a myriad of different award schemes and no evidence to support which are more effective than others. The most common choices seem to be the awarding of a plaque and one-time cash award.¹¹

While the literature on teaching excellence, evaluations, and awards is extensive, as well as the range of types of awards, criteria and processes by which to select them, we have formulated a number of recommendations for the pharmacy academy to consider.

RECOMMENDATIONS

The evaluation of teaching has been one of the most studied and researched topics in higher education.¹⁸ Because of the complex dynamics of teaching and learning, those who are involved, and the many factors that impact these processes, there is little empirical evidence as to how exactly good teaching should be defined and rewarded so that it conclusively leads to improved student learning and faculty satisfaction. Regardless, from the large body of literature reviewed, this task force was able to distill the following recommendations for the recognition of teaching excellence:

• (1) There should be an expectation that ALL teaching faculty members engage in professional development to improve their teaching, with an emphasis on a scholarly, collaborative approach

• (2) Schools and colleges of pharmacy should provide more than a single award to recognize teaching excellence.

  • (a) If providing a single award, the criteria should be comprehensive and multiple sources of data should be used in the selection process.
  • (b) If providing more than a single award, awards should be based on different criteria and employ different methods of selection.

• (3) The criteria and selection process for the awards should be known to faculty members.

• (4) Criteria should ensure that many different faculty members over time are eligible for the awards.

• (5) The selection process should include broad representation and valid methods of selection.

• (6) The award itself should be significant and include multiple components.

CONCLUSION

It is important to recognize teaching excellence for the goals of improved student learning and faculty retention. To accomplish this, faculty members should strive to become excellent, scholarly teachers. This process is greatly enhanced when the culture of the institution promotes its development. Colleges and schools of pharmacy should create valid and reliable methods by which to judge and formally recognize teaching excellence. Institutions should critically examine their teaching awards processes in light of the recommendations and evidence put forth. There is also a need for more research about the impact of teaching awards to create evidence upon which more robust decisions can be made. Continually elevating the teaching enterprise in our colleges and schools should ultimately result in better pharmacists.

Appendix 1. Examples of student evaluation systems.

The IDEA Center - Individual Development and Educational Assessment

The student evaluation tool created by The IDEA Center (www.theideacenter.org) is a validated instrument for course and instructor evaluation. It focuses on student feedback of their own learning progress on 12 specific objectives and their perceptions of the instructor's use of 20 instructional strategies and teaching methods. The instructor determines which of the 12 stated learning objectives are deemed essential, important, or minor. (If students rate a faculty member low on an objective identified as minor, it does not adversely affect the assessment.) The instrument also adjusts scores for 5 factors that are outside the control of the faculty member: student motivation, student work habits, size of the class, course difficulty, and student effort. The instrument is objective and flexible to address various teaching styles and disciplines. Up to 20 additional questions can be added to the evaluation, with response either rated on a Likert scale or answered in writing. A report is provided for each individual faculty member as well as group summary reports for programs, departments, and course sequences. The service is provided by a secure, outside vendor, which adds to the objectivity of the process. The instrument can be administered on paper or online. In addition, documentation, technical, educational, and administrative support are provided.

The comparisons or ratings of an individual faculty member can be made on multiple levels including national norms for pharmacy programs or other health professions. In addition, if multiple colleges or programs at a single institution adopt this system, a second comparison can be made at the institutional level once enough data are collected.

The Flashlight Program: Evaluation, Assessment, Action

The Flashlight Program, developed by the Teaching, Learning, Technology Group (www.tltgroup.org), is a Web-based program for collection and sharing of survey instruments and data. Users may search the database of 500 validated items from the Current Student Inventory, write their own items and surveys, and/or use or adapt formats already present in the system. The Matrix Survey function allows for customizing survey instruments to match different subgroups and populations. Paper forms can be used but the data would need to be entered by hand for analysis. The system can also be used for non-teaching purposes such as creating research groups with colleagues within an institution or with other institutions participating in the Flashlight Online program to share surveys, item banks, rubrics, and data. Comparisons of ratings to other schools or databases are dependent on establishing collaboration with other groups or institutions.

CoursEval

The CoursEval system, developed by Academic Management Systems (www.academicmanagement.com), is a Web-based program for administering course surveys. The system does not provide any validated questions but has the ability to integrate with BlackBoard and university portal systems, generate several different reports at various levels, and provide longitudinal analysis for individuals and groups; can be used for student peer evaluations and experiential programs; and offers flexibility with regard to multiple-choice and narrative-type questions.

Appendix 2. Examples of useful practices for teacher self-evaluation

The Assessment of Learner-Centered Practices

The Assessment of Learner-Centered Practices (ALCP), a guided reflection process, assists teachers to reflect on their own beliefs and practices, how these practices are perceived by their students, and the impact of both teacher and student learner-centered variables on student motivation and achievement. The ALCP survey has been validated with more than 5000 K-20 teachers and more than 25,000 students. A survey for college use is available. The tool is a guided exercise to which teachers honestly respond and reflect on the feedback they receive from student evaluations. After reviewing student evaluations, the teacher is asked to engage in private reflection (working through a worksheet) and then meet with a teacher-mentor to discuss the student comments, the teacher's response to the comments, and how to use the comments for both course improvement and professional development. Guided reflection helps to identify areas of change so that faculty members can create personalized professional development plans. Because this tool is meant to be used in conjunction with student evaluations, perhaps this tool could be paired with the IDEA assessment. Too often teachers receive feedback from students, but are not sure how to use that information to further their own development. The feedback may be seen as unfair, resulting in a defensive response from the teacher, or a popularity contest not related to or focused on student learning and teacher development. This guided self-reflection tool can help teachers better utilize student feedback and mentor-guidance to work towards teaching excellence.

Course Evaluation on the Web

Course Evaluation on the Web, a Web-based system for effective evaluation within a learning community, was developed and implemented in a physiotherapy program in Australia. Since 2000, CEW has been used annually in 60 entry-level courses and 20 postgraduate courses. As of 2003, 500 students, 50 teachers, and 10 program managers had participated in the CEW process.

CEW process.

Students agree to provide feedback as a compulsory course requirement. In exchange, teachers agree to inform students about changes that may be made as a direct result of student feedback. Every 2 weeks students reflect on their learning using specially designed forms and teachers keep a course journal that includes reflections on their teaching as well as other aspects of the course.

CEW components.

After final examinations, students complete a Web-based instrument, using their semester-long reflections as a resource. Student comments are anonymous, but tracked for compliance. Both teachers and managers reflect on a ratings report generated from student feedback, conduct further analysis on student comments, create a summary report of the student feedback, and prepare a response after discussion with a peer. The report includes an overview of the course, teacher's view of best aspects of the course, and areas for improvement. The teacher also responds to student comments and whether he/she agrees with them and what changes will or will not be made based on student feedback (and reasons for these decisions). The teacher and student representatives meet regularly to review the function and use of the instrument, lead educational sessions for new students on how to provide quality feedback, provide professional development for both teachers and students integrated into formal teaching sessions, and facilitate mentor support. All new teachers shadow experienced teachers to further facilitate mentor support and peer guidance for professional development.

Follow up research related to CEW has demonstrated improvements in the quality of all areas of teaching and learning – increases in course performance scores of students, increases in job motivation, faculty job satisfaction – both intrinsic and extrinsic – and increases in teaching satisfaction by 30%, even though there was significant increase in teaching workloads.

Both the ALCP program and the CEW program reported improvements in teacher satisfaction, morale, and productivity. If the strategies outlined in the ALCP and CEW examples are implemented, there are potential positive outcomes. First, educational outcomes improve and faculty members are more satisfied with their positions. Teachers and students become reflective learners and partners in the learning community. Students become more interested in academia as a career, since they are integral in the process and have a better understanding and improved opinion of teaching. Through the use of self-reflection that is guided and shared, teachers became examples of expert learners who demonstrate professionalism and models of inquiry that are necessary for life-long learning. Students have a renewed sense of trust in faculty members and are now stakeholders in the learning community. Students see that their comments are important and valued and that they result in real-time change in the curriculum.