Peer Review of Teaching

Abstract

This article provides an overview and description of peer review of teaching for faculty members and administrators who would like to implement a peer review program. This may include classroom and clinical settings. A brief overview, procedure, and a teaching competence evaluation rubric are provided

Introduction

Peer review is evaluation, by colleagues or peers, of all teaching-related activities for either formative (for development) or summative (for personnel decision) purposes. Because there are different purposes for each type of evaluation, the processes may be conducted independently of each other. Components of either type of review may include course materials, student evaluations, teaching portfolios, documentation of teaching philosophy, teacher self-assessments, classroom observations, and other activities that may be appropriate to a discipline.¹ A trend is growing toward colleagues' helping each other improve their teaching abilites.² National attention was placed on peer review of teaching in 1994 when 12 universities joined forces in the American Association of Higher Education (AAHE) sponsored project, From Idea to Prototype: The Peer Review of Teaching.³ Pat Hutchings, who directed the Teaching Initiative of the AAHE during this project, stated, “The peer review of teaching can, in its most powerful forms, be less a matter of judging teachers than of improving teaching, with the focus moving increasingly to ways we can help each other improve the quality of our collective contribution to students' learning.”⁴

Peer review of teaching is not intended to replace student evaluations. Experts indicate that although students are the most appropriate judges of day-to-day teacher behaviors and attitudes in the classroom, they are not the most appropriate judges of the accuracy of course content, use of acceptable teaching strategies in the discipline, and the like. For these kinds of judgments, peers are the most appropriate source of information.⁵

Teaching observation by colleagues works best when it is carefully organized.⁶ It is intended that the peer review process be conducted by interested, academically responsible individuals. Members of peer observations should be respected for their teaching ability. Ideally, the faculty review system should be integrated into a faculty development program that can assist the instructors to teach better.⁷ Observers should use some sort of checklist (example provided in the appendix). Based on a review of the literature on peer review of teaching,^{6, 8–10} colleagues can reliably evaluate

1. Commitment to teaching and concern for student learning;

2. Selection of course or teaching session content;

3. Mastery of course or teaching session content;

4. Course or teaching session organization;

5. Appropriateness of course or teaching session objectives;

6. Appropriateness of instructional materials (such as readings, media, visual aids);

7. Appropriateness of evaluation devices;

8. Appropriateness of teaching methodology;

9. Student achievement, based on performance on exams and projects; and

10. Support of departmental instructional efforts.

This paper promotes the idea that improving teaching is the result of an integration of refinements in how professors think about teaching. Increasing awareness about how we can influence student learning is an important first step.

Process of Peer Review

This proposed peer review of teaching process is one that is designed to be more formative than summative. It could be a component of a continuous improvement program. Its main purpose is to provide faculty with meaningful feedback that will help the faculty member set goals and take steps toward improving teaching abilities. These goals are derived from the process of faculty self-assessment, peer observation, and collegial dialogue. Utilizing consultations with teachers before and after peer observation is essential and provides insight into the teaching objectives and strategies utilized by the teachers being observed. The results of peer review may also be summative when utilized by faculty department chairs for the purpose of developing growth plans within the institution's faculty appraisal system. It has been observed that peer review programs in many universities contain very similar elements and processes. The similarity is probably due to their basis being derived from the established literature on this subject. This peer review process has been adapted from these same sources.^11–19

Peer review can consist of these basic steps, conducted in this order:

1. Review of course materials

2. Preobservation consultation

3. Teaching observation

4. Postobservation consultation and feedback

5. Written evaluation

6. Monitoring the peer review process

Checklists can be developed and used as a guide for each of these steps based on the information and descriptions below.

Step 1: Review of Course Materials

Reviewing course materials is a starting point of peer evaluation. This allows the reviewers to examine the syllabus, course guides, samples of presentations (eg, Power Point slides), required and suggested readings, handouts, assignments, tests, and other student evaluations. Course materials are very important in supporting the learning that takes place in a course. A look at a test can tell much about the level of learning goals in the course, the instructor's perception of what is important, and the instructor's pedagogical style toward the students.²⁰ Combining review of course materials with evidence gathered in other ways, such as observations, is necessary to arrive at a full picture of instructors' teaching abilities.

When the peer reviewers are selected, the teacher under review can provide a copy of the course materials. Some of the factors that may be evaluated to determine the quality of these materials include the following:

• Are presentations, handouts, readings, and/or assignments relevant and current?

• Are they effectively coordinated with the syllabus?

• Are they challenging for students?

• Do they include opportunities for active learning or collaboration?

• What is your overall judgment on how thoroughly these materials reflect the instructor's preparation for his or her teaching work?

The faculty member may also supply to the reviewer a copy of his or her teaching portfolio containing a statement of the professor's philosophical beliefs about teaching and student learning, along with teaching strategies, sample(s) of best work, and plans for self-assessment of teaching effectiveness.

Step 2: Preobservation Consultation

This step initiates the interaction between the peer reviewer(s) and the teacher. The consultation allows the reviewer(s) to gain insight into the goals and strategies utilized by the teacher. There should be ample time allowed for the development of meaningful and collegial dialogue. It is important to meet with the reviewers ahead of time to discuss the educational intentions, aspects of the teaching for which the teacher seeks feedback and constructive advice, and whether different methods or strategies will be used. The person to be observed generally recommends an observation focus—for example, student activity, presentation skills, clarity of explanation, interactivity, use of aids, and/or questioning skills. Before the teaching observation, the reviewer(s) can schedule a meeting with the instructor, at which time, several points may be discussed, including course materials, overall course objectives, teaching method and objectives for the specific learning event to be observed, and other topics considered important by either party.

Step 3: Teaching Observation

Peer review of another faculty member is not a simple task. It should be no surprise that most faculty, not having been trained to teach or having practiced teaching in a scholarly manner, find peer review of teaching to be complex and confusing. It should be more than simply “come and observe my teachings,” which could result with just dealing with the surface aspects of the teaching.²¹ Classroom observation is perhaps the most familiar form of peer review. An advantage of classroom observation by peers is that the peer's own development may be fostered through the ideas obtained from watching a colleague. Reciprocal classroom observations are a strategy employed in many faculty development programs, such as the New Jersey Master Teacher Program.²²

Observations of primary teaching mode should be the focus. For clinical teaching, peer review in a patient care setting should be considered. Care should be taken to observe the more common teaching mode used. Ultimately, it should be the prerogative of the reviewer to choose the teaching event observed. The reviewer should use a checklist for teaching observation as a guide for evaluating the faculty member's performance. A checklist rubric should be developed by involved faculty in order to obtain evaluation criteria that can be objectively and fairly applied during peer review. This should be developed according to the objectives and goals of the institution. Areas that need improvement, are satisfactory, or are exceptional can be defined. For both classroom and clinical teaching, the appendix provides an example. In addition to an evaluation rubric worksheet, an observation checklist of clinical teaching strategies can be utilized. In a review of the literature, Heidenreich et al²³ identified the following strategies in clinical teaching: orienting learner, prioritizing or assessing the learner's needs, problem-oriented learning, priming, pattern recognition, teaching in the patient's presence, limiting teaching points, reflective modeling, questioning, feedback, and teacher/learner reflection.

At the end of the visit, the reviewer can ask the students to complete a feedback form on whether the activities observed were typical of the course's norm. Sample questions can include: Any reactions to or comments about today's activities? How typical were today's teaching activities compared with others? How useful were today's activities?

Step 4: Postobservation Consultation and Feedback

This meeting works best as a discussion, not just the observer giving feedback. Giving the teacher the first opportunity to comment allows for active participation and discussion. Start by discussing what the teacher is doing well, and why. Areas of focus from the first consultation can be revisited. Also, the same items mentioned earlier in the area of self-reflection can be a starting point.

In order to provide timely feedback, the reviewer can schedule a meeting with the faculty member within 1 week of the observation. Several areas may be discussed. The teacher and reviewer may wish to review the performance in the categories listed on the observation checklists. The most and least effective elements of the teaching observed can also be discussed, as well as suggestions on how to improve performance. This may also be a good time for the teacher to clarify any elements of the mode of teaching that are unfamiliar to the reviewer. In short, this time can be used to cover any topics considered important by either party.

The process of giving feedback is critical for improvement of teaching. How feedback is provided is as crucial as the process of peer review itself. The chances that teaching improvement will occur increases when feedback is accurate and specific (often better with examples), is given in a supportive and nonjudgmental manner, provides specific alternatives for aspects of teaching that need change or improvement, is focused and relevant, and allows room for discussion and interaction. In the discussion, the term work or contribution is used to describe the matter on which feedback is given. The reader is referred to David Boud's guidelines for feedback published in his book Enhancing Learning Through Self Assessment.²⁴

Step 5: Written Evaluation

Within a reasonable period after the postconsultation, the reviewer can prepare a written evaluation of the professor's teaching and supply the evaluation to the teacher's supervisor. The supervisor can then supply the faculty member with a copy of the evaluation. The faculty member should then have the opportunity to respond to this evaluation in writing. After the completion of the peer review, a copy of the evaluation should be provided to the faculty member and the department chair. As in all faculty evaluations, these documents should be kept confidential.

Step 6: Monitoring the Peer Review Process

The faculty member being reviewed may complete an exit survey, which will serve to give the peer review committee feedback regarding the strengths and weaknesses of the peer review process. The exit survey can also be given to the department chair. This can cover the following areas:

• Was there an effective exchange of teaching and learning strategies with the reviewer(s)?

• Does the evaluation document the observed use of teaching and learning strategies?

• What was the most positive or helpful aspect of the peer review process for you?

• What was the least helpful or most negative aspect of the peer review process for you?

• What would be the most helpful or beneficial to the development of your teaching effectiveness?

The peer review committee, in consultation with interested parties, should reevaluate this peer review procedure periodically. The results of this review can be presented to the faculty, along with any recommendations on how the procedure should be amended.

Challenges

Some challenges and obstacles of initiating a valid peer review process include: lack of evaluation resources (including time), overburdened faculty, evaluator subjectivity, lack of peer reviewer training, teacher anxiety or skepticism, scheduling conflicts, and lack of a reliable reward system for faculty achievement. Institutions with smaller budgets may find creative ways of rewarding faculty for their efforts and growth. These may include personal holidays, recognition meetings, and/or thoughtful gifts.

There may be skepticism centering on the notion that colleagues are experts on subject matter, not on pedagogy, and that reliable agreement among raters is rare. Research has shown that when colleague ratings are based solely on classroom observation, only slight interrater agreement can be expected. However, research also demonstrates that if peer review evaluators are given proper training and experience, their ratings based on classroom observations are sufficiently reliable.² Some of the problems appear to be connected with key factors such as lack of consensus on standards for evaluation, lack of observation or content analysis skills on the part of the reviewers, and lack of systematic process and documentation.²⁰

In introducing any new program it is essential to be aware of and plan for the potential challenges that may lie ahead. This will provide a better chance of successful implementation. Because of the above possible challenges, it is recommended that the program be introduced gradually. This process may include individual faculty peer review exercises with preliminary information gathered, reviewed, and discussed openly. Also, before full implementation of peer review begins, a pilot program may be utilized. Ultimately this peer review of teaching program should be accomplished through a supportive and nonpunitive approach. How much the above or other peer review challenges may exist in an institution should determine the speed of implementation.

Appendix: Teaching Competence Evaluation Rubric

(This form may be downloaded at no cost from: www.journalchiroed.com)

Teaching competence	Needs improvement	Satisfactory	Exceptional
Commitment to Teaching and Student Learning	Exhibits a lack of enthusiasm and excitement toward teaching and students Discourages student's questions, involvement, and debate Makes accessibility and availability difficult for students Discourages individual expression	Often demonstrates enthusiasm and excitement toward teaching and students Encourages student questions, involvement, and debate Is accessible and available to students Allows for individual expression	Consistently demonstrates enthusiasm and excitement toward teaching and students Has a well-established learning environment that encourages student questions, involvement, and debate Makes students a priority in being accessible and available to their needs Encourages and allows for individual expression
Selection of Teaching Content	Rarely selects examples relevant to students experiences, “real-world” applications, and/or objectives Does not relate content with what was taught before and what will come after Does not present views other than own	Selects examples relevant to students experiences, “real-world” applications, and/or teaching objectives Relates content with what was taught before and what will come after Sometimes presents views other than own when appropriate	Frequently selects examples relevant to students experiences, “real-world” applications, and/or teaching objectives Often relates content with what was taught before and what will come after Presents views other than own when appropriate and provides explanation for possible differences of opinion along with evidence
Mastery of Teaching Content/Knowledge	Rarely explains difficult terms or concepts Does not present background of ideas and concepts Does not present best evidence and up-to-date developments in the field Does not answer students' questions adequately or does not admit error or insufficient knowledge	Explains difficult terms or concepts Presents background of ideas and concepts Presents best evidence and up-to-date developments in the field Answers students' questions adequately or admits error or insufficient knowledge	Explains difficult terms or concepts in depth and in more than one way Presents background of ideas and concepts in depth Frequently presents best evidence and up-to-date developments in the field Answers students' questions in depth and admits error or insufficient knowledge with commitment to seek out information
Organization	Does not begin on time and is disorganized Fails to preview material to prepare students for the content to be covered in patient encounter or workshop Fails to summarize main points at the end of session Does not provide clear directions and procedures Does not plan on a daily or weekly basis	Begins on time Previews patient cases or session content Summarizes main points at the end of session Explains directions and procedures Plans for daily and weekly activities	Begins on time in an orderly, organized fashion Consistently previews patient cases or session content Summarizes and distills main points at the end of session Consistently explains directions and procedures Plans daily and weekly activities and follows-up on plans that was not able to complete
Meeting Teaching Objectives	Teaching content and methods do not meet stated objectives of syllabus or as stated by teacher	Teaching content and methods are geared to stated objectives of syllabus and as stated by teacher	Teaching content and methods clearly meet stated objectives of syllabus and as stated by teacher
Instructional Materials (Readings, Media, Visual Aids) Didactic	Fails to provide students with instructional materials	Incorporates various instructional supports such as slides, visual aids, handouts, etc.	Incorporates various instructional supports such as slides, visual aids, handouts, etc; Also provides references for materials presented when appropriate
Intern Evaluation and Achievement (Methods and Tools)	Fails to provide students with assessment criteria and instructions Does not perform minimum number of assessments required Assessments are of poor quality, have minimal information, and do not lend themselves to meaningful student feedback Feedback is not provided or is minimal	Provides to students assessment criteria, instructions, and expectations Provides satisfactory number of assessments required by department Assessments are of satisfactory quality, have adequate information, and lend themselves to meaningful student feedback Feedback to students is adequate	Provides to students the goals of assessment, along with criteria, instructions, and expectations. Also provides examples of expectations and type of feedback given Provides beyond satisfactory number of assessments required by department Assessments are of exceptional quality, have in-depth information including comments, and lend themselves to meaningful student feedback Feedback to students is exceptional and allows for student's self-evaluation and reflection with steps for improvement
Teaching Methodology and Presentation	Fails to use a variety of clinical teaching strategies to address diverse learning styles and opportunities Fails to responds to changes in student attentiveness Speech is inaudible and unclear Is unprofessional and use of humor is negative and inappropriate Fails to establish and maintain eye contact with students Does not provide demonstrations when needed Does not promote life-long learning Does not promote students to be independent learners	Uses a variety of teaching strategies to address diverse learning styles and opportunities Responds to changes in student attentiveness Speaks audibly and clearly Models professionalism Establishes and maintains eye contact with students Provides demonstrations as appropriate Mentors students in life-long learning skills Allows students to be independent learners	Uses a large variety of teaching strategies to address diverse learning styles and opportunities Responds to changes in student attentiveness with comfortable transition of teaching strategies Consistently speaks audibly and clearly Models professionalism and use of humor is positive and appropriate Establishes and maintains eye contact with students while communicating a sense of enthusiasm toward the content Provides demonstrations as appropriate and has students demonstrate their understanding Routinely mentors students in life-long learning skills Guides students to be independent learners
Support of Department Instructional Efforts	Is unaware of department's instructional efforts Does not demonstrate support of department instructional efforts	Is aware of department's instructional efforts Demonstrates support of department instructional efforts	Has a comprehensive understanding of department's instructional efforts Demonstrates support of department instructional efforts and demonstrates leadership in progressing instructional programs

Note: This appendix is based on ideas in references 12 and 15 and the authors' experience.