Investigating the faculty evaluation system in Iranian Medical Universities

Abstract

Introduction:

To achieve a valid evaluation of faculty members, it is necessary to develop an inclusive and dynamic system of evaluation addressing all the activities and responsibilities of faculty members. Among these responsibilities, educational activities comprise an important part which needs to be investigated. This study aimed to investigate the current system of evaluating the faculty members’ educational duties.

Methods:

In this descriptive cross-sectional study, a checklist for investigating the current evaluation system and was developed confirmed by a focus group. The data for checklist were collected through a researcher-made questionnaire and interview with eight experts of faculty evaluation that worked in different Iranian Medical Universities. For completion of information, the available documents and records were studied. Finally, the current evaluation system of different universities was depicted.

Results:

The developed checklist had six themes and 123 subthemes. The extracted themes included: Tools, evaluators, processes, appropriateness of faculty field of work with evaluation, feedback status, and university status regarding decisions made based on faculty evaluation results. As for comprehensiveness, all evaluation items except for evaluation and assessment skills and religiosity from personality traits subtheme were fully investigated. The evaluation tools were not enough for different types of education such as clinical education. In six universities, the feedbacks provided were only for making inter/intra department comparison, and no scientific suggestions were included. The results of evaluations were used only for the faculties’ promotions.

Discussion:

Suitability between evaluation and performance components is a necessity in every evaluation system. The study showed this does not exist in Iranian Universities. For instance, there was no appropriate tool for the evaluation of clinical education. Also, the results of the faculty evaluation were not used for the improvement of their educational performance.

INTRODUCTION

Higher educational system is an influential factor in achieving cultural, social, and economic goals of every country. Thus, in most countries, in order to develop higher education both qualitatively and quantitatively, policy makers adopt a variety of strategies.[1] One such strategy is performance evaluation of all parts of the system, especially educational performance. Faculty performance evaluation as a pillar of the system can undoubtedly guarantee the quality of education and lead to stability and elevation.[2,3]

Faculty evaluation is necessary for gauging their success in achieving educational goals. This has to be done through collecting data, measuring the data against some appropriate criterion, and finally judging each faculty member.[4,5,6]

This was first done in 1902 in England affecting the teachers’ payment procedure.[7]

Harvard University also evaluated its faculty for the first time in 1924.[3]

Literature review shows that the first faculty evaluation in Iran took place in 1971 in Hamadan University.[3,8]

In his book The Hidden Love, Kasiri documents an earlier evidence indicating that students of Dr. Hakami – A renowned pioneer medical doctor in Esfahan – Evaluated his performance in 1963, which earned him a reward for his efforts.[9]

Faculty performance evaluation needs to be kept in proportion with their responsibilities and duties and include all of them. These, according to the latest law approved by the Iranian ministry of health, include:

• Education

• Research

• Self-improvement

• Executive and managerial endeavors

• Provision of specialized health-care services

• Out-of-university specialized endeavors

• Cultural efforts.

As it is seen, education as the basic reason for the very existence of every higher education institute is the most important duty of the faculty.[3]

It includes teaching theoretical and practical courses, advisory services, theses supervision, clinical training (ward rounds, out-patient rounds, operating room, laboratory diagnosis, supervision of internship and clerkship courses, community-oriented medicine courses, active participation in grand rounds and morning reports, night shifts, on-call services, journal club presentation, conferences) seminars, continuing education, workshop presentations for other faculties, student and staff, writing questions and taking tests, developing course plans, participation in curriculum planning, and offering answers to the students’ questions.[10]

Many educational experts agree on the importance of teacher evaluation. One chief reason for this is the effect it is supposed to have on the quality of educational programs and their efficiency.[11,12]

Accountability is another effect of teacher evaluation since it provides evidence for how well the faculty, as the most costly component of the educational system, is providing services.[13,14,15]

Shinkfield believes faculty evaluation helps evaluate and develop individual capabilities, spot and reward outstanding teaching performances, and also revise inefficient teaching endeavors.[7]

To achieve these objectives, an evaluation system, based on general theory of systems, composed of input, process, output, and result has been devised. These work interactively and continual supervision of their total performance can guarantee their appropriate functioning.[16] There are a variety of evaluation systems from simple to complicated ones; from the task-based system which is feedback-free to those defined based on the mission of each university or college and for the purpose of providing professional promotions for the faculty. In this system, through a structured multi-stage feedback, and sometimes through the faculty member and the dean who exchange ideas and negotiate the points, they move toward achieving the ultimate educational objective.[17]

For the faculty evaluation system to be effective, a number of key parameters are essential. First, it should take all the priorities and values of the educational center into consideration and involve the faculty both in design and implementation of the evaluation endeavor; second, it should also be able to provide input for both managerial decisions and feedback for faculty performance improvement. Furthermore, it needs to cover both formative and summative evaluation issues and address them separately. An effective faculty evaluation system should use a variety of sources.[2,14,15,18,19]

Quite a large number of researches have been done on faculty evaluation.[20,21,22]

Each has addressed a certain component completely or in part. One main concern of all these endeavors has always been the insufficient validity and reliability of the questionnaires used.[22,23,24,25,26] In one study, for instance, considering a logical need for customization of the questionnaires, one researcher suggests an individualized cooperative model.[4]

Lack of reliability and validity of information sources or evaluators is another concern.[27,28,29,30,31] To handle this problem, researchers suggest using various sources of information to achieve more precise[32] and less biased evaluations.[27,33,34,35,36]

Another challenge in this regard is the effect of faculty evaluation on the teaching process.[13,30]

Almost no comprehensive research has been conducted on faculty evaluation in Iranian Universities. Thus, in this study, attempts were made to address all parameters involved in the matter through comparing faculty evaluation programs applied in Chief Medical Universities in Iran. Next, a number of working procedures were suggested for the improvement of the current programs.

METHODS

In this cross-sectional and descriptive study, faculty evaluation systems of Iranian Medical Universities including Tehran, Shahid Beheshti, Esfahan, Shiraz, Kerman, Tabriz, Mashhad, and Ahwaz were investigated in 2011. These universities were chosen because they seem to have more complete evaluation systems compared to other Iranian Universities. It was necessary to develop a checklist. Thus, 32 domestic research articles were reviewed and the themes and subthemes involved in faculty evaluation were extracted. As a result, a checklist comprising six themes and 123 subthemes was developed, investigated completed, and confirmed in a focus group including educational experts. To collect the necessary information for the completion of the checklist, three methods were used:

• An open-ended questionnaire

• Face-to-face interview and on-the-phone interviews, and

• Investigation of the available records and documents.

The open-ended questionnaire included three general questions on evaluation process, feedback provision, and subsequent decision making and were posted to and administered by the educational development centers of the universities. The purposeful sampling method was chosen and the interviews were performed with eight key educational experts who either used to work or still worked as the heads of faculty evaluation offices in selected Medical Universities. The reason for selecting these participants was that they could provide us with the needed information for faculty evaluation system due to their experience. So, the inclusion criterion was being responsible for faculty evaluation in the university for some time and the exclusion criterion was refusing from being interviewed. To address the ethical issues, their informed consent was obtained and their views and experiences were used without mentioning their names. A researcher-made questionnaire was also used and the responses were recorded and later analyzed. There were 23 questions on the current situation and three questions on the ideal situation. Four out of 23 questions were targeted at the necessity and importance of an evaluation system and educational responsibilities of the faculty; three questions were related to the evaluation tools; six questions were about the current practice; eight inquired about the application of the results; and two questions probed particular measures each university could take for the improvement of the faculty evaluation system. The three questions on the ideal situation were intended to derive ideas and suggestions on revising the current state of affairs. Validity of the questionnaire was confirmed by experts. Also, through correspondence with the universities, the documents, forms, and tools used for faculty evaluation were gained access to.

Next, the collected data were analyzed through the checklist, the current state of each university was depicted, and best possible suggestions for improvement were provided.

Since the study was not aimed to compare or validate faculty evaluation systems employed in the universities under the study, the findings were presented as descriptions of the systems. Also, attempts were made to observe ethical considerations such as confidentiality of personal ideas and information.

RESULTS

Literature review of the Iranian research on faculty evaluation resulted in the development of a checklist. This was investigated and finalized in a focus group. The checklist included six themes: Evaluation tools, evaluators, evaluation process, feedback process, evaluation results, and finally faculties’ field of work considerations. Each theme included several subthemes. Through the checklist and data from eight completed questionnaires, eight interviews, and 130 types of faculty evaluation forms from eight type one Medical Universities, faculty evaluation systems were studied. The results are presented separately here.

As it is evident from Table 1, experienced faculty and department experts cooperated in tool provision, and revision of tools was done in less than 3 years in most cases. In a few universities, partial revision occurred annually.

Table 1.

Results of investigation of faculty evaluation tools

Tool variety included two subthemes: Appropriateness regarding teaching environment and appropriateness regarding educational departments. As for clinical training, it was not evaluated through specific forms for clinical education. Instead it was assessed through the forms which were used for other educational environments.

Four universities evaluated clinical training performance of their faculty by only considering hospitalization, outpatient care, health center, and ward round. Evaluators were students or residents in seven universities, and in one university, the ward supervisors and the educational deputy of the health center acted as the information sources or evaluators.

Appropriateness regarding educational departments was not seen in any of the universities under the study. Yet, some attempts had been made in a number of the universities and customized tools were being used in a number of departments. They are included in Table 1.

The findings show that in all universities, the undergraduate students evaluated their teachers, and in most universities, this was also done by postgraduates. Although educational documents were considered necessary for faculty promotion committees in some universities, class observation or even document consideration was not a common practice in any of the universities.[Table 2].

Table 2.

Evaluators in faculty evaluation system

As Table 3 shows, evaluation process includes three phases: Pre-, in-the-course-of-, and post-evaluation. There was a directive on pre-evaluation procedures in all universities, but it was not followed. There was an exception in one university where representatives of the classes were briefed, and reminder E-mail and text messages were sent. Although, due to atomization, continuous revision of the evaluation process was being done; most universities did not find it feasible or necessary to revise their evaluation process. Briefing of the evaluation technicians was done in person-to-person sessions in most universities. Only in two cases, two universities had administered evaluation workshops only once

Table 3.

Phases of evaluation process

As for in-the-course-of-evaluation process, in compliance with the directive, in six universities, students evaluated their teachers before final examinations in class courses and on the last week in clinical courses. Faculty evaluation by colleagues or administrators was done once a year in January. In one university, attempts had been made to apply formative evaluation as well, but others used only summative evaluation of their faculty. In almost all universities under the study, evaluation was done online (three universities) or was beginning to be done online (three universities). In non-online cases, the data were scanned and then analyzed on a computer.

Another aspect studied was the appropriateness of the faculty filed of work with the evaluation. This included type of education (theory, practical, clinical), Level of education presented (for Bachelor, Master, or Doctorate levels), quantity of education, and the responsibilities assigned for faculty member [Table 4]. This factor was neglected in all the universities studied. Only in some schools in two of the universities, the type of education and addressees were considered in faculty evaluation.

Table 4.

Appropriateness of faculty field of work with evaluation

Except for one university in which the results were provided as per request of the faculty member himself, in other universities, the results of educational performance evaluation of the faculty members were corresponded as written with each faculty member. Descriptive feedback ranged from a mere raw score to inclusion of a reason for the score to explaining the weak and strong points of the faculty. In some schools of universities, department heads or deans informed the faculty orally and provided explanations in addition to written reports. [Table 5]

Table 5.

Feedback status

Besides feedback presentation, decisions made based on faculty performance evaluation results make up another important aspect of evaluation [Table 6].

Table 6.

University status regarding decisions made based on faculty evaluation results

Decisions based on the evaluation results mainly included promotion and raise. In one university, a list of faculty gaining a qualifying evaluation score for promotion or raise was dispatched to the related committee. In most cases, a higher director was the one to issue rewards or punishments, but in two universities, a selected committee or the selection committee of the university including educational vice-chancellors and a number of educational directors and experts made those decisions.

Some other results reached following interviews with a number of experts (not tabulated)

Since data were also collected through a semi-structured interview, a number of obstacles were detected. The interviewees mentioned contextual problems, i.e. regulations and dominant culture of each university. Another issue was related to procedures, inputs, and outputs. To develop a comprehensive evaluation system, it is necessary to tackle these problems; therefore, they are listed below:

• Lack of trust among faculty

• Fear of disclosure of results a of peer or administrator evaluation

• Director's low reliance on evaluation results

• Negligence of religious values in development of faculty evaluation systems

• Overemphasizing research work rather than educational endeavor

• Need for money awards for faculty performing high quality educational work

• Unfair completion of evaluation forms by directors trying to avoid probable confrontations

• Putting unduly emphasis on evaluation forms at the expense of other components of the system.

DISCUSSION

This study investigated faculty evaluation systems in type one Iranian medical universities regarding sources of evaluation (evaluators), tools, process, feedback provision, results and their impact, and finally the faculty's field of work.

Investigation of the tools showed that regarding content comprehensiveness, the current tools are incapable of measuring evaluation and assessment skills and “religiosity” of the faculty, but can acceptably cover other aspects.

Considering the great significance that the quality of clinical education bears in community health promotion and the importance of practical skills in medicine, it becomes evident that this part of faculty's duties-teaching clinical skills – Is extremely important. Thus, it is necessary to monitor and improve clinical education through evaluation continuously and comprehensively.[37]

Yet, clinical education was noticed to be little attended to and suffered more shortcomings in our universities; forms did not have adequate validity, which was in accordance with the study by Tootoonchi et al., where faculty expressed a negative idea about evaluation forms and demanded revision.[23] One explanation could be that evaluating faculty's clinical performance is difficult per SE yet, it deserves attention and better tools need to be developed in Medical Universities. Sarchami has proposed a faculty evaluation form covering all their duties and responsibilities.[21]

A factor in achieving efficient evaluation is developing the evaluation tools, if not the whole process, with the help of the evaluatees.[8,38] Thus, a more efficient evaluation system could be established. Using more information sources will result in more reliable data, will earn the faculty's trust and as a result, will improve efficiency.[39]

Presently, faculty evaluation is mainly focused on students’ evaluation. Though this has its own advantages.[24], and because the results are not reliable, some researches including Parsa Yekta conclude that it should be done continuously during their course of study and also the results should never be used for making following administrative decisions.[27,29,33,34,36]

A more integrative approach combining a variety of other techniques can remove the present deficiencies and end in more reliable results.[8] However, our findings showed that other techniques such as evaluation by colleagues or even by directors are practiced only in a few universities. One reason also existing in similar cultures is fear.

Colleagues or director fear that their opinions reflected in evaluation forms may not be kept confidential. Any leakage may cause confrontation; therefore, evaluation forms are completed with this fear in mind. The result is unfair evaluation which disturbs reliability of the data.

Direct observation by an observation expert also brings about confrontations. Since the faculty would not welcome criticism, this has failed in one university. Another study also mentions that because of its stressful nature, direct observation may be a source of friction among faculty.[8]

Considering the value of these techniques, however, before applying these techniques, possible solutions for these problems must be thought of.[8]

The investigation showed that in spite of directives and regulations, they are not followed precisely; therefore, faculty evaluation process has confronted problems and has gone astray. One main reason could be the fact that administrative staffs are not committed to their duty since they have not received any training. Since continuous training does not only serve the purpose of providing knowledge and can also help improve attitudes and performances, training courses can result in the improvement of the performance of the administrative staff by changing their attitude toward the evaluation process.[40] However, with spread of electronic evaluation systems, problems such as faculty/students briefing, time, and evaluation environment are expected to be reduced or totally removed.

There must also be an appropriate relation between a faculty's evaluation and his/her field of performance. This study showed that in most cases there was no appropriate relation between evaluation and type of educational duty, type of addressee, and quantity of duty. For instance, a faculty mostly teaching clinical courses to residents, is expected to be evaluated by the very same residents, but for a variety of reasons, this never happened. There was no appropriateness observed in the process of faculty performance evaluation.

A deep deficiency in the current university evaluation systems is lack of correct use of the evaluation results.[13,4] The results should be used for two purposes:[20,41] Formative and Summative purposes. The former includes faculty and student performance improvement due to improvement of teaching/learning process. The latter includes faculty promotion and system improvement and facilitation of managerial decisions. If the results are not used appropriately, the integrity of the whole system will be questioned.[7,11,18,20,24,42]

The faculty evaluation system in our country pursuits both purposes.[10]

Thus, it is expected that faculties receive active feedback. As they are the first to initiate any change in teaching-learning system, they are entitled to be informed of their evaluation results. The faculties, knowing their own status in the systems, should learn about their weak/strong points and receive any useful suggestion for modification of their approaches.[28]

Another important point pertaining to feedback is that each faculty member's evaluation report should be compared with his/her previous records,[41] and the faculty should be rewarded if any improvement has been made.[43] This may include payment raise, overtime payment raise, etc.[44] Comparing the faculty's performance with those of his/her colleagues, although not observed in most of the universities in this study, could enhance motivation. Unfortunately in our current systems, the results are not used properly. Educational directors can support reform through correct administration of the total process, using the results duly.[8,38]

The findings showed that the current faculty performance evaluation systems in Iranian Medical Universities need to be revised. It is, therefore, suggested that an appropriate model meeting our educational, cultural, and social needs be developed to solve the current problems. The limitation of the study was that Investigating only parts of the evaluation system.