Designing and evaluation of the teaching quality assessment form from the point of view of the Lorestan University of Medical Sciences students - 2010

Abstract

Introduction:

Education is basically one of the Universities’ and faculties’ leading missions and duties; its promoted quality will also lead to an elevated educational quality in the University. Teacher assessment can be mentioned as essential for the success of the quality promotion process. This article deals with the designing and evaluation of a teaching quality evaluation form for teachers, from the Lorestan University of Medical Science students’ point of view.

Methods:

A two-stage, cross-sectional study was conducted on 290 Lorestan University of Medical Science students. First, evaluation priorities were extracted using the Delphi technique in the fifth section, including teaching skills, communication skills, principles of training, and skills assessment. In the second stage, as the priority and importance of each item was evaluated in the fourth Lickert option, sampling was done in few stages. The study instrument was a questionnaire, which included six areas. The first part of the questionnaire was made up of the demographic characteristics and the second part included five evaluation areas that were obtained from the student. The collected data were analyzed using statistical software SPSS-16 and chi-square test, Kruskal-Wallis test.

Results:

In the areas of teaching skills, mastery of the course, individual characteristics, self-confidence, communication skills, intimate relationship with students, educational principles, rules respecting the beginning and end time of class, skill assessment, and an accurate comprehensive examination at the end of the semester by the students, were chosen as the most important factors. There were significant differences in the majority of expressed comments between the genders and academic status (P < 0.05).

Conclusion:

Students can properly diagnose the essential factors in teachers’ evaluation, but in item prioritizing they may be partly affected by some factors such as gender, academic status, semester, and academic course.

INTRODUCTION

The practice of evaluating teaching and courses in higher education, through Course and Teaching Evaluation Questionnaires is now very widespread, to the extent that it is rare to find colleges that do not routinely evaluate courses.[1]

Evaluation is an organized process of gathering, analysis, and description of information, to determine as to what extent aims are likely to be achievable until the decision-making process implements a promotion, based on evaluation. To generalize the above definition, determining a teacher's success on achieving educational goals can be attributed to teacher assessment.[2,3]

Teacher evaluation, in fact, is a process that is applicable to promote the quality of learning and education and also in the decision-making of jobs, including selection, job stability, and promotion. Therefore, there is no doubt that evaluation is essentially required and beneficial. Teacher evaluation is one of the numerous educational and complex assessments, and its complexity is in bearing with the lack of validity and precision in terms of the tools and measurement methods, because no information source or mentioned method provides any essential or unbiased information that is necessarily needed for appropriately executing the assessment. However, the finding of this evaluation is also indispensable.[3] For success of the evaluation, two types of information are necessary, the educational achievement criteria and an index of success rate to achieve a determined criterion.[3] The assessment criteria consist of related items that are confirmed to judge the desirability of the evaluating items. Furthermore, unanimously selecting a criterion is one of the most pivotal aspects of the evaluation process.[2] Nowadays, for validity and performance, evaluation is considered as an effective factor. Hence, to evaluate the quality of the teachers’ performance and get information on their strengths and weaknesses, a lasting evaluation must be conducted. There are several identified methods to assess teacher educational practice. One of them is student evaluation of teaching, which is commonly used in Universities, in recent times. The most common sources of evaluation data have been students, peers, and teachers themselves[4,5] in this evaluation. Students’ opinions in terms of teachers’ behavior and educational practice will be analyzed by using a multiple choice or an open questionnaire.[2]

Evaluation, however, is logically and essentially beneficial, as it helps to find the positive and negative program aspects, but to achieve this goal, applying an appropriate system, with sensitive and precise tools, is required. In this manner, evaluation would properly be done and its negative outcomes would also be diminished. Decrease in staff satisfaction and motivation, lack of accountability, and system outcomes are caused by poor evaluation.[6,7]

In the Bergman investigation on the students’ and faculties’ opinions with regard to the traits of successful medical teachers and effective education, in the teachers’ and students’ point of view, the indispensable characteristics were scholarship and communication skills, respectively.[8] Moreover; the Ahvaz University of Medical Science survey showed that the most pivotal trait from the student's point of opinion was educational performance. On an average, based on their idea, both the personal and academic characteristics of the teachers’ and students’ assessment procedures were also important.[9] Portfolio views of the Ghazvin University of Medical Science identified that having self-confidence, maintaining a close relationship with students, attention to the start and termination time of the class, and using exact and comprehensive questions, were the essential characteristics of a teacher.[10]

Teaching evaluation can identify common problems existing in teaching and help provide solutions for them. In this process, the evaluators ask students about problems they experienced in their education and report these problems to the appropriate administrative officers, particularly when students are reluctant to report the problem or when they do not know who to report the issues to[11] and they also serve as a basis for decision-making with regard to hiring, contract renewal, incentives, and promotions (summative evaluation).[12–15]

Although many studies have been published about teaching evaluation in medical education, most of them provide only descriptions of the evaluation system. A few studies published focus on the tools for evaluation, which extracted from students’ opinions.[16,17]

Therefore, determining the proper and precise criteria for teacher evaluation is necessary, because teacher's evaluation forms would be designed based on the identified criteria. Lack of studies toward designing teacher evaluation forms, in terms of theoretical teaching quality and its importance, necessitated us to consider and design an investigation in this regard.

MATERIALS AND METHODS

To obtain data, a two-stage (qualitative and cross-sectional) study was conducted. In total, 290 samples were gathered from the Lorestan University of Medical Science students who were studying in several courses. The semester was considered as an entrance criterion and the first semester students were not allowed to enter the investigation, as these students were not adequately aware of teacher evaluation. All the students who were studying in semester 2 and the higher semesters took part in this survey. All the existent University courses were authorized to enter the investigation, furthermore; to respect the ethical issues, registering member names were banned, and subjects who were willing to participate provided an informed consent. To enhance the students’ precise answers prior to filling the questionnaires, the researchers provided students adequately with the necessary information. By announcing the importance of the current study, the scholars endeavored to garner more participation, following which, enough time was allocated to answer.

In the first stage of study, the information gathering method was conducted as follows: From the outset, evaluation priorities were identified through the Delphi technique, in five sections, including training skills, personal characteristics, communication skills, addressing educational regulations, and assessment tact. The researchers announced that the required number of individuals for executing the Delphi technique is 30 people, and some of them, as a whole, believed that increasing numbers would result consequently in a more exact judgment;[18] therefore, all of the 290 students took part in this survey. At the start, the students listed indispensable factors for evaluating teachers through an open question. The announced items were gathered and similar ones classified in distinct categories. The sorted items were secondly referred to students and they were asked to notify the idea. Student recommendations were sorted again and for a third time delivered to the students. In this final stage, neither was any items eliminated nor added.

In the second stage of the study, the designed tool in the final form had two parts entailing demographic part (academic status, semester and course, sex and …) and the evaluation part (training skills, personal characteristics, communication proficiency, respecting education regulations, and assessment skills). Students had to evaluate each item based on the four-point Lickert scale (very important, important, moderately important, and not important). Content and face validity was confirmed by specialists and its reliability was supported using a test-retest on 30 subjects during two weeks; the coefficient was taken as 0.8 - 0.9 in different constructs.

Questionnaires were subsequently given to the students. Fourteen questionnaires that were not accurately filled were finally eliminated. Data were analyzed using SPSS (ver.17) and the Chi- square test used to compare the frequency of each item between groups. The Mann-Whitney Test was used to analyze the difference in the importance of evaluation items from the point of view of students, between the two genders, and Kruscal Wallis Test was used to analyze the difference in importance of evaluation items, from the point of view of students, between the mean grade point average groups.

RESULTS

In total, 179 (64.9%) and 97 (35.1%) students were female and male, respectively. The most prevalent of the subjects were allocated to health 83 (30.1%), paramedical 60 (21%), and medical colleges 51(18.5%), respectively. Likewise, the greatest frequency of participants was in accordance with four semester students, 99 (35.9%), and undergraduates 194 (70.3%). As mentioned at the outset, the fifth part of the evaluation was extracted using the Delphi technique.

A portfolio of finding was achieved due to the second stage of evaluation, with regard to both section importance and each item. In section of the training skills [Table 1], appropriate literacy of the teacher was considerably highlighted as a determining factor. 254 (92%) respondents suggested that appropriate knowledge of the teacher play a pivotal role in the assessment process. There was a statistical significant difference in this view between two genders (P < 0.001) and students academic status (P < 0.001).

Table 1.

The frequency of student's answers toward the importance of the teaching quality assessment form based on training skill and personal characteristics

According to the results, we found that 173 (96.6%) females and 81 (83.5%) males believed that teacher eligibility was very important [Table 2]. 69 (100%) students, who were ranked 16 to 17.9, in terms of the previous academic status, stated that teacher s proper literacy was important in the evaluation process. Furthermore, 20 (71%) of the samples who were ranked in the under 14 academic status group were in agreement with this idea. In this section, allocating time to answer the students’ questions and teaching by using validated resources were prioritized as second. In this part of the assessment, there was also a statistically significant relationship between the gender and academic status groups. Presenting so many lessons was considered as the most unimportant issue from the students points of view; only 22 (22.5%) of the individuals described it as significant compared to 161 (58.3%) of the samples who described it as unimportant. Source validity for teaching was assessed as being highly crucial in the teacher evaluation process by 68 (70.1%) male students compared to 110 (61%) female students [Table 3].

Table 2.

The frequency of student's answers toward teachers’ mastery of the theory, based on sex

Table 3.

The frequency of students’ answers toward validated sources used by teacher, based on sex

Some of the teachers’ characteristics played a pivotal role in the selection, from the students’ point of view, such as, having self-confidence, teacher's attitude toward a lesson and teacher's interest toward the course; meanwhile, the teacher's university rank was less prioritized. In this manner, only 81 (29.3%) of the students were prone to describe it as very important, although, 76 (27.5%) subjects evaluated it as not important [Table 1].

In terms of communication skills, having a sincere relationship with the students, being tolerant toward the students’ manner, continuing with the connection, and being aware of the students’ names, were the significant factors, respectively. There was a significant difference between the genders with regard to having a sincere relationship [Table 4].

Table 4.

The frequency of student's answers toward the teaching quality assessment form, importance based on communication skill, addressing educational regulations, and assessment tact

According to educational principle of respect, on time starting and termination of the class were prioritized as first, 128 (46.4%) students assessed it as very important for teacher evaluation, and there was a statistically significant relationship between the academic status groups (P < 0.05). Other factors considered less important included respect to educational regulations, forcing students to consider educational laws, and roll call, respectively.

According to the evaluation skills, taking the final examination, was ranked first, because 115 (41.7%) of the subjects evaluated its importance as high; furthermore, there was a significant difference between the two genders and the academic status groups. All the students who were ranked 18 to 20 on an average, as well as 21 (75%) students ranked under 14 suggested that holding a final test would be very essential; there was also a significant relationship between these groups. Moreover; female students paid more attention to this factor.

All the students who averaged 18 to 20, as well as 23 (82.1%) of those who were under 14 announced that considering the exact start and termination time of class at the semester was very important. There was a statistical significant relationship between these groups (P = 0.006), as well. All of the students who were ranked 18 to 20, as well as 22 (78%) of those who were under 14, suggested that providing a suitable thinking situation for students toward lesson would be significantly essential. There was also a significant relationship between these groups (P = 0.006).

DISCUSSION

A two-stage, cross-sectional study was done, and 290 students were participated from the Lorestan University of Medical Science. The aim of this study was to designing and evaluating of the teachers teaching quality form. In terms of teaching skills, teacher mastery was ranked first, and students with a higher average emphatically pointed out that this was the most important trait. Male students accentuated more on the validated source used by the teacher for teaching. Some of the teachers’ characteristics were prioritized, like having self-confidence, making friends with students, taking a final test at the end of the semester, and start and end at the exact time of class, which was considered to be the first. In other studies, unfortunately, some of the suitable and attuned results were not accessible. In other similar surveys, an assessment form was used, but the factors considered were not extracted from the students.[19] Designing a commensurate evaluation framework for a teacher is dramatically significant. One of methods that is commonly implemented in all Universities is teacher evaluation by students, using a questionnaire.[19] Evaluation would be beneficial for revising programs and performances if implemented properly, but it would be inappropriate if not used properly. One of the critical principles for investigation, in brief, is importance of the content and questions.

Teachers’ adequate knowledge about the course was described as the most crucial factor in teaching skills, and 92% of the individuals delineated that it played a great role in the teacher assessment process. In Raoufie et al.'s investigation, from the students’ point of view, being eligible to manage the class is one of the essential components of the theoretical teaching quality form.[19] A scientifically qualified teacher is considered to be of high-range importance; however, a teacher who endeavors to teach students and has fluency is described as important by the students. Although teachers are theoretically eligible, some are likely to be negatively classified if they do not have fluency. Therefore, a teacher must be scientifically qualified as well as have fluency and use proper teaching methods. Ghorbani et al.'s study accentuates that some of most overriding teacher traits mentioned in the nursing College of Semnan University of Medical Science are the teacher's mastery of the theory, fluency, lesson organizing method, and a liking to learn, respectively.[20] In the aforementioned study, there is no significant relationship among the students of different courses. Dr Moezzie's survey also shows that there is no meaningful difference among different courses, with regard to the appropriate communication between teachers and students, teacher's mastery of the theory, fluency, and being eligible to teach, teacher using updated sources to teach, teacher's social behavior with student, teacher's positive effect on the student, and accepting recommendation and criticism.[21] Results showed that with regard to training skills, allocating time to answer student's questions and validated content to teach came second.

In the Vakili survey, many of the students described that paying time to problem-solving and student's questions played an important role in the teacher evaluation score.[22] A teacher who allocated enough time to deal with the students, would not be ignored by the subjects, and consequently this would be beneficial for teacher evaluation. Meanwhile, students considered teacher stringency and examinations as effective factors for reduced evaluation score for teachers, which was in accordance with Amini et al.'s investigation.[23] In the Amini study, there was a statistically significant relationship between the two genders; besides with findings from other studies, the results delineated that age, sex, ethnicity, and nationality were effective agents for evaluation.[23,24] Students allocated higher evaluation scores to teachers highlighted by characteristics of being sincere, extroverted, and had an interest in teaching.[25]

Updated lessons teaching was considerably significant and effective in the teacher evaluation process. These findings suggested that presenting new scientific information about lessons was a pivotal variable, which should be considered as important by the teacher. Having self-confidence, teacher attitude, and interest in the course, with relevance to the teachers’ personal characteristics, were the respective determinants. In this hierarchy, teacher executive rank was also classified as the least important variable in the Vakili investigation; a vast majority of students described the same variables of personality, self-confidence, and mellowness, as well.[22] According to the aforementioned sentences, teacher self-confidence and personality, as well as teaching skills, were the main factors to faculties evaluation. In the present study, meticulous starting and termination of class time, in terms of respect to educational regulations, was taken as very important by 46.4% of the students. The Navabi and Vakili surveys, similarly, showed that 60.4% of the students went with the same findings.[22,26]

Concerning communication skills, having a sincere relationship with students, flexibility toward student behavior, making communication continuously with students, and knowing and calling students with name were ranged in the importance priority, respectively. In the Amini investigation, 76% of the subjects stated that the teacher's communication skills were important for assessment.[23] Dr. Seif et al.'s, study showed that teacher evaluation seemed less likely to be affected by teaching quality and students learning ability, in contrast, it seemed more likely to be changed by the teacher's methods and procedures.[27] The other study pointed out that there was 75% coefficient due to implementing the evaluation from the students’ point of view, with regard to the popularity surveys form, allocated to favor teachers and the quality of education.[27]

It seems that not only teachers training methods, but also their personality and manner in the classroom can be mentioned as crucial factors for increasing the motivation about learning and promoting the quality of education. In this regard, most of the volume learning can be provided by motivated teachers,[28] 76% of the teachers believe that teacher morality, personality, and performance are greatly highlighted by students, rather than teacher methods and theories — a survey conducted in Jahrom showed this.[29] Teachers think that students are not eligible to judge, due to their lack of adequate knowledge about the training process, thus, student's evaluation is not acceptable,[30] and some teachers also believe that student's personal ideas may have biased the assessment process.[29]

According to the mentioned studies and the Dankin opinion, teacher assessment through students is not usually authorized to be a base for decision-making (promotion, eulogy, punishment, or selecting the eligible teacher).[31] Implementing other evaluation methods, especially those that are determinants for learning ability (the learning rate) and following the teachers’ teaching and education seems beneficial. Singapore Universities experience that the terms of designing a new evaluation method based on ability, for determining the minimum and maximum ability of a teacher can be useful.[32]

CONCLUSION

Students had diagnosed the main factors of teacher evaluation, appropriately; however, factors of sex, academic status, academic course, and term were determinant variables that could play an important role in the teacher evaluation process from the students’ opinion. According to the students’ evaluation, and for importance in terms of increase in the quality and improvement in the education process in Universities, we propose that attuned comprehensive planning, with elite teachers and students from all Universities, come together and design a teacher evaluation form, and alternative methods be executed till functional evaluation of the teacher educational practice is facilitated.

Limitations

Lack of attention about prioritizing the teachers’ evaluation factors, whether the students are native or not, and also, lacks of consideration of the teachers’ ideas toward the gathered students’ opinions, can be mentioned as some of the limitations of the current study.