Evaluating the Quality of Education at Dentistry School of Tehran University of Medical Sciences

Abstract

Background:

Educational evaluation is a process which deals with data collection and assessment of academic activities’ progress. In this research, educational evaluation of Dentistry School of Tehran University of Medical Sciences, which trains students in undergraduate and residency courses, was studied.

Methods:

This descriptive study was done with a model of educational evaluation in ten steps and 13 fields including purposes and mission objectives, management and organization, academic board members, students, human resources and support, educational, research, health and treatment spaces, educational, diagnostic, research and laboratory tools, educational, research, health and treatment programs and courses, process of teaching and learning, evaluation and assessment, alumni, and patients satisfaction. Data were collected using observation, interviews, questionnaires, and checklists.

Results:

Results of the study were mainly qualitative and in some cases quantitative, based on defined optimal situation. The total mean of qualitative results of educational evaluation of dentistry school in all 13 fields was 55.98% which is relatively desirable. In the case of quantitative ones, results of some fields such as treatment quality of patients and education and learning of the students were relatively desirable (61.32% and 60.16% respectively).

Conclusion:

According to the results, educational goals and missions, educational and research facilities and spaces which were identified as the weakest areas need to be considered and paid more serious attention.

Introduction

Educational evaluation (EE) is a formal process performed to evaluate the quality of effectiveness and/or value of a program, process, goal or curriculum.¹,² It deals with data collection and assessment of the progress of academic programs.³,⁴ By considering some principles related to educational measurements and data collection, EE may result in a better understanding of such programs.^5–7 During the past thirty years, theorists have presented numerous methods of evaluation. Worthen and Sanders² mentioned that more than 50 different evaluation approaches has been developed in recent decades. Among these, methods based on internal criteria are known as the ones that can interpret the scientific, educational, and therapeutic authenticity of different educational groups.⁴,⁸ This is greatly welcomed by the academic community and is widely spread to all universities in the world. That is because this method provided a scientific, appropriate, precise, timely, and valid basis regarding the interpretation of decision making system quality and programming for its promotion and development.³ Such a method was successfully carried out in four medical education groups at Supervisory and Expansion of Medical Education Council Secretariat of ministry of Health, Treatment and Medical Education of Iran in 1995.⁸ EE has its most effect, value, and results when it can provide needed information to individuals which are directly related, as well as those who may be benefited from its results.³,⁴,⁶

Educating dental professionals consists of theoretical and practical (clinical, paraclinical, and laboratory) courses, differing in duration, and educational curriculum among different countries. It might vary from 4 years (e.g., in India, Turkey, and Russia) to 6 years or more (e.g., in Iran consisting of 2 years of only basic medical sciences and 4+ years of dentistry courses). Due to numerous practical educational units in dentistry education and with regard to expensive but very critical protocols for infection control, a great deal of resources is consumed in governmental universities of Iran over training every general practitioner with a degree of “Doctorate of dental surgery”. On the other hand, the quality of dental services plays an important role in public health. Considering these issues, dentistry education needs to be cost-effective in terms of optimizing its quality. To date, the pace of advancements in dentistry necessitates a continuous revision of educational programs by officials to achieve new expectations of educational system and determine or update the policies.⁹ Proper evaluation and research in education are accounted as scientific instruments for moving along with these developments in order to achieve improvements in education, health, and treatment qualities.^9–11 Such advancements may depend on education of faculty members and panels and their cooperation, elimi-nation of shortcomings, and approximation of components and educational instruments to standard indices.⁹ Aim and mission of this educational groups are training and tutoring students of undergraduate and residency courses in order to gain complete ability for diagnose and treatment of patients needing this kind of treatments, so that residents can provide treatment and disease prevention services with a desirable quality after learning these courses.¹⁰ This study was conducted in year 2010 to assess the efficacy of educational programs provided at Dental School of Tehran University of Medical Sciences to estimate and address the limitations and strengths.

Materials and Methods

Apart from educational groups dedicated to deliver basic medical sciences to dental students, Dentistry School of Tehran University of Medical Sciences includes twelve only-dentistry educational groups (Table 1). The objective of this descriptive cross-sectional study was to assess the quality of education provided at this school, compared with the standards. The following aims would be fulfilled: 1) Determining the educational and research needs of group directors; 2) Determining those of other faculty members; 3) Defining such needs of under- and post-graduate students; 4) Surveying attitudes of graduated doctors; 5) Surveying regarding research and educational facilities as well as residency resources; 6) Determining educational procedures; 7) surveying patients regarding their satisfaction from received dental cares; 8) Distinguishing human resources; and 9) determining poor fields, needing to be pushed.

Table 1.

Dentistry School of Tehran University of Medical Sciences educational groups in 2010

Through this descriptive cross-sectional study, the efficacy of provided education by mentioned 12 departments were assessed in 13 fields. Research society, educational groups including group director and sub-societies which include faculty teachers, students, alumni, human, and support recourses were completely involved in this survey. These included aims and missions of groups, management and organization, scientific board, students, human resources and support, educational, research, health and treatment spaces, educational, diagnostic, research, and laboratory instruments, educational, research, health, and treatment programs, process of teaching and learning, evaluation and assessment, and satisfaction of alumni and patients. Each field was evaluated through the following steps: 1) Establishment of standards; 2) Data collection; 3) Determining the importance of components; and 4) Analyzing the collected data.

Data were collected using interview, inspection, two checklists, and eleven questionnaires. Inspection, interview, and checklists were to evaluate educational, research, health, and treatment spaces of dentistry sections and educational, research, laboratory and diagnostic instruments.

First questionnaire was used to gather the feedback of panel director and faculty scientific members to establish the coefficient of desired factors. This questionnaire had 13 fields. Another questionnaire was utilized to evaluate 11 fields by multiple-choice questions based on a 5-point Likert scale. Instruments, factors, and measurement sources are summarized in Table 2.

Table 2.

Measurement instruments, evaluated factors and data collection sources

We tried to establish study variables in line with objectives and research questions in providing data collection instruments. For this reason, before preparation of data collecting instrument, a table was created that precisely identified each research question's variables and based on that the instruments were generated. Afterward, in order to increase the validity, questionnaires were reviewed by experts and the straight and vague questions were addressed.¹²

Considering the reliability of data collecting instrument, after preparation of questionnaires according to arranged subjects, confusions about some questions were identified and removed, taking the use of a pilot study in a 15-individual group and interviews with academic board members in educational groups. Eventually, the final data collecting instrument was designed. In order to determine the coefficient of each of the 13 criteria, feedback forms were utilized. Besides, through interviews and Delphi technique the academic board members’ opinions were recorded and the importance of each criterion was determined. Results indicated that all academic board members have given equal coefficient values to all questions. After determination of evaluated factors and sources of gathering related data, criteria for each factor were designated (e.g., an evaluated factor was management and organization of group, and a criterion related to this factor was group director). A marker was created for each of these criteria.

In order to perform this, a) specificities of desirable condition were described; b) a marker of desirable condition was set, c) for assessment of goal achievement, the criteria condition was compared to the desirable condition marker.

In order to analyze Likert multiple-choice questions, scores 1 to 5 were respectively assigned to highest and lowest scores. Utility rate was determined by the percentage of the related index. To facilitate the assessment of components and evaluated factors, the desirability level of each factor was classified based on the score percentage: Desirable, more than 75%; Relatively desirable, 50-75%; and Not desirable, less than 50% (Table 3).

Table 3.

Average educational evaluation results of educational groups in dentistry school of Tehran University of Medical Sciences in 2010

Descriptive statistics were calculated using SPSS₁₀. EE results were analyzed based on SWOTS (strengths, weaknesses, opportunity, and threats) method in evaluated educational groups.

Results

Educational evaluation committee accepted fluency and clarity of the missions and aims of educational groups in undergraduate and graduate courses. Revision of aims was suggested in three areas of knowledge, attitude, and practice. The quality of departments’ educational systems were relatively desirable (55.98) in all 13 fields. For management fields, average results were relatively desirable (52.9%) based on specificities of desirable condition. In this field, 72% of group members believed that group managers have acceptable scientific and educational background, 55% were aware of group manager selection criteria and 90% of academic field members were satisfied with the organization and management abilities of the group manager. In evaluation on academic board, mean age of educational group members was 43.9 ± 3.2 years old and mean teaching background was 14.4 ±3 .9 years. Most of academic board members were men and all of them were assistant professors and officially occupied.

80% of academic board members were involved in research projects; more than 80% were mentoring specialty course thesis; and 50% were satisfied with workspace condition. An important specificity of this group was that they were involved in programming of theoretical and applied education of students in undergraduate and residency courses. Mean EE result in academic board field was relatively desirable (56.92%).

In the field of learners, all residents were asked about their association, correlation with academic board, study duration, and educational, research activities, and student projects. This field was relatively desirable (53.89%).

Desirability in the field of human resources and support in sections educational, treatment, research, audiovisual, library, diagnostic laboratory, radiology, and facilities was 54.58%, considered relatively desirable. And in both fields of educational, research, health, and treatment spaces and educational, research, diagnostic, and laboratory facilities EE results were relatively desirable (50.36% and 51.55% respectively).

In the field of educational, research, teaching process and learning courses, nearly all academic board members (91%) were involved in educational programming of the group. 82% of group members believed that aims and mission of educational courses in groups were of their interests.

All group members believed that students had regularly attended the educational programs. Most of the group members (91%) believed that Cultural Revolution Committee headings were applied in educational programming and 82% of them were involved in both basic and clinical teachings. These fields had the average of 58.09% and 60.16% respectively and considered relatively desirable.

In the field of graduates, mean age of individuals which have entered the university in year 1997 and graduated in year 2003 was 26.3 and 80% were men. Considering the service condition, most graduates (66.7%) were spending their duty project. Regarding their satisfaction, this field was in a relatively desirable (58.72%) condition.

In the field of patients, mean age of service takers was 24.8 ± 7.6 years. More than 63% of patients of general (undergraduate) ward and 73% of patient which referred to residency wards stated that their reason for selecting this university was that they trusted in the precise and effective treatment delivered by the students at this university. After comparison of results to the desirability criteria regarding the trust of service takers to precise treatment and a good referring size of the ward, this fields’ condition was considered relatively desirable (61.32%).

General results of the performed evaluation are displayed in Figure 1. The evaluated educational groups have given suggestions about quality improvement in education, research, health, and treatment in four levels (group, faculty, university, and ministry) considering their strength and weaknesses.¹⁰

Figure 1.

The mean of educational evaluation in departments of educational school of dentistry Tehran University of Medical Sciences, 2010.

Discussion

EE can study and assess the educational programs utilizing standards, predetermined aims, or educational quality²,³,⁶,⁷,¹⁰ After such assessments, it is possible to address these shortages in educational system and arrange an efficient educational system.¹,¹³,¹⁴ EE of educational programs is an important and basic task of medical universities.¹ Rate of occupational capabilities and medical alumni performances in order to offer educational and research programs, health and treatment services with the aim of supplying and improving society members’ health is related to the rate of educational programs’ realization.¹ If educational programs are not well designed and performed, it can impose irreparable damage and harmful social, economical, and cultural effects on individuals, society, alumni, and also faculty and finally university's credit.¹ Considering the importance of education and its major role in improving nearly all aspects of societies, educational programs should be carefully probed to elucidate shortcomings and advantages in order to improve programs’ efficacy.¹⁵ The quality of education relates to the knowledge and experience of teachers. As well, teaching methods such as workshops, seminars, presentations, interactive teaching methods which may involve the students and thus increase their learned topics and introducing outlines of the lessons in the first place.¹⁶ However, unfortunately most teachers are not trained regarding teaching techniques and are not completely ready to undertake educational responsibilities.¹⁷,¹⁸ An effective way of improving this shortcoming is conducting courses in teaching techniques and skills.¹⁹ Evaluating the draw backs, preferences, and priorities of teachers’ teaching skills, as well as other educational shortcomings may enable the program directors of such courses to canalize the materials to which is more needed by teachers and educational groups.²⁰ Universities may determine their position in national and international levels to further improve their programs.²¹,²² For example, India, as a country with the highest number of medical universities and thus the most number of medical faculty members, has developed such programs called National Teachers Training Centers (NTTC). These programs train teachers to gain skills, and to find certain teachers as group directors and leader, in long-term. Such programs are globally called “faculty development”.²³

The overall EE findings in educational groups were relatively desirable (mean score = 55.98%). Based on the findings, activities of group managers, educational management, and academic board members in these groups were performed in order to improve the procedure of assessment. Results of other national studies shows that school of medicine,¹³ school of nursing and midwifery,²⁴ and school of rehabilitation²⁵ had average 75.3%, 80.4% and 77.8%, respectively and the quality of education, research, and treatment were desirable. Therefore, the Dentistry School showed poorer results.

Applying EE in Iranian educational system was started with implementation of a pilot EE study in six educational groups in 1996.²⁶ Results showed that EE in Iranian culture would lead to improvement.²⁶ 30 national medical educational groups in medical universities of the country implemented the EE project.²⁷ Farzianpour and Bazargan in 1999 revealed that EE is the best measurement index for evaluation of university hospitals.²⁷ In 2004, fifteen basic sciences and clinical educational groups of Tehran University of Medical Sciences have reported their EE results desirable.²⁸ Saberian et al²⁹ from school of midwifery and nursing of Semnan described results of EE in surgery ward desirable in international congress of educational evaluation in 2004 which was held in Edinburgh. Olyaei et al¹⁴ from rehabilitation school noted that the results of their EE was 76.2% and was desirable. Farzianpour et al²⁸ from Tehran University of Medical Sciences, and Harden et al³⁰ from Dundee University of England had positive attitudes towards EE and improvement of education and research in clinical fields. Farzainapour et al²⁸ have reported EE in 15 educational groups 71% and desirable. Researches in world's educational system report that EE is an effective way to find out the strengths and weaknesses of an educational system.¹⁴,^27–34 Universities might follow standards of education evaluation so that the result could be better comparable. Until then, a comprehensive comparison is difficult. However, diverse techniques have been described for educational evaluation.³⁵ These included expertise-oriented, management-oriented, and objectives-oriented.³⁵ Kirkpatrick³⁶ has introduced a 4-step assessment. The first stage would assess the reactions of instructors to the program. Then learners’ skill and knowledge would be assessed. Afterward the application of theoretical knowledge of learners in their practice would be assessed. Finally the impact of program on the institution and community would be evaluated. Until conducting a unified method, international comparisons would be difficult to perform.

Conclusion

The Dental School showed relatively appropriate results regarding the quality of education. This was lower than that of other evaluated schools. Quality of patient care and students’ learning were the best fields (respectively 61% and 60%). Educational aims and objectives, and research and educational spaces had the poorest results (respectively 49% and 50%). Comparing to other schools, Dentistry School needs an overall attention. However, this should be started from the poorest fields.