Abstract
Introduction:
In recent years curriculum reform and integration was done in many medical schools. The integrated curriculum is a popular concept all over the world. In Shiraz medical school, the reform was initiated by stablishing the horizontal basic science integration model and Early Clinical Exposure (ECE) for undergraduate medical education. The purpose of this study was to provide the required data for the program evaluation of this curriculum for undergraduate medical students, using CIPP program evaluation model.
Methods:
This study is an analytic descriptive and triangulation mixed method study which was carried out in Shiraz Medical School in 2012, based on the views of professors of basic sciences courses and first and second year medical students. The study evaluated the quality of the relationship between basic sciences and clinical courses and the method of presenting such courses based on the Context, Input, Process and Product (CIPP) model. The tools for collecting data, both quantitatively and qualitatively, were some questionnaires, content analysis of portfolios, semi- structured interview and brain storming sessions. For quantitative data analysis, SPSS software, version 14, was used.
Results:
In the context evaluation by modified DREEM questionnaire, 77.75%of the students believed that this educational system encourages them to actively participate in classes. Course schedule and atmosphere of class were reported suitable by 87.81% and 83.86% of students. In input domain that was measured by a researcher made questionnaire, the facilities for education were acceptable except for shortage of cadavers. In process evaluation, the quality of integrated modules presentation and Early Clinical Exposure (ECE) was good from the students’ viewpoint. In product evaluation, students’ brain storming, students’ portfolio and semi-structured interview with faculties were done, showing some positive aspects of integration and some areas that need improvement.
Conclusion:
The main advantage of assessing an educational program based on CIPP evaluation model is that the context, input, process and product of the program are viewed and evaluated systematically. This will help the educational authorities to make proper decisions based on the weaknesses and strengths of the program on its continuation, cessation and revision. Based on the results of this study, the integrated basic sciences course for undergraduate medical students in Shiraz Medical School is at a desirable level. However, attempts to improve or reform some sections and continual evaluation of the program and its accreditation seem to be necessary.
Keywords: Integration , Medical students , Basic science , Curriculum
Introduction
In recent years, curriculum reform and integration was reported in many medical schools. The integrated curriculum is a popular concept in medical schools all over the world. The main aim of integration is to break the boundaries between basic and clinical sciences. Integration supports maintenance of knowledge by development of thought and the application of concepts (1).
In Shiraz medical school, the integrated program was initiated by establishing a central curriculum committee for revision of basic sciences courses in 2007 (2). Considering different types of integration models, analyzing the situation, doing library-based studies, studying the viewpoints of defenders and detractors and after a visit some professors from Canada, Britain, Malaysia and Singapore universities, the horizontal integration model was confirmed and Early Clinical Exposure (ECE) for vertical integration was considered (3).
For program evaluation, the Context, Input, Process and Product (CIPP model) was used to measure the difficult features of medical education programs and provide formative evaluation data to stakeholders with the aim of improvement of the program (4-6). The first component, context evaluation, is useful when an established program is going through a planned change or must adjust to the conditions changed. The second part, input, helps to establish an appropriate program model to assure the identified needs. Process evaluation provides formative data for guiding revisions whilst the program is running. The last component, product evaluation, produces valuable information in order to judge program outcomes (7,8).
This article elaborated the use of the CIPP evaluation model as a comprehensive framework to help to initiate, develop, install, and evaluate a new undergraduate medical education program in a medical school in the south of Iran.
As the CIPP model is based on this concept that the most important aim of assessment is to improve and modify but not to confirm a plan, in this paper we aimed to provide some data for decision makers to improve the quality of the integrated medical curriculum, using four parameters (context, input, process and product).
Methods
Study context
This study was done in Shiraz Medical School and Education Development Center of Shiraz University of Medical Sciences. Shiraz Medical School is one of the largest and oldest schools in the south of Iran. The program of this school was composed of two and half years of integrated basic sciences, one year of pathophysiology of diseases, a one-year externship, and 18-month internship. The program contains Early Clinical Exposure (ECE) for integrated basic sciences curriculum.
Data collection
The present study is a triangulation mixed method study carried out in Shiraz Medical School in 2012 to look into the quality of the integrated basic sciences course curriculum for undergraduate medical students based on the views of basic sciences professors and first and second medical students, using CIPP evaluation model. In this study, both qualitative and quantitative tools were used. Table 1 shows four complementary sets of evaluation phases which were compatible with the four components of the CIPP evaluation model.
Table 1.
Components of CIPP evaluation model for evaluation of integrated basic sciences curriculum in Shiraz University of Medical Sciences
| Domain under study | Variables measured | Tools | Target group | |
|---|---|---|---|---|
| Qualitative | Quantitative | |||
| Context | Review of current evidence on integration and consultation with experts | Modified DREEM questionnaire | Students, faculty and administrators | |
| Input | Content of curriculum | - | A researcher made questionnaire | Faculties and curriculum committee |
| Process | The process of learning | - | 1-Researcher-made questionnaires for evaluating the quality of each integrated course | First and second year medical students |
| Process of teaching | - | 2-Researcher-made questionnaires for evaluating the quality of ECE | ||
| Product | Students’ Performance The process of teaching and learning | 1-Learner centered integrated basic science portfolios | - | First and second year medical students |
| 2-Brainstorming (students) | ||||
| 3- Semi- structured interview (professors of basic sciences) | Professors of basic sciences | |||
Phase 1: Context
In order to understand the need for change, we conducted a complete context evaluation in 2012. The projects included review of the existent literature on integration, consultation with experts in curriculum design and assessment of the educational environment from the students’ perspectives, using the modified Dundee Ready Education Environment Measure (DREEM) (2,9).
Phase 2: Input
A researcher made questionnaire based on the five point Likert scale was used for input evaluation. The questionnaire consists of some questions about facilities in laboratories, available equipment, etc. The content validity of the questionnaire was determined by medical education experts. Reliability of the questionnaire was determined after a pilot study (r=0.83).
Phase 3: Process
A researcher made questionnaire based on the five point Likert scale was used to evaluate the quality of 9 integrated modules including the genital system, digestive system, skeletal system, nervous system, endocrine system, visual and auditory systems, cardiovascular system, renal system, and pulmonary system. The validity and reliability of the questionnaires are determined in our previous study (2).
A researcher made questionnaire was used to assess the quality of the ECE, containing 7 questions on the viewpoints of first-year medical students. The validity and reliability of the questionnaire were confirmed in our previous study (2).
Phase 4: Product
For measuring the product, we designed a portfolio for assessing the students’ learning and reflection. The students reported their reflection in their portfolio and also brainstorming sessions with students was done to obtain their view and pros and cons about integration systems. Another method used for process evaluation was the semi-structured interview with basic sciences faculties. These semi-structured interviews about integration were done by an expert researcher through the conversational style. The view of basic science was gathered. Some of the samples of the interview questions are mentioned below:
To what extent was the integration of clinical cases in the courses of basic sciences effective? How can the integration plan be improved? Was the volume of course content decreased in your department? If yes, how and if no, what is your suggestion? And what is your general opinion about the integration phases and your suggestions for further changes?
Statistical analysis
The statistical analysis was done on quantitative data using SPSS software, version 14. For analysis of qualitative data, content analysis method was used. For trustworthiness of data in the qualitative section, member check and peer check were done.
Results
Context Evaluation
In this part, reviewing the current evidence on integration and consultation with experts were done. Also, the modified DREEM questionnaire was used to measure the educational environment. Table 2 shows the results of modified DREEM questionnaire (Table 2).
Table 2.
Students’ viewpoints about the educational setting
| No. | Items | Percent of students that are agree |
|---|---|---|
| 1 | This educational system encourages me in active class participation. | 77.75 |
| 2 | Teachers are knowledgeable enough to teach courses in the integration program. | 35.91 |
| 3 | Teachers value the students’ viewpoints. | 42.73 |
| 4 | Presentation of the materials in the integration system results in my progress now and in future. | 43.62 |
| 5 | Courses are well scheduled in the integration system | 87.81 |
| 6 | Examination time is well scheduled. | 79.12 |
| 7 | Manner of presentation and arrangement of the integrated courses are boring. | 5.10 |
| 8 | In this system, teachers establish a good relationship with the students. | 42.11 |
| 9 | Education in this system reinforces my self-confidence. | 45.51 |
| 10 | For the education of each course in this system, a specific time period is assigned. | 51.54 |
| 11 | This education system emphasizes real learning. | 66. 22 |
| 12 | I can put most of the materials in my long-term memory. | 62.76 |
| 13 | Professor advisors provide the students with proper education feedback. | 58.69 |
| 14 | The integration system has made me more interested in education. | 46.29 |
| 15 | The office of vice-chancellor for education has provided a proper situation for constructive criticism. | 76.72 |
| 16 | Teachers provide proper situations for constructive criticism. | 45.41 |
| 17 | Teachers give clear examples for learners’ better understanding. | 55.57 |
| 18. | Learning objectives in the integration system are clear for me. | 35.28 |
| 19 | Teachers precede class sessions based on educational objectives. | 63.56 |
| 20 | Teachers are dissatisfied with this education system. | 31.21 |
| 21 | Despite the stresses in the new system, I enjoy more when participating in it. | 67.72 |
| 22 | The atmosphere of the class sessions motivates me more as a learner | 83.86 |
| 23 | The educational setting is favorable. | 54.81 |
| 24 | Long-term learning is more valued than short-term learning. | 61.22 |
| 25 | Education is more teacher-oriented. | 19.11 |
Input Evaluation:
The result of input evaluation is shown in Table 3.
Table 3.
The result of input evaluation
| Variable | Number of responses | Mean±SD (from 5) |
|---|---|---|
| Faculties interest in teaching | 157 | 3.63±1.14 |
| Competent instructors | 157 | 3.26±1.87 |
| Complete teaching facilities in classrooms | 157 | 3.46±1.84 |
| Capability of laboratories | 157 | 3.10±1.78 |
| Sufficient facilities in libraries | 157 | 3.19±1.81 |
| Allocation of budget and financial resources | 157 | 3.25±1.12 |
| Sufficient cadavers | 157 | 1.95±1.19 |
| Need assessment of modules | 157 | 3.89±1.22 |
| Training program for faculties before stating the courses | 157 | 3.65±1.06 |
| The length of basic science education | 157 | 1.84±.98 |
Process Evaluation
Process of learning- teaching the ECE with patient:
The results of students’ views about early clinical exposure sessions are shown in Figure 1.
Figure1.
Students’ viewpoints about the quality of Early Clinical Exposure program
The results of quality of education of each integrated module:
Table 4 shows a sample of the results of the quality of education in the digestive system from the students’ viewpoint (Table 4).
Table 4.
The results of quality of digestive system integrated module
| Variable | Number of respondents | Min | Max | Mean±SD (From 5) |
|---|---|---|---|---|
| Introduction to Anatomy and physiology of abdomen | 136 | 3 | 5 | 3.83±1.97 |
| Nerve roots, veins, arteries in abdomen | 136 | 3 | 5 | 4.16±1.69 |
| Integrated anatomy and physiology of Inguinal canal | 136 | 3 | 4 | 3.58±1.10 |
| Integrated anatomy and physiology of liver | 136 | 3 | 5 | 4.19±1.78 |
| Integrated anatomy and physiology of GI tract | 136 | 2 | 5 | 3.66±1.75 |
| Integrated anatomy and physiology of spleen | 136 | 3 | 5 | 4.21±1.15 |
| Integrated anatomy and physiology of nervous system in GI tract | 136 | 2 | 4 | 3.27±1.32 |
Product Evaluation:
Brainstorming sessions
Discussion at brainstorming session with students focused on 2 primary areas: determining the positive and negative points of the integrated curriculum and defining the necessary changes needed within this integrated curriculum to address the future demands of medical students.
Some of the positive and negative points expressed by students are mentioned below:
Positive points
Drawing the attention of students toward basic sciences
Active and in depth learning
Better application of theoretical knowledge in basic sciences
More joy from studying basic sciences
Relevance of presented topics to each other
Negative points
▪ Curriculum overload
▪ Confusion and stress about new integrated modules
The necessary changes ascertained by the students are:
* More cooperation between faculties and students
* Consistent course template to facilitate access to content
* Advanced study guides for each integrated model
* Technology enhanced teaching and learning facilities
* Alternative education materials such as a well defined course syllabus
Educational portfolio
One instrument that was designed in this study for evaluating the product was a learner centered integrated basic science portfolio. Students are expected to complete the portfolio and especially mention their educational activities and their reflection. Most students mentioned their satisfaction of learning in integrated curriculum in their portfolio and believed that the process of learning in this new curriculum is better and they are able to learn the materials of different disciplines as a whole. Students ascertained that consolidation experiences such as using problems, quizzes and formative assessments help them to solidify their understanding and improve their learning.
Semi structured interview with faculties
Semi-structured interview was done with basic science faculties that were responsible for teaching and learning. The following results were obtained:
Positive results of integration:
o After implementation of horizontal integration, the amount of duplicate material in different disciplines decreased.
o To some extent, separating the core curriculum was done in some modules.
o Faculties from different disciplines are aware about their colleagues’ activities.
The areas that needs improvement in the faculties’ viewpoint:
* Basic sciences comprehensive exam should be redesigned based on the integrated modules and Early Clinical Exposure program.
* There should be an opportunity to teach some topics at patient bed side.
* The volume of curriculum should be lowered more.
Discussion
The major aim of this study was to evaluate the integrated curriculum for basic sciences in undergraduate medical students in Shiraz Medical School, based on the CIPP evaluation model. The CIPP is a useful model for stakeholders for evaluating their educational program.
The results of the present study showed that in content evaluation, educational environments in Shiraz Medical School are satisfying in comparison to our previous studies (2,10). The results of a study on evaluation of integrated curriculum in Tehran University of Medical Sciences showed that there was no significant difference between traditional and integrated curriculum in overall DREEM questionnaire, but in some domains students in the integrated curriculum evaluated the environment better than those in the traditional curriculum (8).
In the input evaluation
The duration of training period was mentioned as the most important obstacle that was similar to other published article in Iran (11,12). It seems necessary that the duration of basic sciences should be decreased to 2 years.
As to facilities, the shortage of cadavers for education was mentioned as a problem in Shiraz Medical School. In recent years, it has been recognized that anatomy laboratory education using cadavers for teaching human morphology and also some aspects of professionalism such as dealing with death should be emphasized in the curriculum (13).
In the process evaluation
We used a valid and reliable researcher made questionnaires based on the Likert scale for each course. As compared to our previous study, the results of the quality of integrated courses are improved over the time (2).
Another questionnaire used for process evaluation in this study was the Early Clinical Exposure (ECE) questionnaire that showed improvement in these sessions in comparison to our previous studies (2,3). Similar results were reported from other published articles (14-17).
In the product evaluation, we used the students’ brainstorming session and portfolios and semi- structured interview with basic science faculties. In comparison to our previous study, the results were better than before. Students and faculties are now more familiar with integration system in Shiraz Medical School (2). Well informed faculty members ensure the successful implementation of every curriculum. Regular communication between faculties is very necessary for the success of integration programs (18). Faculties also recommended changes in assessment methods, especially for comprehensive exams. One of the tips in integration program that is mentioned in Melik et al.’s study is selecting appropriate assessment method (18).
The results of students’ portfolios and brainstorming sessions showed that students believed that horizontal integration was positive and it seems that presentation of basic science in a simultaneous and integrated manner led to a better understanding of the subjects. The students ascertained the need for decreasing the amount of educational content and using new methods of teaching and learning in new curriculum. Sobral reported similar results (19).
Also, Brauer and Ferguson emphasized that integrated curricula should be designed to separate the core content from unnecessary content but should not diminish the importance and necessity of basic sciences (1).
Yamani et al. reported that the concepts of core curriculum and integration are closely linked together and indeed integration is one of the models of designing core curricula and both concepts address the need for creating meaningful learning (20).
This study was done because it was one of the important priorities in medical education research in Iran and Eastern Mediterranean Region (21,22). One of the limitations of our study was that data were collected based on the student and faculties’ viewpoint that may not show a precise description of situation. Another limitation was that we did not have control group for comparing the results.
Conclusion
The main advantage of assessing an educational program based on CIPP evaluation model is that the context, input, process and product of the program are viewed and evaluated systematically. This will help the educational authorities to make proper decisions based on the weaknesses and strengths of the program on its continuance, cessation and revision. Based on the results of this study, the integrated basic sciences course for undergraduate medical students in Shiraz Medical School is at a desirable level. However, attempts to improve or reform some sections and continue the evaluation of the program and its accreditation seem necessary. The following suggestions are required for improving the integrated program in future:
- Using a wider range of factors for evaluation in the domains of context, input, process and product.
- Using a wider range of interest groups, involved in the program evaluation such as top managers and executives, graduates, supervisors in the health system and the university staff, in order to evaluate the quality of the program more precisely.
- Evaluating the teaching-learning methods for each of the integrated courses based on the professors and students’ viewpoints, using a qualitative study.
- Designing the same standard assessment tool with usability to all universities is recommended to evaluate the integrated educational programs internationally which contributes to comparison of the quality of integrated educational programs in all universities with the standard.
Acknowledgement
This paper has been extracted from the thesis of the first author MS Degree, by registration number 6635-92. The authors acknowledge research deputy of Shiraz University of Medical Sciences for financial support, colleagues who cooperated with us in performing the study and students who participated in this study.
Conflict of Interest: None declared.
References
- 1.Brauer DG, Ferguson KJ. The integrated curriculum in medical education: AMEE Guide No. 96. Med Teach. 2015;37:312–22. doi: 10.3109/0142159X.2014.970998. [DOI] [PubMed] [Google Scholar]
- 2.Amini M, Kojuri J, Mahbudi A, Lotfi F, Seghatoleslam A, Karimian Z, Sham M. Implementation and evolution of the horizontal integration at shiraz medical school. JAMP. 2013; 1(1): 21–7. [Google Scholar]
- 3.Ebrahimi S, Kojuri J. Comparison of two educational environments in early clinical exposure program based on Dundee Ready Educational Environment Measure. JAMP. 2013; 1(1): 36–7. [Google Scholar]
- 4.Gandomkar R, Jalili M, Mirzazadeh A. Evaluating assessment programs using program evaluation models. Med Teach. 2015; 37:792–3. doi: 10.3109/0142159X.2015.1042436. [DOI] [PubMed] [Google Scholar]
- 5.Frye AW, Hemmer PA. Program evaluation models and related theories: AMEE Guide No. 67. Med Teach 2012; 34: e288–99. doi: 10.3109/0142159X.2012.668637. [DOI] [PubMed] [Google Scholar]
- 6.Mokhtarzadegan M, Amini M, Takmil F, Adamiat M, Sarveravan P. In service trainings for Shiraz University of Medical Sciences employees: effectiveness assessment by using the CIPP model. JAMP. 2015; 3(2): 77–83. [PMC free article] [PubMed] [Google Scholar]
- 7.Stufflebeam DL, Shinkfield AJ. Evaluation theory, models, and applications. Somerset: Wiley; 2007. [Google Scholar]
- 8.Mirzazadeh A, Gandomkar R, Mortaz Hejri S, Hassanzadeh GhR, Emadi Koochak H, Golestani A, et al. Undergraduate medical education programme renewal: a longitudinal context, input, process and product evaluation study. Perspect Med Educ. 2016; 5:15–23. doi: 10.1007/s40037-015-0243-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Miles S, Swift L, Leinster SJ. The Dundee Ready Education Environment Measure (DREEM): A review of its adoption and use. Med Teach. 2012; 34:227–36. doi: 10.3109/0142159X.2012.668625. [DOI] [PubMed] [Google Scholar]
- 10.Rezaee R, Shokrpour N. Performance Assessment of Academic Departments: CIPP Model. European Journal of Social Sciences. 2011; 23(2): 227–36. [Google Scholar]
- 11.Siabani S, Moradi MR, Siabani H, Rezaei M, Siabani S, Amolaee Kh, et al. Students view points on the educational problems in medical school of Kerman university of medical sciences, 2007. Behbood J. 2009; 13(2): 162–71. Persian. [Google Scholar]
- 12.Adibi I, Abedi Z, Memarzade M, Adibi P. Early Clinical Exposure: Report of a clinical trial. Iranian Journal of Medical Education. 2002; 1(5): 5–11. [Google Scholar]
- 13.Louw G, Eizenberg N, Carmichael SW. The place of anatomy in medical education: AMEE Guide no 41. Med Teach. 2009; 31: 373–86. doi: 10.1080/01421590902825149. [DOI] [PubMed] [Google Scholar]
- 14.Soltani Arabshahi K, Kouhpayezadeh J, Sobuti B. The Educational Environment of Main Clinical Wards in Educational Hospitals Affiliated to Iran University of Medical Sciences: Learners' Viewpoints Based on DREEM Model. Iranian Journal of Medical Education. 2008; 8(1): 43–9. Persian. [Google Scholar]
- 15.Vahidshahi K, Mahmoudi M, Ranjbar M, Shahbaznezhad L, Ehteshami S, Shafie S. The Effect of Early Clinical Experience on Attitude of Medical students towards basic science and clinical. Strides in Development of Medical Education. 2012; 8(1): 94–100. Persian. [Google Scholar]
- 16.Shirzad H, Moezzi M, Khadivi R, Sadeghi B, Madhkhan A. Effect of early clinical exposure on attitude and performance of first year medical students. JSKUMS. 2011; 13(1): 69–78. [Google Scholar]
- 17.Ahmadipour H, Hajmohammadi F. Horizontal integration in basic sciences at Kerman University of Medical Sciences: medical students’ viewpoint. Res Dev Med Educ. 2016; 5(2): 93–6. [Google Scholar]
- 18.Malik SA, Malik RH. Twelve tips for developing an integrated curriculum. Med Teach. 2011; 33(2):99–104. doi: 10.3109/0142159X.2010.507711. [DOI] [PubMed] [Google Scholar]
- 19.Sobral DT. Medical students' self-appraisal of first-year learning outcomes: use of the course valuing inventory. Med Teach. 2004; 26(3): 234–8. doi: 10.1080/0142159042000192028. [DOI] [PubMed] [Google Scholar]
- 20.Yamani N, Rahimi M. The core curriculum and integration in medical education. Res Dev Med Educ. 2016; 5(2): 50–4. [Google Scholar]
- 21.Nabeiei P, Amini M, Ghanavati SH, Marhamati S. Research priorities in medical education at Shiraz University of Medical Sciences: categories and subcategories in the Iranian context. JAMP. 2016; 4(1): 26–32. [PMC free article] [PubMed] [Google Scholar]
- 22.Amini M, Kojuri J, Lotfi F, Karimian Z, Abadi ASH. Research priorities in medical education in the Eastern Mediterranean Region. EMHJ. 2012; 18(7): 687–92. [PubMed] [Google Scholar]