Comparative assessment of students’ performance and perceptions on objective structured practical models in undergraduate pathology teaching

Abstract

INTRODUCTION

Assessment is an important factor that drives student learning, as students tend to mainly focus on the material to be assessed. The current practice in teaching pathology extensively applies objective-structured practical examination for the assessment of students. As students will have to deal with real patients during clinical years, it is preferred that students learn and practise via potted specimens and slides instead of picture plates. This study aimed to assess the preferred assesment method of pathology practical exercises.

METHODS

This was a cross-sectional survey carried out in two consecutive batches of Phase 2 medical students. Student competency was assessed using both the traditional (TD) (i.e. use of potted specimens and slides) and picture plate (PP) methods. To compare the two assessment methods, we compared the mean scores obtained by the students and examined student perception of the two methods.

RESULTS

The mean scores obtained via the PP method were significantly higher than those obtained via the TD method for almost all the components tested.

CONCLUSION

We found that students performed significantly better (p < 0.05) when assessed using the PP method instead of the TD method. PP preparations might provide better visuals, thus aiding understanding, than the TD method. The findings of this study are valuable in identifying and improving our current teaching and assessment methods of medical students, in line with advancements in information technology.

INTRODUCTION

Assessment is considered the most important driving factor of student learning,(1) as students tend to mainly focus on the material to be assessed.(2) Research shows that in the context of higher education (including medical education), the type of assessment method adopted can influence student learning.(3) As the subject of pathology bridges the gap between basic sciences and clinical medicine, a proper understanding of its processes is critical in medical practice. The main goal of teaching undergraduate pathology is to equip students with knowledge of the functional and structural changes in diseases, so that they are able to understand and interpret clinical signs and symptoms.(4) In most medical schools, practical exercises are an integral part of pathology courses, and the assessment of these practical exercises is greatly important. In the teaching of medical subjects, it is current practice to use extensively applied objective structured practical examination (OSPE) for the assessment of students.(5,6) As students have to deal with real patients during the clinical years, it is deemed preferable for them to learn and obtain practice through the use of potted specimens and slides prepared from real patients, rather than through picture plate (PP) preparations. However, significant changes have been occurring in medical school curricula, with many existing undergraduate programmes being reshaped, partly in response to the General Medical Council’s recommendations published in the circular, Tomorrow’s Doctors.(7) While the circular encouraged greater integration of pathology teaching and assessment with clinical subjects, the effects in assessing the knowledge, skills and attitudes of students were not well studied.(8) Unfortunately, limited published information is available on how pathology is taught in undergraduate medical degree courses.(9) Thus, the present study aimed to identify tools useful for improving our current way of teaching and assessing medical students. Medical students were assumed to prefer the traditional (TD) method of assessment to the current method, which uses artificial PP preparations.

METHODS

The present study was a cross-sectional survey carried out in two consecutive batches of Phase 2 medical students – 94 students from the year 2011 batch and 101 students from the year 2012 batch. All students were from Universiti Kuala Lumpur Royal College of Medicine Perak, Malaysia. These students had already gone through the entire curriculum for general and systemic pathology, and were thus selected for participation in the present study. Teachings had included both the use of conventional potted specimens and microscopic slides (i.e. TD method), as well as the PP method.

Formative assessment that required students to give a spot diagnosis per station was conducted. The scores of the students were analysed according to the batches they belonged to (i.e. the 2011 and 2012 batches). Both the TD method and current PP method of OSPE assessment were conducted for the two batches of students. There were 30 stations for each of the two methods studied, and the students had to identify the pathology and answer a question within a minute for each station. Hence, the total time for assessment of each student was one hour. For the purpose of standardisation, both batches of students had the exact same variety of OSPE stations. This was however not made known to the students during the briefing prior to the assessment. One mark was awarded for each correct answer at each station.

The stations were designed to assess all topics that were taught in the pathology curriculum, such as gross specimen examination (i.e. identifying organ specimens), histopathological examination of tissue slides, cytopathological examination of tissue cells, blood cell examination, and blood films. All microscopic stations had microscopes that were already focused, so that students only needed to look at the focused field (aided by a built-in arrowhead pointer in the microscope); movement of the slides was not allowed. The focus of interest had already been taught and thoroughly explained to these students during the practical exercises in the pathology course. In the TD method of assessment, real specimen pots and tissue slides with cell smears or blood films were provided, whereas in the PP method, all preparations were high-resolution picture plates. The PP preparations selected for the PP assessment method were those that demonstrated features almost identical to that of the TD counterpart. For example, when a real potted specimen of breast cancer was displayed, almost similar gross morphological pictures representing breast cancer for the PP preparations were obtained from Google search and the recommended websites of reference universities (e.g. PathLab: http://library.med.utah.edu/WebPath). For the assessment on cytopathology, specimens used were limited to cells appropriate for the level of undergraduate medical students, such as classical Langhans-type giant cells in tuberculosis and osteoclast-like multinucleated giant cells in bone resorption. Students would have already been exposed to these features via both microscope slides and PPs during the practical sessions of the pathology course.

In the final examinations (i.e. the formal summative assessment of the students), the time allocated for each OSPE station was 5 min instead of 1 min (as in the formative assessment). This is because, in the final examinations, the students are required to answer some short questions within the 5 min. These questions require more in-depth knowledge than the questions used in the formative assessment. The total score for each station was five marks.

To analyse the students’ perception of the two methods of assessment, each student was instructed to fill in a questionnaire after the assessment. All students who participated in the present study attended an orientation session, during which the objectives of the survey and guidelines on how to fill up the questionnaire were explained. All student answers were kept confidential and all students were aware of their right to decline participation in the study.

Mean, standard deviation and median were calculated for continuous variables (i.e. the marks scored), and percentages were calculated for categorical variables (i.e. questionnaire items). Continuous variables were compared using paired t-test. A p-value < 0.05 was considered statistically significant. All data was analysed using the Statistical Package for the Social Sciences version 17.0 (SPSS Inc, Chicago, IL, USA).

RESULTS

Table I shows the score distribution of the Phase 2 medical students from the 2011 (n = 94) and 2012 (n = 101) batches for both the TD and PP assessment methods. There were statistically significant differences (p < 0.05) in the mean total marks scored by the students in both the 2011 and 2012 batches for both the TD and PP assessment methods.

Table I.

Score distribution of the Phase 2 medical students from the 2011 and 2012 batches.

Table II shows the difference in the mean scores of the Phase 2 medical students from the 2011 and 2012 batches when the TD and PP assessment methods were compared. Comparing the TD and PP assessment methods in the two batches of students, we found that while the difference in mean scores was not significant (p > 0.05) for the assessment on gross specimens, it was significant (p < 0.001) for the assessment on histopathology slides (i.e. identifying microscopic sections of tissues). In the assessment on tissue cytopathology slides, we found a significant difference in the mean scores obtained via assessment using the TD method versus that using the PP method in the 2012 batch (p < 0.001). However, this difference was not observed for the 2011 batch (p > 0.05). In the assessment on the identification of blood cells, a significant difference between the mean scores obtained via assessment using TD method and that using the PP method was observed in the 2011 batch (p < 0.001), but not in the 2012 batch (p > 0.05). When comparing between the TD and PP assessment methods, significant differences (p < 0.001) in the mean scores of both student batches were also observed in the assessment on the interpretation of blood films. Overall, the differences between the total mean scores obtained via assessment using the TD method and that using the PP method were significant (p < 0.001) in both student batches. Table III shows the percentage responses of the medical students from the 2011 and 2012 batches to the questionnaire that was administered upon completion of the OSPE.

Table II.

Difference in the mean scores of Phase 2 medical students from the 2011 and 2012 batches when the traditional (TD) assessment method was compared with the picture plate (PP) method.

Table III.

Responses to the questionnaire administered to the medical students from the 2011 and 2012 batches.

DISCUSSION

In 2006, Domizio stated that the main goal of undergraduate pathology teaching is to provide students with knowledge on the functional and structural changes that occur due to diseases, so that clinical signs and symptoms can be understood and interpreted.(4) In almost every medical school, practical exercises are conducted in pathology teaching, and the assessment of these exercises is vitally important. From the outcome of the study conducted by Rahman et al in 2007, OSPE is a better choice as an assessment method than the TD method, as the former measures a wider range of practical skills.(6) Rahman et al also concluded that it is important to consider competency-based performance discrimination and methods to improve students’ performance quality in laboratory exercises.(6)

From the results of the present study, we found that the students from the 2012 batch had a significantly higher mean score than students from the 2011 batch (Table I). This observation may be because the specimens and slides used for the 2012 batch of students were exactly the same as those used for the 2011 batch of students. Therefore, the 2012 batch of students could have already had a preformed idea of the assessment. When the TD method was compared with the PP method, no significant difference was observed in the mean scores of both batches of students in the assessment that required them to identify gross specimens. These results suggest that the ability of the students to identify gross pathological changes in organs were similar in both assessment methods.

However, comparing the TD and PP assessment methods, significant differences in mean scores were observed in both batches of students for the assessment on histopathology slides (i.e. identification of microscopic sections of tissues) and blood film interpretation (Table II). A possible reason is that PP preparations give the observer a better visual and are easier to understand than TD preparations. These results suggest that students prefer learning by pictures from online sources and textbooks rather than by microscopic observation of slides and specimens, which may be deemed as tedious and time-consuming by students.

For the assessment testing the ability to identify cytopathology slides and blood cells, a discrepancy between the two batches of students was found. Mean scores obtained via the TD assessment method were found to be significantly different from those obtained via the PP method, with respect to the identification of cytopathology slides and the identification of blood cells in the 2012 and 2011 batches, respectively. Based on these results, it is difficult to conclude whether the students’ preference is for the TD or PP assessment method. The results suggest that the students in the two batches differ in performance and preference, likely based on their abilities and knowledge on the subject. However, both batches of students differed significantly in their overall mean scores, with students obtaining a significantly higher overall mean score when assessed using the PP method, as compared to the TD method. This result nullifies our hypothesis, as we had expected students to have a preference for the TD method over artificial PP preparations.

The present study provides evidence that with the current advancements in information technology, students are more inclined toward learning and assessment via the PP method than the TD method. Pathology is a subject that bridges the gap between basic sciences and clinical medicine. As such, a proper understanding of pathological processes is not only critical and important in medical practice, but also in undergraduate medical education. We believe that a routine and balanced exposure to both PP and TD specimens in classes is necessary to equip and prepare medical students for their future role as doctors. Pathology educators should be aware of the goal of teaching pathology – to cultivate understanding of the pathologic basis of diseases through the identification of morphologic changes in disease conditions rather than the mere training of students’ memories. To the best of our knowledge, there is limited published information on how pathology can be effectively taught in undergraduate medical degree courses. Thus, the findings of this study are of great value should modification of pathology teaching to be in line with advancements in information technology be considered.

Table III reflects a preference for the PP method of assessment by the majority of the students from both batches. This observation is congruent with their performance outcome. A majority of the students indicated that they viewed the formative OSPE test as a useful learning tool and revision exercise for pre-exam preparation. The students also indicated a preference for individual competition rather than group competition. This is probably because the students were keen to know their individual performance and ability, and this may not be apparent in a group competition. Students expressed positive attitudes, and accepted the assessment as a useful method for learning. Based on a previously conducted study and the results of the present study, pathology crossword puzzle competitions(10) and formative OPSE tests are useful tools in undergraduate pathology teaching and assessment.

Prior to the present study, we assumed that students would prefer the TD assessment method to the current PP method. However, the results of the present study showed that the students performed better when examined using the current assessment method with PP preparations than when examined using the TD method with real potted specimens and slides. This may be because, compared to TD preparations, PP preparations provide the observer with better visuals, which in turn aids understanding. The findings of the present study are valuable as they identify tools and methods that can be used to improve our current way of teaching and assessing medical students; the tools and methods identified are in line with advancements in information technology and web accessibility. In the present study, the mean scores obtained by the students are in accordance with their response to the questionnaire, which analysed their perceptions and preference with regard to assessment. The students were found to prefer the PP assessment method of assessment over the TD method.