Perceptions and Attitudes of Medical Students towards Two Methods of Assessing Practical Anatomy Knowledge

Abstract

Objectives:

Traditionally, summative practical examination in anatomy takes the form of ‘spotters’ consisting of a stream of prosections, radiological images and dissections with pins indicating specific structures. Recently, we have started to administer similar examinations online using the quiz facility in Moodle™ (a free, open-source web application for producing modular internet-based courses) in addition to the traditional format. This paper reports on an investigation into students’ perceptions of each assessment environment.

Methods:

Over a 3-year period, practical assessment in anatomy was conducted either in traditional format or online via learning management software called Moodle™. All students exposed to the two examination formats at the College of Medicine & Health Sciences, Sultan Qaboos University, Oman, were divided into two categories: junior (Year 3) and senior (Year 4). An evaluation of their perception of both examination formats was conducted using a self-administered questionnaire consisting of restricted and free response items.

Results:

More than half of all students expressed a clear preference for the online environment and believed it was more exam-friendly. This preference was higher amongst senior students. Compared to females, male students preferred the online environment. Senior students were less likely to study on cadavers when the examination was conducted online. Specimen quality, ability to manage time, and seating arrangements were major advantages identified by students who preferred the online format.

Conclusion:

Computer-based practical examinations in anatomy appeared to be generally popular with our students. The students adopted a different approach to study when the exam was conducted online as compared to the traditional ‘steeplechase’ format.

Advances in Knowledge

1. This study reports on an online method of organising anatomy “spotter” examinations.

2. It provides insight into student perceptions of assessment methods in medical education.

3. Over 44% of the students surveyed preferred the online examination method.

During the last 20 to 25 years, digital technology has emerged rapidly from a novelty to being ubiquitous in medical education.1 The advances in computer technology and software available for medical education purposes have concurrently brought technological and software advances for the development and administration of computer-based examinations for medical students.2

Several advantages of computer-based examination over traditional methods have been suggested. These include an increase in the efficiency and reliability of the assessment process,3 immediate scoring and feedback for the student and the instructor,4 the opportunity for item banking and the collection of item statistics, and the ability to incorporate multimedia into examinations, as well as a wide range of new items and new testing formats.1

Concurrent with studies reporting advantages, others attempted to examine student attitudes toward computer-based testing and its impact on students' study habits. One such previous study found that paediatric residents' attitudes were strongly positive toward the process and that the majority of residents preferred computer-based test administration over written examinations.5 Later, Rattan et al.6 also reported that medical students preferred a computer test over a paper and pencil test. Miller et al.7 in a study on computer-based examination in anatomic pathology similarly reported that medical students valued such a method and supported its use.

In addition to the many practical advantages alluded to, computer-based testing could also facilitate the development of more valid assessment strategies. This is because one of the most important goals of a good examination design is to minimise influences that interfere with the unambiguous expression of an examinee’s ability. Examinations should therefore be designed so that all examinees have an opportunity to perform their best in relation to the constructs being measured. The medium through which such examination is administered (on paper or online) should not act as a hindrance to the students’ performance and not just reflect their ability to adapt to a technology or to an examination format.1

The medical curriculum at Sultan Qaboos University extends over seven academic years. The first phase lasts for four years following which the degree of B.Sc. Health Sciences is awarded whilst the remaining three years comprise a clinical programme leading to the MD degree. An academic year consists of two semesters, each of 15 weeks duration.

Anatomy teaching is provided mainly during the B.Sc. Health Sciences programme, first as an introductory course and later integrated with other disciplines in organ-system courses.8 The teaching approach involves didactic tutor-led sessions, student-centered tutorials, based on patient scenarios that illustrate relevant anatomy, and practical classes in the dissection room. The anatomy practical uses prosected cadavers, plastinated organs, plastic models and radiological images. Although students learn on cadavers, these are mainly tutor-led demonstrations in small groups on already dissected specimens. For cultural reasons, there is no established body donation programme in the country; therefore, all cadavers have to be imported from abroad. Over the last decade it had become increasingly difficult to obtain cadavers due to restrictions placed on export by the countries of origin; this is partly because such countries do not seem to have enough cadavers to satisfy the need of their own medical schools.9

Assessment in anatomy is conducted in the form of written examinations such as multiple choice or short answer questions. In addition, students’ ability to identify anatomical structures or correlate morphology with function is traditionally assessed through practical examinations. These consist of a circular stream of prosections, radiological images and dissections with pins pointing to specific structures (popularly referred to as ‘spotters’). Specific questions are then asked about where the pin is placed so as to identify a structure, its source of blood supply or function. With the increasing difficulty in obtaining a constant supply of cadaveric material, the wear and tear of the existing ones became a real problem to the extent that the normal anatomy has become increasingly unrecognisable. In such a situation, innovation is needed both in teaching and assessment so as to use the available cadaveric material optimally. This led to the idea of developing an online summative assessment system using digitised images of the same material used in teaching. Our goal has been to create digital resources for conducting examinations similar to the traditional anatomy practical examination (spotter).

Computer-based assessment is not just an alternative method of delivering examinations, it represents an important qualitative shift from traditional methods such as paper based tests.10 Consequently, when examinations migrate from paper-and-pencil format to computer delivery, a wide range of issues could arise such as test fairness, students’ preparation habits and their perception of the examination format. In this study, we investigated our students’ perceptions of both online and traditional methods of practical assessment in anatomy. In particular, we were interested in investigating whether students adopt similar or different learning and test-taking strategies depending on the examination method.

Methods

Ethical approval for this project was obtained from the Research and Ethics Committee of the College of Medicine & Health Sciences at Sultan Qaboos University. Data collection was carried out during the six fall and spring semesters from 2007 to 2009 (inclusive) when organ-system courses were delivered. Teaching of anatomy in a 7-year medical curriculum is delivered over three academic years (years 2 to 4) within integrated organ-system courses. Practical examinations are administered at the end of each course. Typically, the number of questions ranges from 40–50 items for each organ-system course.

The practical examination in the online environment utilised digitised ‘assessment objects’ from materials used for teaching. These included dissected cadavers, surface anatomy and radiological images and microscopic slides [Figure 1]. In addition, short video clips of 30 seconds duration demonstrating muscle action were also used. These ‘objects’ were optimised for online viewing and then uploaded onto the learning management software, Moodle™ (a free, open-source web application for producing modular internet-based courses).11 This software has been used for online learning experiences in a variety of courses for the health professions.12–16 Using the quiz tool in the software, a bank of questions was created and linked to the assessment objects. Access to the software and the questions is through the password-protected secure University server.

Figure 1:

Two examples of online spotter questions.

With regards to the examination in the traditional format, the actual specimens were laid out in a circular stream in the dissecting room. Lettered tags were then attached with pins to specific spots on the specimens and questions were asked regarding those tagged spots. The number of questions was similar to that of the online environment. The questions were reviewed by a team of two faculty members to ensure accuracy and concordance with course objectives before the start of the examination. The perception of medical students towards anatomy practical examination was studied among those in years 3 and 4 (comprising about 120 students in each cohort) for three consecutive years, and anatomy practical examinations in organ-system courses were conducted both in traditional and online formats for all students. Students are informed by the course director about the format of practical examination at the beginning of each course. Our policy specifies that only course directors familiar with Moodle are permitted to administer online examinations. Others not yet trained, or not favourably inclined towards virtual learning philosophy, administer the practical examinations in the usual traditional format within the dissecting room. Immediately after each examination, and before results were announced, a paper-based or electronic student survey was carried out requesting voluntary feedback on their experience. Students had the right to decline taking part in the survey and were informed that the data would be published. The responses were categorised into those from senior students (year 4) and junior students (year 3). A chi-square test for independence between gender and seniority was computed using GraphPad Prism, version 5.00 for Windows (GraphPad Software, San Diego, California, USA, www.graphpad.com). A P value of 0.05 or lower was considered significant.

Results

A total of 515 completed responses were returned (71.5 % response rate). This comprised 165 (32%) senior and 350 (68%) junior students. There was an equal gender distribution amongst both junior and senior respondents (male: female = 1: 1.01). The actual distribution of students that preferred one of the examination formats, or had no preference, is presented in Table 1a. Overall, 229 (44.5%) of all respondents preferred the online environment whilst 190 (36.9%) preferred the traditional format and 96 (18.6%) had no particular preference [Table 1a]. A significant proportion (82%) of students who preferred the traditional format were junior students (P <0.001). Similarly, there was a slight but significant gender difference with females and junior students expressing more preference for traditional examination format (P <0.05) [Table 1b]. Interestingly, amongst those respondents expressing no preference, 91% were junior students with no significant gender difference [Table 1a].

Table 1a:

Distribution of students that preferred either examination format or had no preference

	Gender
Seniority	Online examination		Traditional format		No Preference		Totals
	Male	Female	Male	Female	Male	Female
Junior	60	45	70	87	40	48	350
Senior	70	54	8	25	2	6	165
Total	130	99	78	112	42	54	515

Table 1b:

Chi-Square statistics on preference of examination format by medical students

	Gender	Seniority
Chi-Square	5.9	245.2
Degree of freedom (df)	1	1
Significance	<0.05	<0.001

Amongst all students who responded, the major reasons given for prefering the online examination format (n = 229) included better specimen quality (151), ability to manage the total time for the examination rather than being forced to spend an equal time on each question (160), and not having to keep moving throughout the examination, but remaining seated in one place (123) [Figure 2a]. On the other hand, slow Internet connectivity and fear of computer software/hardware failure were some of the major concerns expressed by students who preferred the traditional format [Figure 2b].

Figure 2a:

Distribution of the most important reasons for student preference of online examination format.

Figure 2b:

Distribution of the most important reasons for student preference of traditional examination format.

When asked about the influence of the type of examination format (online or traditional) on their mode of preparation and study habits, 129 (78%) of senior students indicated being significantly influenced by the examination format while, only 27 (8%) of junior students agreed that the examination format had influenced their study habits and mode of preparation [Figure 3]. In addition, 119 (72%) of senior students and 116 (33%) of junior students said that they did not study on cadaveric specimens when the examination was conducted in the online format [Figure 4].

Figure 3:

Distribution of student responses and their mode of preparation and study habits when asked if the examination format (online or traditional steeplechase) significantly influenced the way they prepare for the examination.

Figure 4:

Distribution of student responses when asked about the likelihood of them studying on cadaveric specimen when the examination format is conducted online.

Discussion

The computer-based examination setting has been in use for over four decades.17 Such a setting is attractive because it makes automated marking possible, and the students receive instant feedback. Recently, apart from formative assessment, computer-based examinations are increasingly being used for end point, summative examinations (such as the US medical licensing examination). Its acceptability as a means of assessment in high stakes examinations is now well established. Several surveys have shown that computer-based tests were more popular than traditional tests. The preference for online assessment found in our results is similar to that reported earlier in the literature regarding computer-based tests.5,7 Based on the results [Table 1a and Figure 2a], the main reasons for this preference were higher quality specimens and easier time management whereby students could scan through the questions at their own pace. This is in contrast to the traditional format where they are restricted to the same length of time for each question regardless of difficulty.

Being seated for online format examinations as opposed to moving about constantly during the examination could potentially be beneficial in reducing the level of anxiety. This could be an added explanation for student preference of the online format.

The impact of gender on computer related attitudes, abilities, and usage has been actively documented.19,20 Many findings in the literature indicated that males had more positive attitudes, higher ability, and used computers more.21 Other studies, however, suggest that the differences between males and females may be on the decline, although male dominance is still prevalent with respect to attitude, ability, and use of computers.19,20 In this study, we found a clear overall gender difference in the preference for examination format with female students more likely to prefer the traditional than the online format. Amongst a single cohort of students senior or junior, however, the proportion of males and females who preferred the online format was similar. This suggests that the gender ‘digital divide’ within the same group of students in our setting is not as wide as has been reported elsewhere.22

When asked about the influence of the examination format on their study habits and preparation, the majority of students indicated that they usually study as normal before an examination. However, a great majority of senior students indicated that their study habits and preparation for the practical examination later became influenced by the format of examination. We observed a growing tendency amongst the majority of students to prepare for examination using image atlases rather than dissected specimens. This is not surprising considering the fact that assessment is well known to be a major determinant of how students learn.23 A likely explanation for this occurrence could be due to the difference in specimen quality in both examination formats. Because of the extreme wear and tear on some of our specimens, the traditional format has been associated with examination difficulty since exact identification in some situations became problematic. The senior students became aware of this reality and thereby modified their preparation according to the examination formats. This was in sharp contrast with junior students, possibly due to their lack of both experience and exposure to the two examination formats.

The results in this study have a number of implications for anatomy teaching in settings similar to ours. First, medical students are in general favourably disposed to and comfortable with the use of computer-based technology for the assessment of learning. This attitude should help to reassure faculty members intending to include such tools in their teaching. Another equally important implication is to what extent should one rely on anatomy learning on cadavers if the assessment of such learning will be conducted differently? In a situation where the physical condition of cadaveric specimens is not appropriate for learning, or the specimens are in short supply, should the use of technology be given more emphasis for teaching? A more crucial implication of this study is related to the quality of students certified through computer-based assessment; will they be any better or worse than students who are assessed in the traditional format? Are both formats equally valid? Clearly, further research is indicated to address some of the questions raised above.

Conclusion

In conclusion, computer-based practical examinations in anatomy appear to be generally popular with medical students in our setting. However, we found a clear split in this popularity, with students in senior classes more favourably disposed to computer-based practical examination. The majority of students adopted a different approach to study when informed that the practical examination would be conducted online as compared to the traditional format. We suggest further investigation to determine whether there is a significant difference in student performance between the two examination formats.