Abstract
Aim
To evaluate daily-written 10-question quizzes in a medical anatomy course as a way to integrate assessment into the course and to evaluate their effect on the course success.
Methods
Students answering correctly 8/10 or more questions were awarded 0.5 points per quiz. There were 34 quizzes with a maximum point score 17. Measurable outcomes of academic progress in anatomy course (pass rates on 4 examination terms, total pass rate, and average marks) were calculated, and 2007/08 academic year was compared with the previous academic year in which daily written quizzes were not a part of the course. The relationship between cumulative points on daily quizzes and 3 components of the final examination (written, practical, and oral) for 2007/08 academic year was assessed by non-parametric correlation testing.
Results
Individual scores on quizzes ranged from 1.5 to 13.5 points. There was a positive correlation between scores on quizzes and grades on 3 components of the final examination: written (Spearman ρ = 0.784, P < 0.001, n = 79), practical (Spearman ρ = 0.342, P < 0.002, n = 79), and oral (Spearman ρ = 0.683, P < 0.001, n = 79) part. Compared with students in the previous academic year, students attending the course with daily quizzes significantly improved their academic achievement, expressed as the pass rate at the first examination term (39% vs 62%, respectively, χ2 test, P = 0.006) and the average course grade (2.71 ± 1.08 vs 3.38 ± 1.26, respectively; t test, P < 0.001).
Conclusion
Despite their frequency and possible associated stress, daily quizzes were associated with better academic success in the anatomy course.
Anatomy is one of the most important preclinical subjects; not only because of its shear volume and complexity, requiring the adoption of skills and attitudes, but also because it is the first truly medical course for university “novices,” who are still in the process of transition from secondary school pupils to university students. Furthermore, in the traditional medical curriculum, with separate preclinical and clinical subjects, anatomy is perceived as an introduction to “real” medicine, playing an important role in the process of professional socialization (1). Thus, anatomy teachers are not only teaching anatomy but are also university educators with distinct scholarly roles. One of these roles is encouraging students to take a widely cited “deep approach” to studying anatomy (2-4), leading to a more complete understanding of the subject matter. This should also include the capacity of independent, intrinsically motivated study. We believe that these, very much desired, educational aims can be reached only if the following 3 critical components of a good quality curriculum are optimally harmonized: recommended literature, teaching methods/learning activities, and assessment. It is argued that curriculum design, with a true integration of various forms of assessment into learning activities (5,6), should produce a measurable positive impact on students’ knowledge and understanding of anatomy (3). Being aware of the importance of correct assessment procedures, we introduced a significant change in our continuous assessment. There was a long tradition of oral forms of assessment at each practical session in our curriculum, which appeared to be both time consuming and lacked objectivity. Models of active-learning pedagogy include frequent assessments, structured as either daily quizzes or brief exams that cover small units of instruction combined with in-class discussion and/or group problem-solving activities monitored and assessed by the tutor (7). Introducing daily written mini examinations (quizzes) in 2007/08 academic year had the following aims: 1) full integration of assessment into the course, 2) provision of students with continuous feedback about their progress, 3) identification of underperforming students, and 4) rewarding hard-working students. As a positive side-effect of our frequent testing, we also hoped to change the negative perception of the examination process. We wanted our students to accept testing as a normal and integral part of everyday teaching/learning process. We were also aware of the possible negative side-effect that our quizzes might have: the ongoing conflict between “getting a good grade” and “really mastering the material,” with the tendency of assessment-domination in the academic efforts of our students. At the end of the first academic year in which daily quizzes were integrated into the anatomy course, we investigated the academic success of students and compared it with the previous academic year, in which daily quizzes were not offered.
Materials and methods
Course design
At the University of Split School of Medicine, gross anatomy is taught as a 3-month course in the first year of the medical curriculum. The course is divided into 34 units delivered using different didactic approaches (lectures, seminars, and laboratory classes including prosections and dissections).
Students’ knowledge assessments are performed throughout the entire length of the course in the form of written, practical, and oral examinations. At the end of the course, students take the final examination consisting of 3 consecutive components: written, practical, and oral. If the first attempt to pass the final exam is unsuccessful, students are given 3 more chances to sit the exam during the academic year. The written examination can be passed during the course in the form of 2 interim examinations or at the end of the course as a single written test for those candidates who did not pass one or both interim exams. The first interim examination (A1) includes general and radiological anatomy, osteology, syndesmology, and basic morphology of the central nervous system, while the second interim exam (A2) includes topographic anatomy of the head and neck, upper and lower extremities, thorax, abdomen, back, and pelvis and perineum. Satisfactory result on the written part of the exam in a multiple choice questions format is followed by the practical examination, where students identify labeled structures on cadavers and models, and an oral examination consisting of a set of 7 questions covering all topics in our anatomy curriculum.
During the 2007/08 academic year, a new reward system was introduced into the gross anatomy assessment in the form of 10-question daily quizzes after each seminar in addition to the brief oral and practical exam within every laboratory class. Daily assessments were administered under test-like conditions and graded by instructors. The quiz was followed by an extensive self, peer, and instructor feedback the next day. The formats of quizzes included open-ended/uncued questions, two types of extended matching questions, and true/false questions (Table 1). Individual quizzes contained a single type of questions.
Table 1.
Samples of questions formats in daily quizzes
| Type of question | Example |
|---|---|
| True-false | If you think the statement is true circle T, and if you think the statement is false circle F |
| 1. Scapula has two faces, acromion, and three angles. T F | |
| 2. Articulatio humeri is articulatio elipsoidea by mechanics. T F | |
| 3. Clavicula is directly connected to humerus. T F | |
| Uncued | Complete the statement: |
| 1. Structure that lies upon sulcus inter-tubercularis is _________. | |
| 2. Distal insertion of m. deltoideus is __________. | |
| 3. The highest point of spina scapulae is called __________. | |
| Extended matching I | Each lettered item from A to C can be used once, more than once, or not at all: A – humerus, B – scapula, C – articulatio acromioclavicularis |
| 1. Fossa supraspinata A B C | |
| 2. Margo lateralis A B C | |
| 3. Lig. coracoclaviculare A B C | |
| Extended matching II | Each lettered item from A to C can be used once, more than once or not at all: A – spina scapulae, B – extremitas acromialis, C – facies posterior |
| 1. It is a part of humerus A B C | |
| 2. Divides facies posterior into two unequal parts A B C | |
| 3. It is a part of scapula A B C |
Quiz scoring
Students answering correctly 8 or more out of 10 questions received 0.5 points for the quiz. The maximum score at the end of the 3-month course was 17 points. These “bonus” points were added to the score of the written part of the final examination.
All 3 parts of the final examination were graded separately using the following grades: 2 (sufficient), 3 (good), 4 (very good), and 5 (excellent).
The relationship between daily quizzes and all 3 parts of the final examination was assessed by Spearman correlation test.
We also compared academic achievement in the 2007/08 anatomy course with that in the previous academic year, when the course did not include daily quizzes.
Statistical analysis
The scores on individual quizzes were presented as median with range or interquartile range because the distribution of data was not normal. The association between the score on daily quizzes and 3 components of the final exam was tested using Spearman ρ correlation test. The percentage of students passing the anatomy course in the 2 academic years was compared using χ2 test and the average course grades by t test for unpaired samples. Kruskal-Wallis test with post-hoc Dunn test for multiple comparisons was used to compare quiz scores at various examination terms. All analyses were performed using InStat3 software (GraphPad, La Jolla, CA, USA).
Results
Median values of scores on each of 34 quizzes ranged between 2/10 (for quiz No. 15) and 9/10 (for quizzes no 4, 14, 21, 23, 30, and 33) (Figure 1, Table 2). The number of students awarded 0.5 points in the quiz ranged from 1/77 for quiz No. 15 to 68/79 for quiz no 14 (Table 2). There was a very high correlation between the number of students awarded bonus 0.5 points per particular quiz and the median value of quiz scores (Spearman ρ = 0.96, P < 0.001).
Figure 1.
Average scores (out of maximum 10 questions) of individual quizzes (median and interquartile range).
Table 2.
Students attendance and performance on quizzes
| Quiz number | Median score (range) on daily quizzes | Number of students rewarded with bonus 0.5 mark* |
|---|---|---|
| 1 | 6 (2-8) | 17 |
| 2 | 8 (3-10) | 54 |
| 3 | 7 (3-10) | 34 |
| 4 | 9 (6-10) | 61 |
| 5 | 7 (2-9) | 32 |
| 6 | 8 (5-10) | 54 |
| 7 | 8 (5-10) | 46 |
| 8 | 6 (2-10) | 11 |
| 9 | 7 (0-10) | 37 |
| 10 | 6 (1-9) | 15 |
| 11 | 4 (1-8) | 4 |
| 12 | 6 (1-10) | 24 |
| 13 | 5 (1-10) | 23 |
| 14 | 9 (5-10) | 68 |
| 15 | 2 (0-8) | 1 |
| 16 | 2.5 (0-8) | 3 |
| 17 | 7 (4-10) | 35 |
| 18 | 7 (4-9) | 25 |
| 19 | 7 (3-10) | 23 |
| 20 | 7 (3-10) | 39 |
| 21 | 9 (0-10) | 61 |
| 22 | 7 (3-10) | 39 |
| 23 | 9 (5-10) | 59 |
| 24 | 6 (2-8) | 9 |
| 25 | 8 (3-10) | 54 |
| 26 | 4 (0-9) | 14 |
| 27 | 8 (4-10) | 53 |
| 28 | 5.5 (0-10) | 21 |
| 29 | 8 (4-10) | 53 |
| 30 | 9 (6-10) | 65 |
| 31 | 8 (6-10) | 51 |
| 32 | 2 (2-10) | 37 |
| 33 | 9 (0-10) | 55 |
| 34 | 6 (1-10) | 16 |
*The number of students sitting the quizzes ranged from 72 to 79 out of total 79 students in the 2007/2008 cohort.
Out of a possible 17 points that could be achieved on all 34 quizzes, students achieved from 1.5 to 13.5 points. Although quiz points were added to the score of the final written exam, only in the case of a single student did the addition of points push the score over the pass/fail threshold. For other students, it increased the final grade on the test.
Scores on daily quizzes were positively associated with the grades students got on the written (Spearman ρ = 0.784, P < 0.001, n = 79), practical (Spearman ρ = 0.342, P < 0.002, n = 79), and oral (Spearman ρ = 0.683, P < 0.001, n = 79) part of the final examination.
Students who passed the final exam in the first examination term had a median of 8.5 points, which was 4 points higher than had those who did not pass the exam (Table 3). Students who did not pass the final exam during the 2007/2008 academic year had a median score on daily quizzes of 4.5 points. The quiz scores of the students who passed the anatomy exam in the first exam term were significantly higher than quiz scores of students passing in other exam terms, except the third term (Table 3). No significant difference was found between the quiz scores achieved in other terms, due to small sample size in the third and fourth exam term.
Table 3.
Quizzes’ median scores (out of maximum 10 points) for students who passed the final course exam at available examination terms in 2007/2008 academic year
| Exam term | No. students passing exam | Median score (range) on daily quizzes |
|---|---|---|
| First | 49 | 8.5 (5.5-13.5)*† |
| Second | 12 | 6.2 (5.0-11.0) |
| Third | 4 | 4.5 (2.5-6.5) |
| Fourth | 4 | 6.2 (3.5-7.5) |
| Course failed | 10 | 4.5 (1.5-6.0) |
*P = 0.006 vs second and third exam terms; Kruskal-Wallis test and post-hoc Dunn test for multiple comparisons.
†P < 0.001 vs “course failed,” Kruskal-Wallis test and post-hoc Dunn test for multiple comparisons.
The overall academic achievement in the anatomy course, assessed as the number of students passing the final examination and as their average course grade, was compared between 2006/2007 academic year, which did not include daily quizzes in the anatomy course, and 2007/2008 academic year when the course included the quizzes.. Students in 2007/2008 academic year had significantly greater course success than students in 2006/2007 academic year (Table 4).
Table 4.
Academic achievement in passing the anatomy course in 2006/2007 and 2007/2008 academic year
| Academic year | No. students attending course | No. (%) students passing exam |
Average course grade (mean ± standard deviation) | |
|---|---|---|---|---|
| first exam term | total | |||
| 2006/2007 | 80 | 31 (39) | 67 (84) | 2.71 ± 1.08 |
| 2007/2008 | 79 | 49 (62)* | 70 (89)* | 3.38 ± 1.26† |
*P = 0.006 vs previous academic year, χ2 test.
†P < 0.001, t test for unpaired samples. Passing grades at Croatian universities range from 2-sufficient to 5-excellent.
Discussion
The aim of this study was to test if and how benefits of continuous and frequent knowledge assessment during anatomy course can be objectively quantified and translated into the measurable academic progress of first year students. The scores on daily quizzes correlated well with students’ grades on all 3 components of the final examination: written, practical, and oral. The highest correlation was between students’ scores on daily quizzes and those on the written part of the final examination, despite the fact that both interim exams and the final written exam had multiple choice questions format, whereas the daily quizzes came in a variety of different formats but never in the multiple choice question format. There were 2 reasons why we used non-multiple choice type questions for the quizzes. First, it has been shown that extended matching and open-ended/uncued type questions stimulate learning approaches which encourage more effective recall of information and improve problem solving abilities, leading to long-term knowledge retention (8-11). Second, we did not want the students to regard the quizzes only as training for either interim or final written exams or to see them as a mere chance to earn reward points. However, by introducing frequent examinations, we taught students that testing is a “fact of life” in medical education (8). After initial surprise and some resistance, the majority of students realized that not only was their studying fitness improving, but also by that their attention was steered toward key-learning issues and that they were provided with ongoing feedback (8).
There was a marked decrease in students’ performance (quizzes 15 and 16) at the beginning of the second month of teaching period, which immediately followed their first interim exam. This drop was most probably related to the combination of tiredness after the first interim exam and the very demanding anatomy material in teaching units 15 and 16 (topographic anatomy of the head). However, students recovered rather quickly and got back to previously reached performance levels. We did not compare quiz scores with different quiz formats and/or difficulty of anatomy material represented in 34 teaching units. This requires precise standardization of quiz formats, as well as assessment of particular anatomy material (eg, number and difficulty of included anatomical nomenclature), and will be the focus of our future follow-up study.
The quizzes contained an important reward component because students were able to earn points which would increase their success on the final anatomy exam. It is well known that building in a prominent reward component into the assessment procedure has multiple benefits (3): encouraging consistent work, building students’ confidence, alleviating end-of-term exam stress, and reducing the time needed for exam preparation.
Although we were not able to measure the intensity and length of discussion upon the completion of a quiz, our impression was that they could be used as an indicator of students’ involvement and interest. At times, it seemed that it was almost possible to correlate the level of noise in the room with the difficulty/success of a quiz. The day after the quiz, the tutor addressed the entire class at the beginning of the laboratory session, providing correct answers and explanations, which were followed by either individual or group feedback discussions. These short sessions were a good opportunity to address “close-but-not-right” answers, as well as to obtain in-depth information on marginal students (9). Benefits and organization of a well controlled post-test feedback sessions are discussed in detail by Kitchen et al (7). Although our feedback sessions were not firmly structured, they were perceived to be very valuable not only for students but also for us. We were open to students’ opinions and criticism regarding format and level of difficulty of quizzes and changed some aspects accordingly.
We tested the impact of daily quizzes by comparing generation 2007/2008 with the previous generation of students, which did not have quizzes as a part of their assessment. The 2007/2008 generation showed better performance in all categories tested. A possible limitation of this study was a lack of information regarding the academic profile of the students involved (such as secondary school grades and faculty entrance exam scores). Although this study showed a strong association between the score on the quizzes and academic success, it is unlikely that the merit for these findings can be attributed only to the new assessment method. An ideal way to test the impact of quizzes would be offering them to a half of single generation of students and compare their exam performances, but this is not possible in the setting of the anatomy course and would be unfair to students. Despite limitations, it is hard to ignore the fact that more than 60% of the generation 2007/2008 passed the exam during the first examination term, while less than 40% of their colleagues managed to do so a year before. Also, the difference between average final marks points to the value of bonus points earned on daily quizzes.
How hard does the effort really need to be for a student? A student needs to score at least 9 points in quizzes to pass the final exam at the first attempt. That translates to achieving a score of 80% or more on at least 18 out of 35 quizzes, so more than a half of the entire content has to be mastered at a high score level. This is important information for future students because it gives them something very clear to aim for during the course.
In conclusion, the results of our study strongly support the recommendation of Fenderson et al (8) to use mini-examinations/quizzes as a tool to encourage and monitor students’ progress. To put it in pharmacological terms, quizzes produce good results even if they are “prescribed” more frequently than weekly (almost daily), the majority of students “tolerate” them rather well, “desired effects” include better academic progress and decreased exam anxiety, and they are a great “vehicle” for higher quality student-teacher communication.
Acknowledgments
The authors thank Irena Zakarija-Grković for critical reading of the manuscript, Snježana Tomić for her kind advice and helpful hints, and to Mile Zorić for his help with statistics.
I. G. is the member of the Management Board of the Croatian Medical Journal and Vice-dean of the University of Split School of Medicine, one of the owners of the journal. To ensure that any possible conflict of interest has been addressed, this article was reviewed according to best practice guidelines of international editorial organizations.
References
- 1.Smith CF, Mathias H. An investigation into medical students' approaches to anatomy learning in a systems-based prosection course. Clin Anat. 2007;20:843–8. doi: 10.1002/ca.20530. [DOI] [PubMed] [Google Scholar]
- 2.Eizenberg N. Applying student learning research to practice. In: Bowden JA, editor. Student learning: research into practice. Parkville: Centre of the Study of Higher Education; 1986. p. 21-60. [Google Scholar]
- 3.McLean M. Introducing a reward system in assessment in histology: a comment on the learning strategies it might engender. BMC Med Educ. 2001;1:7. doi: 10.1186/1472-6920-1-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Mattick K, Knight L. High-quality learning: harder to achieve than we think? Med Educ. 2007;41:638–44. doi: 10.1111/j.1365-2923.2007.02783.x. [DOI] [PubMed] [Google Scholar]
- 5.Krasne S, Wimmers PF, Relan A, Drake TA. Differential effects of two types of formative assessment in predicting performance of first-year medical students. Adv Health Sci Educ Theory Pract. 2006;11:155–71. doi: 10.1007/s10459-005-5290-9. [DOI] [PubMed] [Google Scholar]
- 6.Palmer EJ, Devitt PG. Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper. BMC Med Educ. 2007;7:49. doi: 10.1186/1472-6920-7-49. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Kitchen E, King SH, Robison DF, Sudweeks RR, Bradshaw WS, Bell JD. Rethinking exams and letter grades: how much can teachers delegate to students? CBE Life Sci Educ. 2006;5:270–80. doi: 10.1187/cbe.05-11-0123. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Fenderson BA, Fishback J, Damjanov I. Weekly mini-examinations (quizzes) based on extended-matching questions as a means for monitoring medical student performance. Croat Med J. 1996;37:283–7. [Google Scholar]
- 9.Damjanov I, Fenderson BA, Veloski JJ, Rubin E. Testing of medical students with open-ended, uncued questions. Hum Pathol. 1995;26:362–5. doi: 10.1016/0046-8177(95)90134-5. [DOI] [PubMed] [Google Scholar]
- 10.Fenderson BA, Damjanov I, Robeson MR, Veloski JJ, Rubin E. The virtues of extended matching and uncued tests as alternatives to multiple choice questions. Hum Pathol. 1997;28:526–32. doi: 10.1016/S0046-8177(97)90073-3. [DOI] [PubMed] [Google Scholar]
- 11.Lukic IK, Gluncic V, Katavic V, Petanjek Z, Jalsovec D, Marusic A. Weekly quizzes in extended-matching format as a means of monitoring students' progress in gross anatomy. Ann Anat. 2001;183:575–9. doi: 10.1016/S0940-9602(01)80073-X. [DOI] [PubMed] [Google Scholar]