The use of anonymous pop-quizzes (APQs) as a tool to reinforce learning

INTRODUCTION

This paper reports on the use of anonymous pop quizzes (APQs) during stand-alone and multi-session library instruction sessions. APQs were used at the end of a class as a tool to test how well students have understood the material presented to them, to reinforce retention of important points, to increase student engagement through an interactive task, and to assist the instructor in planning and preparing future classes.

Pop quizzes are frequently used in the classroom. They are popular with students as they often allow for bonus points [1, 2] and popular with teachers as they encourage student attendance and pre-class preparation [3]. Padilla-Walker has shown that students who performed well on daily extra-credit quizzes did better on final exams [2], while Nevid found that “mastery” quizzes—pop-quizzes given at the beginning and end of a class lecture to indicate key lecture concepts—led to a marked improvement by undergraduate psychology students in terms of retention of key concepts [4]. The research on announced versus unannounced quizzes is contradictory: Some studies indicate students performed better on unannounced quizzes (i.e., pop quizzes) as they encourage attendance in class and the review of materials prior to coming to class [5, 6], while other work found announced quizzes led to improved exam performance and final grades for undergraduate medical students [7].

The few examples of anonymous quizzes in the literature (called ConcepTests or interactive anonymous quizzes) were used in the late-1990s in physics and chemistry classes [8–10]. However, these examples—which tended to be highly structured, focus on a single question, and involve a great deal of in-class discussion amongst students—are quite different in format from the APQs as used in this study at the University of Saskatchewan.

While the literature is scarce on the use of APQs in instruction, overall, the use of pop quizzes in the classroom setting has been shown to identify problem areas that need additional reinforcement [6], encourage class attendance and pre-class preparation [11], and allow students to preview sample test questions [11, 12]. It was with these potential benefits in mind that the use of APQs was implemented in library instruction in a health sciences library.

SETTING

The University of Saskatchewan is a publicly funded research institution located in Saskatoon, Canada. In 2011/12, 16,578 undergraduate students and 2,890 graduate students were enrolled in more than 17 different colleges including arts, law, business, medicine, agriculture, and engineering, among others. This work was carried out at the Health Sciences Library, which has 4 librarians acting as liaisons to 391 academic faculty and 3,222 students in the Colleges of Medicine, Nursing, Dentistry, Pharmacy and Nutrition, and Kinesiology and the Schools of Public Health and Physical Therapy. During the study period, APQs were used in a variety of library instruction sessions, including stand-alone single sessions for nursing and public health and for multi-sessions built into the curriculum for dentistry.

METHODS

The idea of APQs originally came about during lesson planning for the library component of the first-year dentistry class, “Application of Dental Research to Clinical Decision Making,” where students are introduced to dental research and, by the end of the course, are expected to be able to access, understand, and evaluate the dental scientific literature. The library gives three consecutive classes as part of the course during which basic library skills are taught and built upon, and a test is given during the last class. The mark received on this test is built into the final grade of the dentistry course. Owing to a sabbatical in the 2010/11 academic year, responsibility for this class fell to a different librarian than the one who had taught it in previous years. While keeping the basic structure of the library section intact, it was decided that students (and the instructor) might benefit from a form of pop quiz that would aid students in preparing for the types of questions they might face on the final test.

The free program SurveyMonkey <www.surveymonkey.com>, was used to create the APQs. The link to the APQ was given on the final PowerPoint slide presented to classes 1 and 2. Students did not include their names or student numbers when filling out the quiz in order to keep things entirely anonymous. All questions were mandatory and multiple choice. Questions had either a single right answer or multiple correct answers, and students were expected to select all that applied. The quizzes were short (6–7 questions), took approximately 5 minutes to complete, and focused specifically on content covered in the class. Sample questions included:

Doing the keyword search “dental floss” will find articles where:

a) The words dental AND floss appear somewhere

b) The words dental OR floss appear somewhere

c) The words dental floss appear side-by-side

Point of care tools (like DynaMed) provide the following:

a) Summarized evidence on a topic

b) Patient handouts

c) Extensive references

d) Treatment options

e) a, b, and d

f) All of the above

Medical Subject Headings (MeSH) are a “controlled vocabulary” that lets you use one term to: [select all that apply]

a) Capture variations in spelling

b) Capture variations in language (e.g., French, Arabic)

c) Capture variations in terminology

After class, the instructor compared the number of completed quizzes with the number of students in the class to ensure all students had completed the quiz. These two numbers matched in all classes, but if there had been discrepancies, the instructor would have more strongly encouraged participation and more actively circulated around the room while students were supposed to be completing the quiz. The student answers were then analyzed and compared to the correct answers to identify areas where the students had clearly understood the material and areas in which learning might need to be reinforced. For example, student responses indicated more time should be spent describing MeSH, as many students did not answer these questions correctly (Figure 1, online only).

After analyzing the students' results, it was easy to determine which areas needed more attention in class. At the start of classes 2 and 3, the results of the quizzes from classes 1 and 2 (respectively) were gone over in class. Each question was addressed and discussed with the students to further reinforce learning and to strengthen areas where students had performed poorly.

RESULTS

During the final dentistry class, the formal course test was administered with students scoring an average mark of 79%. Unfortunately, it was not possible to determine if the APQs had an impact on students' test grades as the test was different in 2010/11 from previous years and was marked by a different librarian. Determining this relationship is, however, a definite possibility for future classes. However, informal feedback in class from the students was encouraging. During the overview of the previous weeks' quizzes, students discussed animatedly who had gotten questions right or wrong and feedback was forthcoming when students felt another answer was correct. This relaxed interaction with the students helped create a rapport between the instructor and students and resulted in an extremely positive and rewarding teaching experience. In addition, formal written class evaluations from several students mentioned their enjoyment of the quizzes and said that they found them helpful for their learning, and a thank you email from the professor of the course noted that students had commented very favorably on the library classes and teaching.

After the success of the dentistry class, the author used APQs in a number of other classes in nursing and public health that were stand-alone (i.e., single sessions with a group of student who would not be seen again). In these cases, the same format was followed: quizzes were constructed using SurveyMonkey, a link to the quiz was given near the end of class, and multiple choice questions focused on content covered in the class. While these stand-alone sessions did not offer the instructor the opportunity to review correct versus incorrect answers with students in subsequent classes, they still offered a chance to reinforce class learning and helped identify which areas of the lesson might need more attention in future sessions. For example, if numerous students struggled to understand the difference between “Explode” and “Focus” (an advanced search feature in many health sciences databases), it was clear that the future sessions should spend more time exploring that area. This information would be of particular value in library classes in which the same content is taught multiple times to different sets of students. Results from the quizzes of the first few classes could inform content covered in later classes, so that areas where students are weakest are given more attention.

DISCUSSION AND CONCLUSION

While more research with quantitative evidence is needed before the true impact of APQs can be determined, preliminary work supports the idea that tests taken immediately after the material has been delivered is a more effective way of learning than isolated studying, repeated studying, or concept mapping [13]. APQs are a particularly helpful tool in library instruction sessions, which are typically structured very differently from traditional university courses, as they are often stand-alone, may offer minimal chance to engage with the students, and are not graded. While this report focused on two uses of APQs in the library classroom, there are numerous other ways that they could be used in an instruction session. Examples include using quizzes with the same questions at the beginning and end of a class to assess learning over the course of a session or short quizzes that students take in the middle of a class so that the instructor has time to go over the results in class. Future directions for research could more closely examine the different ways that APQs can be used in library instruction, while also determining the actual impact of APQs on student learning and retention through the gathering of quantitative data. Ultimately, the use of anonymous pop-quizzes in library instruction sessions provides an opportunity to engage students with an interactive component; test students' learning without accompanying test “anxiety”; serve as an effective posttest if following a bridge, objective, pretest, participatory learning, posttest, summary (BOPPPS) or similar model; and allow the instructor to identity teaching areas of strength and weakness for future sessions.