Impact of Spacing of Practice on Learning Brand Name and Generic Drugs

Abstract

Objective. To test the impact of schedules of retrieval practice on learning brand and generic name drug information in a self-paced course.

Methods. Students completed weekly quizzes on brand and generic name conversions for 100 commonly prescribed drugs. Each student completed part of the drug list on a schedule of equal, expanding, or contracting spacing, one practice (massed) or study only in a partial block design.

Results. On measures of long-term retention, the contracting spacing schedule led to superior retention (67%) compared to the massed practice (50%) and study-only condition (46%); contracting practice also was significantly higher than expanding practice (58%,) or equal practice (59%). Overall performance decreased by almost 50% (final exam 95%, long-term retention 55%) over a 6-week period.

Conclusion. A contracting spacing schedule was the most effective schedule of practice, and all spacing schedules were superior to massed practice or study-only conditions.

INTRODUCTION

Within a course, students may study material with the only opportunity to test themselves occurring on the examination. Alternatively, lower stakes quizzes provide greater learning opportunities before these higher stakes examinations. Despite the evidence that practice is an important part of learning, lessons or courses may not optimally use practice testing or spacing of practice.

Students must learn factual, foundational information so they can apply it in a clinical context, an example of which is brand and generic drug names. Because learning facts is important for higher order learning and critical thinking, there is continual research focused on methods to improve the long-term retention of material. One area of research is spaced practice.

Based on laboratory research, individuals who space their practice over time have greater long-term retention than those practicing all at once (ie, massed practice).^1-5 Spacing can be accomplished in various ways. The expanding schedule involves shorter initial intervals followed by longer intervals. The contracting practice schedule reverses this pattern by having longer initial intervals followed by shorter intervals. Finally, the equal spacing schedule uses uniform intervals. To date, there is no consensus regarding which schedule of practice is best under laboratory conditions.^6-11 In an authentic classroom setting, there is less information and agreement regarding the impact of spacing, particularly spacing schedules.^12-17 Most studies conducted in a classroom or simulated classroom focus on the spacing of practice versus massed practice, but not on schedules of spacing. Spacing of practice within the classroom has resulted in medium to large effect sizes.^12,17 The goal of this study is to investigate the impact of various spacing schedules of testing on learning brand and generic drug names in a self-paced course.

In general, students can space their study, restudy, and practice periods. This study focused specifically on test-type practice (ie, “testing effect” or retrieval practice). Retrieval practice produces greater learning and long-term retention than restudying material for an equivalent length of time.^18-20 Spaced retrieval practice works to enhance memory by balancing the success of retrieval (ie, correctly answering) and the difficulty of that retrieval. The sooner a retrieval is attempted, the greater the likelihood of a successful retrieval.¹⁸ Although an easier retrieval may lead to an increased retrieval sucess, it has not shown any benefit for long-term retention.²¹ In contrast, increasing retrieval difficulty due to longer spacing intervals leads to greater long-term retention benefits due to greater elaborative processing.^22,23 Finding the correct balance between retrieval difficulty and success rate is challenging. This study focused on successful retrieval through testing soon after study, and increasing retrieval difficulty by re-testing at longer spacing intervals.

The next question involves the type of test to administer. Multiple-choice testing offers cued retrieval and increases the likelihood of successful retrieval. This question type contrasts with open-ended questions, such as fill-in-the-blank, that require response generation and focus more on recall than recognition.^24,25 Past research have reported that more generative effects lead to a more powerful testing effect, but recent evidence suggest they can be equally efficacious.^13,15,16,26 The use of generative questions is important in this study as these questions more closely mimic the pharmacist role and require the correct spelling needed to avoid look-a-like, sound-a-like medical errors.

Both testing and spacing increase how well material is stored in memory (ie, storage strength) and how easily one can retrieve needed material (ie, retrieval strength). Researchers have found that longer retention intervals tend to favor spacing more than short retention intervals.^27-29 In addition, some research suggests that the optimal spacing schedule depends on the desired retention interval.¹⁰ For this study, both short-term retention (ie, final examination) and long retention intervals (ie, retest 6 weeks after the final examination) were examined.

This study investigated the impact of spacing of practice in a self-paced course introducing medical terminology and Top 200 medications. This course was chosen as it emphasized the importance of memorizing factual, foundational information that are core competencies for the PharmD degree.³⁰ This course is fitting for self-paced learning as it requires independent study and memorization.³¹ This study will also attempt to answer the question -- can instructors better design learning activities and courses through better understanding of spacing intervals.

METHODS

Study participants were159 students enrolled in the PharmD program at a large, public university in the southeastern United States. Approximately 70% of the student population is female, with an average age of 22. Entering grade point average is 3.5 (out of 4.0), standard admission test scores (ie, the PCAT) averaged 86%, and approximately 81% of students have a prior degree. Medical terminology is a two-course series identical in format and credit hours (0.5 hours). The focus of this study is the first course in the series, Medical Terminology 1, which occurs in the fall semester of the first year. The course format was self-paced, focusing on medical terminology and introductory aspects of frequently prescribed medications (ie, Top 200 medications). For this latter material, students were asked to learn brand and generic name, class, and general indications. This study focused on the brand/generic name information.

Students were assigned 100 drugs for the semester, separated into 10 lists based roughly on systems corresponding to the concurrent pathophysiology course; the remaining 100 drugs were covered in the second semester course. At the beginning of the semester, students were given a matched list of brand and generic drug names that they would be responsible for memorizing, with some generics having more than one brand name. Students were instructed to study two drugs lists (20 drugs) for each weekly quiz and were required to look up class and indication. Weekly quizzes opened each Friday morning and were due Sunday night. Students had 15 minutes and one attempt to complete the quiz on the learning management system (Sakai, Apereo Foundation, NJ) at their own convenience (location and time). Quizzes were fill-in-the-blank type, and required correct spelling for full credit. After 5 weeks, all 100 drugs were initially studied and quizzed once (except the study-only condition). For the following 5 weeks, students completed weekly quizzes on unannounced drug lists according to the spacing conditions in Figure 1. The final examination occurred during finals week (week 12 of the semester).

Figure 1.

Spacing schedule. Study only had zero practice quizzes. Massed had a quiz immediately after study. Expanding had a second quiz closer to the initial quiz than the final exam. Contracting had the second quiz closer to the final exam than the initial quiz. Equal had a second quiz equidistant from the initial quiz and the final.

The study examined five conditions with two drug lists randomly assigned to each condition (Figure 1). The “Study Only” condition did not assess students on the drug list during the weekly quiz and represented a classroom situation where students study material but never practice except on the examination. The “Massed” practice condition assessed students on the drug list immediately after study but not during any future quizzes. In the “Equal” practice condition, students were quizzed on the drug list immediately after study and quizzed equidistant between the original study time and the final examination. In the “Expanding” practice condition, students were quizzed on the drug list immediately after study and quizzed again at a period closer to the original study period compared to the final examination. The “Contracting” practice condition quizzed students on the drug list immediately after study and again closer to the final examination than the original study period. Students were balanced across the conditions to minimize list-dependent effects. For example, for List 1 half of the class was contracting, while the other half was in the expanding condition. In this design, students were aware of content within the first quiz, but not for future quizzes so they can focus on retrieval effects versus re-study effects.

The primary outcome was performance on the final examination, which included 50 brand/generic questions, sampling one list from each condition. The designsed two examination formats assessing all 100 drugs. Half the students received Lists 1, 3, 5, 7, 9 and the other half received Lists 2, 4, 6, 8, 10. The final was similar to the quizzes (fill-in-the-blank, with correct spelling required for full credit) completed in class during finals week. The secondary outcome was performance on a measure of long-term retention. This assessment was the same as the final examination and was completed 6 weeks later during the first day of class of the sequential course in the spring semester. Both examinations were completed in ExamSoft (ExamSoft Worldwide, Inc., Dallas, TX).

This study used a within-subjects design, and a repeated measures ANOVA with a Sidak post hoc to assess differences between conditions with significance set at p<.05. Cohen’s d effect size with 95% confidence interval was constructed between each condition and the study-only condition. This study was deemed exempt by the university’s Institutional Review Board.

RESULTS

Table 1 contains descriptive statistics for the quizzes, final examination, and long-term retention assessments.

Table 1.

Summary of Assessments^a

When compared to the study-only condition, effect sizes for the three spacing conditions (contracting, equal, expanding) were moderate to large, and the 95% confidence interval did not cross zero for all three spacing conditions (Table 1). The contracting condition was statistically higher than all other conditions (Table 2). There was no difference between the equal and expanding schedules, but both conditions were significantly larger than the massed and study-only conditions.

Table 2.

Post-hoc Results From Comparison of Conditions on Measures of Long-term Retention^a

Conditional analyses were conducted to explore how performance on the quizzes affected the final and long-term assessments. Conditional analysis regarding overall performance was examined using a two-way ANOVA for condition (contract, equal, expand) and retrieval success (0,1,2). Table 3 shows the proportion of correct responses on the final exam and long-term retention as a function of the initial learning condition and retrieval success on the initial quizzes (successful on one or more vs. unsuccessful on all quizzes). There was a main effect for success (p<.001) but not for condition (p=.15) or the interaction term (p=.97). In most cases, two successful attempts led to higher performance on long-term retention measures for all conditions but in no case did only one successful retrieval impact performance on long-term retention measures.

Table 3.

Summary of Final Test Performance and Long-term Retention Performance Conditionalized by Initial Retrieval Success on Quizzes^a

Analysis of student cues with brand versus generic names showed that students performed equally well regardless of receiving the generic or brand name cue on the first quiz (90% vs 91%, p=.46), second quiz (87% vs 88%, p=.44), final examination (96% vs 96%, p=.77) or long-term retention assessment (62% vs 60%, p=.51).

The final analysis considered whether the loss of performance was sustained during the following semester. The weekly quizzes for the spring semester included the drugs from the prior, fall semester. Performance was high with Mean (SD) 90 (40) and a range=82-94% throughout the semester.

DISCUSSION

This study examined the potential advantage of spacing quizzes (practice) to improve long-term retention in a classroom setting. All spacing schedules proved superior to massed and study-only conditions and demonstrates the benefit of multiple retrieval attempts compared to re-study. Incorporating quizzes within self-paced courses should be intentional by the course designer as students may not self-test on their own as a study strategy.^31-33

The contracting practice schedule leads to superior long-term retention of brand and generic name information and this finding contrasts with older studies showing expanding practice superior for long-term retention.^6,34 However, clarity in this area is difficult as some recent research suggest equal spacing is best or that the best spacing schedule depends on the desired retention interval.^10,11 As an example, Kupper-Tetzel found the contracting schedule superior for medium length intervals, but expanding or equal spacing was better for longer length intervals.¹⁰ To confirm the results of the Kupper study, the current study would need even longer retention intervals, which are challenging given reinforcement of the content from other courses within the curriculum and experiential practice opportunities which may bias results.

The impact of spacing schedules is a function of retrieval success versus retrieval difficulty. For testing to be impactful, the first retrieval attempt must require effort which may require that studied content to be cleared from primary memory.^11,35 Early research demonstrated that delaying that initial recall helps latter recall, supporting the notion of a desirable difficulty in retrieval.^36,37 Within this study, it is unclear if that first retrieval attempt required sufficient effort as it was administered online and could have been preceded immediately by study. The second quiz (content unknown to the student) may have provoked more effort. As such, the impact on retention and learning was considered with respect to these initial quizzes. Within this study, increasing the initial space (from quiz 1 to quiz 2) increased the retrieval difficulty and resulted in the best long-term performance. These results are in agreement with studies demonstrating that greater retrieval difficulty leads to improved memory traces due to more significant elaborative processing.²² This study demonstrates the benefit of multiple successful retrieval attempts that increase performance on tests of long-term retention. The most important takeaway may be that students who retrieved information successfully on both quizzes performed the best on measures of long-term retention. This suggests that a single quiz and an examination may be insufficient to cause long-term retention benefits and instructors need to continually test students on prior material within a course and throughout a curriculum. This is consistent with other research showing that more successful retrievals yield stronger learning. ³⁸

The strength of this study was the use of a within-subject design, and the minimization of list dependent effects by using an incomplete block design. Within this study, list-dependent effects were minimized by dividing the class into two halves with students receiving lists in different spacing conditions. Full block design was not an option due to practical considerations. Thus, some list-dependent effects may still occur. In addition, each condition consisted of two lists, but the absolute spacing within the lists was different (Figure 1). This combination adds more variability, strengthening the results as significant effects were demonstrated with highly variable data. Participants were assessed on 50 of the 100 drugs for practical considerations of time during the examination. A potential limitation involves the testing of participants on one of the two lists versus combining lists (ie, 50% from List 1, 50% from List 2) since not all students retrieved all lists on the dependent variables.

As with any classroom research, it is a challenge to remove outside influences. This study was conducted during the fall semester when a minimal number of courses could hypothetically reinforce the content. Within this study, there were no differences at the final examination between conditions, most likely from acute study before that assessment. This acute, prior re-study could increase retrieval strength but may not impact storage strength as seen by information loss over time.

This study demonstrated the importance of practice after study, the importance of many retrieval practices spaced over time, that successful retrieval plays a significant role in the testing effect, and that content is rapidly forgotten after study stops. Immediate retrieval of information was helpful for learning but not as useful as multiple attempts over time, reinforcing the importance of retrieval after the initial learning phase. It is common for intervention studies to use acute measures of performance as the primary outcome, but time is needed to observe the effect of an educational intervention. This type of assessment may represent retrieval strength (how accessible the information may be) but not reflect learning or storage strength (how well it is stored in memory) whereas an assessment after a longer period with little reinforcement indicates storage strength.

CONCLUSION

While many studies have examined the spacing effect in controlled environments, few have considered the impact in a live classroom. This study showed that the contracting spacing schedule was superior to alternative schedules at improving long-term retention using a 6-week retention interval.

Appendix 1. Top 200 Drug List