Retrieval practice in the form of online homework improved information retention more when spaced 5 days rather than 1 day after class in two physiology courses

Abstract

Studies have shown that practicing temporally spaced retrieval of previously learned information via formal assessments increases student retention of the information. Our objective was to determine the impact of online homework administered as a first retrieval practice 1 or 5 days after introduction of physiology topics on long-term information retention. Students in two undergraduate courses, Anatomy and Physiology (ASCI 240) and Animal Physiological Systems (ASCI 340), were presented with information on a specific physiological system during each weekly laboratory and then completed an online homework assignment either 1 or 5 days later. Information retention was assessed via an in-class quiz the following week and by a comprehensive final exam at semester’s end (4–13 wk later). Performance on homework assignments was generally similar between groups for both courses. Information retention at 1 wk did not differ due to timing of homework in either course. In both courses, however, students who received homework 5 days after class performed better on final exam questions relevant to that week’s topic compared with their day 1 counterparts. These findings indicate that the longer period between introducing physiology information in class and assigning the first retrieval practice was more beneficial to long-term information retention than the shorter period, despite seemingly equivalent benefits in the shorter term. Since information is typically forgotten over time, we speculate that the longer interval necessitates greater retrieval effort in much the same way as built-in desirable difficulties, thus allowing for stronger conceptual connections and deeper comprehension.

INTRODUCTION

Beginning in the 1990s, there has been a push from professional organizations such as the National Science Foundation and National Academies of Science to improve undergraduate education in science, technology, engineering, and mathematics (STEM) (23). The goal of this initiative has been to improve student learning, assessment, and knowledge retention by developing more effective methods for teaching STEM subjects (11). These newer teaching strategies seek to turn classrooms into student-centered, peer-driven environments rather than the traditional lecture environments most commonly used (12, 23). One strategy that has been explored is the use of assessments as retrieval practices (3). Traditional assessments, such as homework, quizzes, and exams, are widely used to evaluate how much new knowledge students have gained, but recent studies demonstrate their value as tools to increase learning and information retention, as demonstrated by performance on subsequent summative assessments (3, 25, 26, 32, 33). The benefit of repetition on information retention has also been well documented in academia (21, 22) and is a common study tactic used by students (19). The concept of engaging students in repetitive retrieval practice was introduced a number of years ago (18) but has not been widely adopted by educators at the college level. The use of assessments as a form of retrieval practice requires students to retrieve and apply previously learned information in context more than other forms of practice, such as rereading textbooks passages or rewriting notes. The increased benefit of this method is known as the testing effect and has been shown to produce more durable learning (3, 5, 25, 26, 31). The magnitude to which the testing effect benefits learning is influenced by the amount of effort required to successfully complete the assessment (8, 9), as more challenging assessments produce greater knowledge retention. We postulate that greater intervals of time between the initial learning period and first retrieval assessment will necessitate more effort to successfully complete it, thus mimicking the beneficial effects of desirable difficulties.

Despite the demonstrated benefit of spaced assessments on learning (10, 27), there remains a gap in knowledge regarding the impact of interval length between initial introduction of physiology information and first assessment on learning, particularly in college students. Studies investigating different testing/studying sequences in college students learning medical biology (9, 10, 25, 26) found that testing is more effective than studying, and that repetitive testing spread over a longer period of time is more effective than when it is massed into a small window (10). Our present study sought to build on these findings by investigating how timing of the first retrieval practice influences its impact on information retention. Homework is a common assessment tool in college science courses, and thus our objective was to determine whether assigning homework after a relatively short 1-day interval or longer 5-day interval from class was more beneficial to retention of physiology information in college undergraduates.

METHODS

Informed Consent of Participants and Approved Animal Use

All procedures for data collection in this study were approved by the Human Subjects Institutional Review Board at the University of Nebraska-Lincoln (UNL). In addition, laboratory activities utilizing live animals were approved by the UNL Institutional Animal Care and Use Committee, which is accredited by the Association for Assessment and Accreditation of Laboratory Animal Care International. Before the study, students were informed of their option to participate in this study by an independent party without instructors or teaching assistants present. Students were not informed of the specific objectives of the study, and all students were required to complete all assignments and assessments as part of the requirements for the course. Only data from consenting individuals were included in analyses. Students who agreed to participate received a 1.5% bonus on their overall course grade as compensation for their participation. The same bonus was offered to nonparticipating students for an equal alternative effort.

Course Structure and Student Demographics

Sixty-three students who successfully completed the required core course, Anatomy and Physiology of Domestic Animals (ASCI 240) and thirteen students who successfully completed the elective course, Animal Physiological Systems (ASCI 340), were assessed in this study. Total course enrollments were 72 and 14, respectively. Both courses were offered as part of the curriculum of the Department of Animal Science at UNL. Enrollment in ASCI 240 consisted primarily of sophomore (52%) and junior (28%) undergraduate students pursuing Bachelor of Science degrees in Animal Science (57%), Fisheries and Wildlife (22%), Veterinary Science (9%), or other biological fields. The class consisted of 30% men and 70% women and was designed for students with career interests in animal industries. Enrollment in ASCI 340 consisted primarily of juniors (83%) pursuing a Bachelor of Science degree in Animal Science (92%) or Veterinary Science (8%). The class consisted of 15% men and 85% women, 70% of whom had explicitly stated goals of continuing their education in veterinary or graduate school after completing their undergraduate degree.

Both ASCI 240 and ASCI 340 were 16-wk courses consisting of three 1-h lecture periods and one 2-h laboratory period each week. Both courses were designed to cover the functions of all major systems of the mammalian body, as well as the anatomical features that comprise each. Topics assessed for this study in ASCI 240 were as follows: cells and tissues, the muscular system, the digestive system, and the immune system (Fig. 1). These four topic areas plus the reproductive system were assessed in ASCI 340. Weekly laboratory sessions were organized by physiological system. Each laboratory period began with a 10- to 15-min mini-lecture by the laboratory coordinator or teaching assistant that consisted of an overview of the main concepts for the week’s topic. The remainder of the laboratory period was devoted to the guided activities described below.

Fig. 1.

Weekly laboratory topics for ASCI 240 (A) and ASCI 340 (B) relative to the final laboratory exam at semester’s end.

Retrieval Practices and Assessments

In-class worksheets.

Guided activities performed by the students in laboratory were designed to teach the practical applications for each topic of interest. Each week, students completed an in-class worksheet during laboratory using information from the mini-lecture and the guided activity. Each of these 2- to 5-page worksheets consisted of questions designed to highlight key concepts and help students perform the guided activities explained in the Guided Laboratory Activities section below. Question formats included short answer, true/false, fill in the blank, and matching, as well as diagrams and drawings to be completed. Students worked in pairs or small groups and were encouraged to utilize their notes, textbook, and the internet to complete their worksheets. Additionally, students were encouraged to ask questions of the laboratory coordinator and teaching assistants. Worksheets were collected at the end of each laboratory, graded, and returned to the students the following week.

Online homework assignments.

In both courses, students were assigned online homework each week via the Blackboard Learn portal. Using a list randomizer, students were assigned to one of two equally sized groups. The first group received their online homework assignment 1 day after laboratory, and the second group received the same homework assignment 5 days after laboratory (Fig. 2). Each homework assignment consisted of an open-note 10-question assessment that focused on the key concepts of the week’s topic. Questions were formatted as multiple choice, short answer, fill in the blank, and true/false. Students had a 36-h window in which to complete the homework assignment from the time they were notified via e-mail of its availability. Each question required completion before the next question was accessible. After each answer was submitted, students received pre-prepared feedback that elaborated on the correct answer and explained why common incorrect answers were wrong. Groups were re-randomized each week.

Fig. 2.

Weekly schedule for laboratories, homework assignments, and quizzes. Students received identical online homework assignments either 1 or 5 days after class and then completed a short quiz in class the following week.

In-class quizzes.

Each week, students were given a 10-min quiz at the beginning of laboratory to assess their retention of the previous week’s information (i.e., 7 days after initial introduction of the information). Each in-class quiz consisted of 10 short-answer questions that were conceptually similar to the questions presented on the respective in-class worksheet and homework assignment. Quizzes were closed-note and individually completed.

Comprehensive final exam.

In both courses, a cumulative laboratory practical exam was administered in the final week of the semester. Students were given 2 h to individually complete a closed-note written exam that required them to apply the information learned throughout the semester in a practical way. Questions were formatted as fill in the blank, multiple choice, and short answer. Each weekly topic evaluated in the present study was represented on the final exam by a minimum of three questions worth a total of 10 total points, and percentage of total points earned for questions pertaining to each topic was recorded. Questions that related to topics not evaluated as part of this study made up 30 and 23% of the final laboratory exams for ASCI 240 and 340, respectively.

All laboratory mini-lectures, guided activities, worksheets, homework assignments, quizzes, and final laboratory exams were developed by the laboratory coordinator and were approved by the instructor of the course before the start of the semester. Assignments were graded automatically when possible. Questions (i.e., short answer) that could not be graded automatically were hand-graded by the same individual (one of two teaching assistants not involved in the study) for all students to minimize grading bias. Students were informed by the syllabus of their right to appeal any grade to the laboratory coordinator and instructor.

Guided Laboratory Activities

The cells and tissues laboratories focused on basic cellular processes, such as gene transcription/translation and the cell cycle, using videos and interactive computer programs. Students identified optimal concentrations of various components involved in protein synthesis, identified major mammalian cell types, and used microscopes to visualize the tissues that comprise the major systems discussed throughout the semester. The muscular system laboratories characterized the different muscle types, explored the major muscle groups via fetal pig dissection, and identified names and locations of the major muscles of the mammalian body. ASCI 340 students also learned about myogenesis by performing stem cell isolations from rat muscle and tracking their growth and differentiation in culture. The digestive system laboratories described digestive processes and demonstrated the differences between nonruminant and ruminant systems via dissections and displays. For ASCI 340, this laboratory also included the urinary system, kidney function, and kidney dissection. The immune system laboratories explored various human and animal pathologies associated with the physiological systems covered throughout the semester, provided information about the major cellular and noncellular components of the immune system, and overviewed common clinical symptoms related to immune responses. ASCI 340 students also learned about clinical hematology diagnoses. The reproductive system laboratory (ASCI 340 only) introduced the anatomical structures of male and female reproductive tracts, as well as hormonal regulation of folliculogenesis, spermatogenesis, and pregnancy.

Statistical Analysis

Data for students in ASCI 240 and ASCI 340 were analyzed separately. Student performance on each worksheet, homework, quiz, and final exam section was determined as the percentage of total points that were earned. Performance scores were analyzed using the MIXED procedure of SAS (SAS Institute, Cary, NC) with cumulative grade point average (GPA) at the end of the junior year (ASCI 240, 3.06 ± 0.06; ASCI 340, 3.34 ± 0.15) as a covariate and weekly topic as a repeated measure. Effect sizes were estimated via partial η² from individual ANOVAs. Performance scores were correlated (Pearson) with GPA via the CORR procedure of SAS. Students who did not successfully complete the course, who optioned the course as pass/fail, or who did not consent to participation were not included in this study. Two additional ASCI 240 students were dropped after receiving permission to take the final laboratory exam at a later date, leaving 63 and 13 experimental units for ASCI 240 and 340, respectively. All data are presented as mean percentages ± SE.

RESULTS

Performance scores on in-class worksheets did not differ between groups or among topics in either course. In ASCI 240, scores for homework assignments (Fig. 3) did not differ between day 1 and day 5 groups (87.4 ± 1.3% and 86.5 ± 1.4%, respectively) and no interactions were observed, but there were effects due to weekly topic [F(3,161) = 6.39, P < 0.001,${\text{η}}_p^2$ = 0.06] and GPA [F(1,62) = 51.27, P < 0.001, ${\text{η}}_p^2$ = 0.2]. Specifically, homework scores were lower (P ≤ 0.05) for the immune system laboratory in week 10 than for the other laboratories, and GPA was positively correlated with score (r = 0.42; P < 0.001). In ASCI 340, scores for homework assignments did not differ between groups or among weekly topics, and no interactions were observed. Scores for weekly in-class quizzes (Fig. 4) did not differ between groups in ASCI 240 (81.6 ± 1.6 and 82.4 ± 2.1%, respectively) or ASCI 340 (77.7 ± 3.1 and 83.3 ± 4.2%, respectively) or among weekly topics, and no interactions were observed.

Fig. 3.

Homework scores for college undergraduates completing topic-relevant online homework assignments 1 or 5 days after class. Students were enrolled in one of two undergraduate physiology courses, ASCI 240 (A) or ASCI 340 (B). NS, nonsignificant.

Fig. 4.

Weekly quiz scores for college undergraduates completing topic-relevant online homework assignments 1 or 5 days after class. Students were enrolled in one of two undergraduate physiology courses, ASCI 240 (A) or ASCI 340 (B).

Scores on the final laboratory exam questions (Fig. 5) evaluated in ASCI 240 were lower [F(1,62) = 4.77, P = 0.03, ${\text{η}}_p^2$ = 0.06] for students in the day 1 group compared with those in the day 5 group (75.8 ± 1.4 and 83.5 ± 1.6%, respectively), and there were effects due to weekly topic [F(3,182) = 2.53, P = 0.05, ${\text{η}}_p^2$ = 0.02] and GPA [F(1,62) = 28.17, P < 0.001, ${\text{η}}_p^2$ = 0.21]. Specifically, scores were lowest (P ≤ 0.05) for the digestive system covered in week 11 and greatest (P ≤ 0.05) for the muscular system covered in week 5. Scores were also positively correlated with GPA (r = 0.44; P < 0.001). Likewise, scores on the final laboratory exam questions evaluated in ASCI 340 were lower [F(1,12) = 6.93, P = 0.02, ${\text{η}}_p^2$ = 0.08] for students in the day 1 group compared with those in the day 5 group (75.4 ± 2.9 and 84.6 ± 1.9%, respectively), and there were effects due to weekly topic [F(4,43) = 21.26, P < 0.001, ${\text{η}}_p^2$ = 0.38] and GPA [F(1,12) = 13.53, P = 0.004, ${\text{η}}_p^2$ = 0.23]. Specifically, scores were lowest (P ≤ 0.05) for the reproductive system covered in week 10 and cells and tissues covered in week 2 and were greatest (P ≤ 0.05) for the digestive system covered in week 9. Scores were also positively correlated with GPA (r = 0.47; P < 0.001).

Fig. 5.

Summative assessment scores (final exam questions) for college undergraduates completing topic-relevant online homework assignments 1 or 5 days after class. Students were enrolled in one of two undergraduate physiology courses, ASCI 240 (A) or ASCI 340 (B).

DISCUSSION

The use of assessments for retrieval practice has been shown to increase the retention of physiology and biomedical information learned by students in science courses (9, 10, 25, 26). In the present study, we build on these previous findings by testing whether a larger gap between the introduction of new physiology information and the first assessment is more beneficial to sustained retention of the information. Indeed, we found that first assessments in the form of mandatory online homework assigned 5 days after class improved final summative assessment scores at semester’s end compared with the same homework assigned 1 day after class. We postulate that the increased passage of time makes successful completion of the first assessment more difficult, and that the greater challenge ultimately benefits knowledge retention in much the same way as desirable difficulties (8). Since information is typically forgotten over time in the absence of retrieval practice (29, 36), it is reasonable to assume that the information learned in class was less easily recalled by students assigned homework after 5 days. Conversely, students who were asked to complete the homework assignment after only 24 h could more easily reiterate the information. The longer interval would, therefore, necessitate a greater retrieval effort to complete the assignment successfully (i.e., earn a comparable score), and more challenging retrievals have been shown to enhance the testing effect (2, 6, 35), which recruits more neural pathways and improves long-term knowledge retention (37).

It is worth noting that the longer interval in this study produced benefits that were consistent with desirable difficulties in a different context than previous studies. Specifically, our study reflected a relatively long learning period, and the first retrieval practice for both experimental groups occurred well beyond the initial learning session. Additionally, students were not restricted in the methods used to complete these assessments so as to be consistent with typical homework. Nevertheless, the online homework assignments appeared to accomplish the main objectives of effective retrieval practice, including knowledge application, feedback, and a certain degree of effort (30). Of course, we cannot ignore the fact that a longer interval provides more opportunity for independent study, and repetitive study habits such as re-reading textbooks or re-writing notes have been shown to benefit information retention (5, 33). However, the similar homework and quiz scores between the experimental groups indicate that this was unlikely. Although it is impossible to be certain that the benefits of the longer interval were solely due to improved testing effects, the similarities between our results and other studies examining the impact of desirable difficulties in retrieval practices are difficult to ignore.

Interestingly, the benefit of the longer interval between class and first retrieval practice was not apparent in the shorter term, as scores on quizzes taken 7 days after the initial introduction of the information did not differ between experimental groups. This observation was unexpected but not unprecedented, as other studies have observed differing short-term and long-term effects on retention, albeit under different time frames, retrieval practice designs, and subject matter (9, 20, 36). Nevertheless, the benefits of the testing effect have also been documented at time frames of <24 h (18, 29), and thus it is unclear why quiz scores at 1 wk were not different in the present study. One possible explanation is our choice of question formatting, which included multiple-choice, fill-in-the-blank, and short-answer questions, as their convenience for graders makes them popular for homework assignments in large-enrollment courses. It is possible that intrinsic context clues associated with these styles, especially the first two, could have allowed questions to be completed with less internal recall of information, which is known to be less beneficial to learning (28). However, previous studies have shown that properly written multiple-choice questions are as effective at stimulating retrieval as short-answer or essay questions (4, 17, 34), which, combined with the improvement observed at the semester’s end, would make this explanation unlikely. It is equally unlikely that overly difficult material was a contributing factor. Although success rate is a key factor in the effectiveness of retrieval practices (30, 34), homework and quiz scores did not indicate undue difficulty of material or question composition. Rather, we postulate that perhaps the expanded time frame of the learning period in the present study made the quiz at 1 wk a similar relative milestone to the earlier summative assessments in previous studies (9, 20, 36).

Lastly, we show that the longer interval before first retrieval practice benefits student populations with differing experience levels and academic expectations, as estimated effect sizes indicated similar impacts in the upper-level (ASCI 340) and core (ASCI 240) physiology courses. These results were somewhat surprising, as other learning improvement strategies, such as collaborative group testing (13), student-written exam questions (16), and active-learning modules (15), yield greater benefits for low- or average-performing students compared with higher performing students. Similarly, self-regulated learning techniques are profoundly more troublesome for lower performing students (1). Our ASCI 340 student population had higher average 3-yr GPAs and overall course grades compared with ASCI 240, which would indicate better average performance. Moreover, student disinterest is a major source for poor performance in college science courses (7, 14), and the smaller class size and large proportion of students with ASCI majors and physiology-relevant career goals in ASCI 340 almost certainly increased their interest and engagement compared with ASCI 240, yet students from both courses benefited equally. These unexpected similarities may be indicative of proportionality in the relative difficulty of the two courses, as the greater difficulty of the material in ASCI 340 was likely proportional to the greater acuity of the students. Thus the increase in retrieval effort required to complete the homework assignment at day 5 compared with day 1 was likely similar between the two courses, despite the other differences.

To conclude, our findings in this study show that delaying homework assignments by a period of several days after introducing new physiology information in class may optimize its benefit on long-term knowledge retention, even if the greater benefit is not obvious at 1 wk. The greater impact of homework assigned 5 days after class rather than 1 day on summative assessment scores at semester’s end was consistent with greater desirable difficulties, presumably created by the longer period of time in which to lose information. Moreover, the longer interval was effective in student populations with wide-ranging academic performance and expectations. These results show that simply delaying typical homework assignments in undergraduate science courses can capture the advantage of greater retrieval effort and enhance learning. Although more research is needed to fully understand the importance of retrieval practice timing during these longer learning periods, it should be noted that this technique could be implemented with considerable ease for both instructors and students, which would lead to greater adoption and a more profound impact on learning.

GRANTS

This project is based on research that was partially supported by the National Institute of General Medical Sciences Grant 1P20GM104320 (J. Zempleni, Director), the Nebraska Agricultural Experiment Station with funding from the Hatch Act (NEB-26-224), and Hatch Multistate Research capacity funding program (NEB-26-226, NEB-26-225) through the USDA National Institute of Food and Agriculture.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the authors.

AUTHOR CONTRIBUTIONS

C.N.C. and D.T.Y. conceived and designed research; C.N.C. and D.T.Y. performed experiments; C.N.C. and D.T.Y. analyzed data; C.N.C. and D.T.Y. interpreted results of experiments; C.N.C. and D.T.Y. prepared figures; C.N.C. drafted manuscript; C.N.C. and D.T.Y. edited and revised manuscript; C.N.C. and D.T.Y. approved final version of manuscript.