A POINT CONTINGENCY FOR HOMEWORK SUBMISSION IN THE GRADUATE SCHOOL CLASSROOM

Abstract

We explored the effects of points versus no points on the submission of homework assignments and quiz performance in a graduate-level course. Students were more likely to submit homework assignments during points weeks, but quiz scores were relatively unaffected.

Classroom learning opportunities that arrange active engagement in course material and frequent feedback have been shown to enhance students' performance (e.g., Benjamin, 1991; Michael, 1991). Homework assignments are one example of such an opportunity. Recent research on the advantages of homework assignments has raised questions about their practical benefits (see Trautwein & Koller, 2003), largely due to procedural limitations associated with large-group experimental designs. An innovative study by Ryan and Hemmes (2005) was among the first procedurally rigorous studies to evaluate the effects of homework on students' performance in a college course. The authors evaluated the effects of a points versus no points contingency on the submission of homework assignments and quiz performance in an undergraduate psychology course and found that the mean percentage of homework assignments submitted and mean quiz scores were substantially higher in the points condition. The purpose of the present study was to replicate and extend the study reported by Ryan and Hemmes to the graduate school classroom.

METHOD

Participants, Settings, and Materials

Thirty-one graduate students (26 women and four men), pursuing master of science degrees in the Behavior Analysis and Therapy Department at Southern Illinois University, provided informed consent and initially participated in the study. All of the students were enrolled in a required course entitled “Behavioral Assessment and Observation Methods.” The course met for approximately 3 hr once per week for 15 weeks. The course focused on behavioral definitions, observational recording techniques, and issues of validity and reliability of measurement. The first author of this study served as the course instructor, and the second and third authors served as teaching assistants.

We used two textbooks and a number of journal articles. Homework assignments and quizzes were based on the content of weekly assigned readings and were written by the authors. Homework assignments were each worth 10 points and included 5 to 10 questions that were either short-answer or essay questions. Each quiz was worth 15 points and included 4 to 10 questions that were either short-answer or essay questions. Points were assigned for each quiz and homework assignment question based on the complexity of the question. Points for homework questions ranged from 1 to 2 points apiece, and points for quiz questions ranged from 1 to 6 points apiece. Each weekly quiz covered the same material as the previous week's study questions, but the format, wording, and potentially the focus of quiz questions differed from that of homework assignments.

Design

We evaluated the effects of points versus no points on homework completion and quiz performance using two alternating treatments designs in effect simultaneously. At the beginning of the semester, we randomly assigned students to one of two groups (the maroon or silver group), each of which had its own order of alternating points and no-points conditions. Sixteen students initially were in the maroon group, and 15 students were in the silver group. One student in the maroon group dropped out of the course after the 4th week of class. A given condition was in effect for 1 week of the 15-week semester, with the first condition being presented during the 3rd week. The order of points and no-points conditions was counterbalanced separately for both the maroon and silver groups, with the stipulations that each condition be in effect a total of six times throughout the semester, and that no condition be in effect for more than 3 consecutive weeks.

Procedure

A list of students in the maroon and silver groups was shown on the course's Blackboard site at the beginning of the semester, along with the schedule of points and no points conditions for each group for the entire semester. During the first class session, the instructor told the students that the purpose of homework assignments was to help them prepare for the weekly quizzes, but that their completion was optional. The instructor also told the students that they would receive written feedback and a numerical score for all homework assignments submitted, even though the numerical score would not count toward their grade in the course during no-points weeks. Written feedback on homework assignments included both corrective and confirming comments. Points assigned for homework during points weeks allowed students to “buy out” of a portion of a cumulative final exam scheduled for the end of the semester. Students submitted homework assignments electronically on the Blackboard site, and the instructor graded the assignments and returned them to students in class 1 week after their submission. Students completed quizzes during each class session beginning the 3rd week of the semester. Instructors graded quizzes using a standardized answer key developed by the authors and returned the quizzes in class the following week. Graded quizzes included both confirming and corrective comments along with a numerical score.

Scoring and Interrater Agreement

The authors and two independent graders scored 41% of all homework assignments and 34% of all quizzes. The graders graded independently using the same standardized answer keys that the primary graders used. Subtracting the higher score from the lower score to obtain a percentage was used to calculate the level of interrater agreement. Mean interrater agreement was 98% for quizzes (range, 90% to 100%) and 96% for homework assignments (range, 89% to 98%).

Student Satisfaction Questionnaire

The instructor administered a brief anonymous questionnaire on the last day of class. The questionnaire asked students to indicate the extent of their agreement with the following questions: (a) To what extent were the study questions helpful for you in preparing for the quizzes? (b) To what extent did you complete study questions during no-points weeks? (c) To what extent would you prefer a course with weekly assignments relative to a course without such assignments? (d) To what extent would you prefer a course with weekly quizzes relative to less frequent examinations? The rating scale was as follows: 1  =  not at all, 2  =  to a low degree, 3  =  to some degree, 4  =  to a fairly high degree, 5  =  to a high degree, and 6  =  to a very high degree.

RESULTS AND DISCUSSION

Figure 1 shows the mean percentage of assignments submitted during the points and no points conditions for the maroon and silver groups. The maroon group submitted a mean of 92% (range, 87% to 100%) and 21% (range, 13% to 28%) of homework assignments during weeks in which points were and were not assigned, respectively. The silver group submitted a mean of 98% (range, 93% to 100%) and 30% (range, 20% to 60%) of assignments during weeks in which points were and were not assigned, respectively. Students submitted 60% of assignments during the first no-points week, but the percentage of assignments submitted decreased to approximately 20% to 30% in all subsequent weeks. A correlated groups t test applied to data for the maroon and silver groups combined showed that the percentage of homework assignments submitted was significantly higher in the points (M  =  96%; SD  =  3.61) than the no-points (M  =  26%, SD  =  11.81) condition, t(11)  =  22.52, p  =  .000. Effect size was calculated using the formula described by Huck (2000, p. 207). Effect size for homework completion was .97, which can be considered a large effect.

Figure 1.

Percentage of assignments submitted per week during points and no-points conditions for the maroon (top) and silver (bottom) groups.

Figure 2 shows the mean percentage of quiz grades during the points and no-points conditions for the maroon and silver groups. The maroon group scored a mean of 88% (range, 85% to 91%) correct during weeks in which points for homework assignments were assigned and a mean of 90% (range, 80% to 99%) correct during weeks in which no points were assigned. The silver group scored a mean of 91% correct on quizzes during weeks in which points for homework were assigned (range, 86% to 99%) and a mean of 91% correct during weeks in which no points were assigned (range, 90% to 93%).

Figure 2.

Mean quiz scores per week during points and no-points conditions for the maroon (top) and silver (bottom) groups.

Students responded with a mean score of 5.65 and median score of 6 when asked to rate the extent to which the study questions were helpful in preparing for weekly quizzes. They responded with a mean score of 5 and a median score of 6 when asked to rate the extent to which they completed study questions during no-points weeks. Students preferred weekly assignments relative to no weekly assignments, responding with a mean score of 4.57 and a median of 5, and preferred weekly quizzes to less frequent examinations, responding with a mean of 4.96 and a median of 6.

These results replicate those of Ryan and Hemmes (2005), who found that students were more likely to submit homework assignments when points were available than when they were not. Any other potential benefits of homework completion were not sufficient to maintain their completion (or at least their submission) in the absence of the point contingency. The effects of the independent variable on mean quiz performance did not correspond to those reported by Ryan and Hemmes. Although self-report measures may have questionable validity, the majority of students reported that not only were homework assignments helpful in preparing them for the weekly quizzes but that they completed them during no-points weeks. Thus, it is likely that the students completed the homework assignments during no-points weeks, but with no contingency in place for their submission, the students did not expend the effort to prepare their assignments in a format suitable for submission. Guring (2003) reported that undergraduate students rated homework questions as among the most helpful study tools (see also Saville & Zinn, 2009). Unlike the undergraduate students in Ryan and Hemmes, the graduate student participants in this study had lengthy histories during which certain study skills had been selected and refined.