ENDING ON A HIGH NOTE: Adding a Better End to Effortful Study

Abstract

Remembered utility is the retrospective evaluation about the pleasure and pain associated with a past experience. It has been shown to influence prospective choices about whether to repeat or to avoid similar situations in the future (Kahneman 2000; Kahneman, Fredrickson, Schreiber & Redelmeier, 1993). Evaluations about our hedonic past often disregard the duration of the experience and are influenced more by the peak and the final levels of discomfort (Fredrickson & Kahneman, 1993). Two experiments explored the remembered discomfort of an effortful learning experience and the influence of this evaluation on prospective study choices. The design of the studies mimicked Kahneman et al.'s, (1993) cold-pressor study, but used an exceptionally challenging learning experience in place of the painful experience of submerging one's hand in ice water. An extremely effortful study episode extended by a more moderate interval was preferred to a shorter, unextended interval, despite better test performance following the shorter interval. Future study choices reflected this preference. These findings suggest that the act of acquiring knowledge has value in the learning process.

Our hedonic experiences of pleasure and pain guide our choices. The impact of hedonic experience on the evaluation of utility was articulated first by the utilitarian philosopher Jeremy Bentham (1789) and has been broadly elaborated as experienced utility by Daniel Kahneman and collaborators (Kahneman, Wakker, & Sarin, 1997). Remembered utility—a measure of experienced utility—is the retrospective evaluation about the pleasure and pain associated with a past experience. It has been shown to influence prospective choices about whether to repeat or to avoid similar situations in the future (Kahneman 2000; Kahneman, Fredrickson, Schreiber & Redelmeier, 1993). The current study introduces learning experiences into the discussion of remembered utility by exploring the remembered discomfort of an effortful learning experience and the influence of this evaluation on prospective study choices. Kahneman and others have shown that people prefer to repeat uncomfortable physical experiences when those experiences end with a period of relative comfort. Could the same dynamic apply to the experience of learning? When the lessons that students have to to learn are necessarily challenging ones, as they can often be, how do the features of a study experience influence future learning preferences? The answers to these questions have important educational implications.

In a seminal examination of retrospective evaluations of pain, Kahneman et al., (1993) reported the counterintuitive finding that, under the right circumstances, people prefer more pain to less. The team found that when asked which episode they had preferred, people selected the painful experience of submerging their hand into 14°C ice water for 60 s, plus an additional 30 s of 15°C ice water, over a shorter trial of 60 s of 14°C ice water alone. The addition of the more moderate, though still aversive pain, led to the irrational selection of objectively more pain over less. This pattern has been shown in retrospective evaluations about a number of disagreeable experiences, including exposure to unpleasantly loud noises (Schreiber & Kahneman, 2000), and aversive film clips (Fredrickson & Kahneman, 1993), pressure from a vice (Ariely, 1998), and painful medical procedures (Redelmeier & Kahneman, 1996; Redelmeier, Katz & Kahneman, 2003). In each of these cases, the longer experience that ended with less discomfort was preferable to the shorter episode. These findings suggest that our memories of hedonic experiences are inaccurate. A rational and unbiased retrospective appraisal would consist of additive assessments of pain over time; instead, evaluations about our hedonic past often disregard the duration of the experience and are influenced more by the peak and the end levels of discomfort (Fredrickson & Kahneman, 1993).

Remembered utility is an important influence on our future choices. For example, when asked which ice water trial they would rather repeat in the future, cold pressor study participants actually chose to repeat the longer episode that ended less painfully (Kahneman et al., 1993). In an important field study of remembered discomfort, Redelmeier et al. (2003) examined the remembered utility of an unpleasant colonoscopy procedure and its effect on decisions to return for preventive screening tests in the future. For half of the patients the procedure was extended with a short, less painful interval in which the colonoscope was left in the rectum. As expected, this group of patients judged the overall experience as less unpleasant than the group for whom the procedure was actually shorter. There was an important practical consequence of the less negative appraisal. The extended group had a higher rate of return for a repeat colonoscopy some years later.

In the current study, the concepts and principles of remembered utility were applied to mental effort. The objectives of the study were twofold. The first objective was to explore whether extending an effortful study experience with a somewhat less effortful interval would be preferred to a shorter, unextended interval. The second objective was to explore how the remembered utility of prior study would impact on students' willingness to engage in effortful learning in the future. The study presented here mimicked the original Kahneman et al., (1993) cold pressor study, using an exceptionally challenging study experience in place of the painful experience of submerging one's hand in ice water. By applying an existing experimental logic to a new domain, the overarching goals of the study were to provide an important extension of the scope of the effects of remembered utility and to contribute to the understanding of the factors that guide students' choices about learning.

If effortful study experiences were evaluated in a similar way to other aversive experiences then a comparable demonstration of duration neglect should occur with study preferences and future study choices. That is, a longer study episode that ended with somewhat less challenging material should be preferred to a shorter episode, and future study decisions should also reflect this preference. This pattern would indicate that judgments and choices about learning consider the affect associated with a study experience.

However, retrospective evaluations and future choices about learning may not demonstrate the same duration neglect bias as evaluations about other aversive experiences. Instead, evaluations and choices about learning may be guided by performance on the test rather than by the acquisition phase. In the experiments presented here, participants studied both a short and an extended list. Importantly, the test following the study of each list consisted of a 20-item subset of only the most difficult items. They were never tested on the more moderate items. Because a larger number of items had been studied in the extended list, greater interference, and thus, worse recall performance was expected on the test following the extended list. One of the guiding assumptions of theories of self-regulated learning, is that people monitor their past test performance and use this information to regulate their future learning (e.g. Baker & Brown, 1984; Pressley, Borkowski & Schneider, 1987). Adults are very accurate at monitoring their past performance (Finn & Metcalfe, 2007; 2008; Gardiner & Klee, 1976; Robinson & Kulp, 1970), and they use evaluations about prior test performance to make study decisions about subsequent learning trials (Finn & Metcalfe, 2007; 2008; Gardiner, Passmore, Herriot, & Klee, 1977; Halff, 1977; King, Zechmeister & Shaughnessy, 1980; LaPorte & Voss, 1974). When regulating their study decisions, learners often attempt to maximize their performance (e.g. Ariel, Dunlosky & Bailey, 2009; Thiede & Dunlosky, 1999). Thus, if people were primarily relying on memory for test performance when making their retrospective evaluations and future study choices, the opposite pattern should emerge, namely, people should prefer the shorter list because doing so would lead to higher test performance.

Experiment 1

Method

Participants, Design and Materials

Participants were 44 undergraduates participating for course credit or cash. Participants were treated in accordance with APA ethical guidelines in all experiments. Participants were told at the start of the experiment that they would study and be tested on 3 word lists, Lists 1, 2 and 3. Unbeknownst to the participants, one of the study lists was a short list and one was an extended list. List 3 was never studied or tested. There was no indication that the trials would differ, except that the lists would be comprised of different words. The order of the extended and the short lists was counterbalanced across participants. Twenty-three participants studied and were tested on the short followed by the extended list (Short-Extended) and 21 participants studied and were tested on the extended followed by the short list (Extended-Short). Participants were told that they would be tested after each list, but only on the 20 most difficult items. This warning was included to do away with any expectation that either the less difficult items might be included on the test following the extended list, or that the extended list would provide the opportunity to remember more items relative to the shorter list.

In the short trial, participants studied 30 extremely difficult Spanish-English translations. The items came from a list of Spanish-English translations ranging from very easy to extremely difficult that had been normed in a number of other experiments. The extended trial was made up of 30 equally difficult Spanish-English translations, in addition to 15 moderately difficult words that were always the last 15 items presented. The set of extremely difficult items for each list was drawn randomly for each participant from the entire pool of 60 extremely difficult items. Study time was experimenter paced. The Spanish-English pairs were presented sequentially, for 2 seconds each. At the end of each study trial, participants were given a cued recall test, in which the Spanish word was presented and they were tasked with typing in the English translation on a random selection of 20 of the extremely difficult items that they had studied. At the end of the List 1 study-test trial, participants were told that they were going to learn List 2, and pressed a start button when they were ready. At the end of the List 2 study-test trial, participants completed a distractor task that lasted about 7 minutes. Before the expected third trial, participants were told that they needed to answer a few questions before they continued on to List 3. The questions were very similar to or were identically worded to Kahneman et al.'s (1993) “Impressions of Cold-Water Trials” questions and the same ordering was used. See Table 1 for a summary of the questions and responses. The first question was, “Suppose we paid you to come back tomorrow to complete one more study list. Would you rather study a list (of new words) that was more like List 1 or like List 2?” For each question, participants pressed a “List 1” or a “List 2” button, which referred to the first and second trial lists. The next question was, “For the third study list today, you can pick which type of list you would like to repeat. Would you rather study a list (of new words) that was more like List 1 or like List 2?”. Next, participants were told that there were a few more questions that they needed to answer before starting List 3. They were then asked, “Which list was more difficult to learn?”, Which list do you think it took longer to learn?”, “Which list caused you the greater overall discomfort?” and “Which list was tougher for you to cope with?”. These questions were originally applied to assessments about a physically aversive stimulus, and so to be sure that people were able to make this kind of hedonic judgment about study, the experimenter asked participants at the end of the study to discuss their ability to cope with the study episode. Participants had no difficulty using coping language to describe their discomfort during the study. After answering all of the questions participants were informed that there would be no third list and were debriefed.

Table 1. Measures of Remembered Utility: Proportion of Choice for Extended List.

Choices About Future Study
	Experiment 1	Experiment 2
Tomorrow's List	.73	.78
List 3?	.73	.82
List Evaluations
Less Difficult?	.70	.74
Less Long to Learn?	.70	.63 ns
Less Overall Discomfort?	.66	.67 *
Less Tough to Cope With?	.71	.78

All means were significant at p<.05 unless otherwise noted.

^*p =.08.

Results and Discussion

Test performance

A 2(list type: short versus extended) × 2(list order: short list first trial versus short list second trial) repeated measures ANOVA was conducted to assess test performance. There was no effect of list order, F<1, p >.05. Performance was equally poor when the short list was studied on the first (M=.15 SE=.02) or on the second trial (M=.14 SE=.02), F<1, p>.05. All performance measures were significantly positive from zero (ts >1, ps < .001). The list type by list order interaction was not significant, F<1, p>.05, demonstrating that trial had no effect. The analysis did show a main effect of list type, F(1, 42) = 5.54, MSE=.01, p <.05, η²_p = .12. Better performance followed study of the short list (M=.16 SE=.02) over the long (M=.12 SE=.02). If memory for list performance were the basis for future study choices, and research has shown that people can remember and do use their prior test performance to make judgments about future study (c.f. Finn & Metcalfe, 2008; Gardiner & Klee, 1976) the shorter list should have been preferred.

Measures of remembered utility

Next, measures of remembered utility were calculated. There was no effect of the order of long and short lists on any measure of remembered utility (F<1, p>.05). The main result of interest was participants' choice for future study. Chi squared tests were used to evaluate list preference. Kahneman et al., (1993) noted, if participants had been choosing to minimize their exposure to aversive study, the proportion of choices for the extended list would have been zero. As predicted, most people (73%) selected the extended list for study a day later (χ²=9.09, p<.01), as well as for their third list choice (73%, χ²=9.09, p<.01). Two participants were inconsistent in their choices for these two questions, but were included in the analysis. The majority of participants (70%) indicated that the extended list had been less difficult (χ²=7.34, p<.01), less tough to cope with (71%, χ²=7.36, p<.01), and that the longer list had caused less overall discomfort, (66%, χ²=4.46, p<.05). These results suggested that like physical pain, judgments about effortful study were more sensitive to how the episode had ended than to the duration of the episode. In fact, 70% (χ²=7.36, p<.01) of participants actually thought the shorter list had taken longer to learn.

Correlations between future study preferences and study evaluations showed that participants chose to repeat the study experience that they had found less challenging overall. The correlations were r_ϕ= .72, p<.01 between both choice for tomorrow and choice for List 3 and difficulty response, r_ϕ= .81, p<.01 between choice for tomorrow and discomfort response, and r_ϕ= .64, p<.01 (List 3 choice: r_ϕ= .74, p<.01) between choice and coping response. Correlations between future list choice and response for longer list were also significantly positive (Tomorrow: r_ϕ= .72, p<.01, List 3: r_ϕ= .72, p<.01).

Results from Experiment 1 extended Kahneman et al.'s (1993) findings. The results suggested that memory for extremely effortful learning experiences and for physically aversive experiences are subject to similar biases. Moreover, despite a potentially important performance motive favoring the short list, judgments and future study choices demonstrated that people would rather participate in extended learning that ended more pleasurably.

Experiment 2

The goal in Experiment 2 was to replicate the findings in Experiment 1 and extend them with a real-time measure of discomfort taken during study. It was possible that upon presentation of the moderately difficult items the learning episode would no longer be considered effortful, and ratings would plummet to a level of zero discomfort. Some might even find the additional items pleasurable to learn, making preference for the extended list less noteworthy. Real-time ratings were included to provide some assurance that people were indeed experiencing discomfort as they studied, and that the added items, while presumably somewhat less aversive, were still experienced negatively¹.

Method

Participants, Design and Materials

Participants were 27 undergraduates participating for course credit. The design and materials were identical to Experiment 1, except that as participants studied they made discomfort ratings every 6 seconds (after every third word), totaling 9 judgments in the short interval, and 14 judgments in the extended interval. When prompted to make a discomfort rating participants indicated their current level of discomfort on a scale that ranged from 0 (No Discomfort) to 10 (Extreme Discomfort). Participants were given 6 seconds to make the rating, after which the scale disappeared and the next item for study was presented. Participants were given a practice trial making ratings before the list presentation began.

Results and Discussion

Test performance

All performance measures were significantly positive from zero (ts>1, ps< .001). There was no effect of list order, (F<1, p>.05). As in Experiment 1, recall performance was equally poor on the first (M=.11 SE=.02) and second trials (M=.11 SE=.02). The list type by list order interaction was not significant, F(1, 25) = 2.33, MSE=.01, p>.05, η²_p = .09. Like Experiment 1, there was a main effect of list type, F(2, 25)= 8.55, MSE=.01, p<.01, η²_p = .26. Better performance followed study of the short list (M=.14, SE=.02) than the long (M=.08, SE=.02).

Judgments of Discomfort

All items were included in the analyses that follow. As can be seen from Figure 1, real-time ratings of discomfort for the extremely difficult items (ratings 1-9) were very similar (Mean of ratings 1-9, Extended: M=5.21, Short: M= 5.61). Ratings dropped as participants studied the more moderate items in the extended interval. Importantly however, ratings during the extended interval did not drop to zero (t >1, p<.001). As was intended, study of the items in the extended interval was experienced with somewhat less discomfort than study of the extremely challenging set. However, participants still found them aversive. The overall ratings were lower in the extended trial (M= 4.92, SE=.19) than in the short trial (M= 5.61, SE=.13).

Figure 1.

Mean of real-time discomfort ratings in the short and extended trials.

Measures of remembered utility

Proportion of participants who chose the extended trial, for each of the questions in presented in Table 1. There was no effect of list order on any measure of remembered utility (F<1, p>.05). The main variable of interest was choice for future study. Most chose to repeat the extended list tomorrow (78%, χ²=8.33, p<.01) as well as for List 3, (82%, χ²=10.70, p<.01). One participant gave inconsistent responses to these questions. As can be seen in Table 1, except for the response regarding list duration, and the response for overall discomfort, which just missed significance, the list evaluation results replicated those in Experiment 1.

Correlations between future study preferences and study evaluations revealed preferences for the study experience judged less taxing overall. The correlations between both list choice for tomorrow and for List 3 with difficulty rating was r_ϕ= .90, p<.01 and r_ϕ= .81, p<.01, respectively. The correlations were significantly positive between future choices and discomfort response (Tomorrow: r_ϕ= .38, marginal at p=.05, List 2, r_ϕ= .47, p<.05), coping response (Tomorrow: r_ϕ= .57, p<.01, List 3, r_ϕ= .66, p<.01) and long list response (Tomorrow: r_ϕ= .70, p<.01, List 3: r_ϕ= .62, p<.01).

General Discussion

The current investigation extends prior research on remembered utility to the domain of learning. Two experiments showed that the structure of a learning episode can influence students' evaluations about their study experiences. Experiment 2 showed that ratings of discomfort were sensitive to the change in the difficulty of the materials at the end of the list. Importantly, while these items were rated as less aversive than the more challenging items, they were still rated negatively. Like other aversive experiences, effortful study that ended with somewhat easier material was judged to be less difficult, to cause less discomfort, and to be easier to cope with than a study experience that included equally challenging material but that did not include the more moderate material at the end. Second, the study showed that this evaluation shaped future study choices. When asked which type of study experience they would rather repeat, students chose the experience in which a period of effortful study was followed by a period of somewhat less effortful study. They made this choice despite having had better test performance in the short as compared to the extended list.

A clear prediction of the peak end rule is that the addition of more moderate items to the beginning or to the middle of the list, rather than to the end, should not result in a preference for the extended list over the shorter list. That is, according to the peak end rule, the preference for the extended list should only be observed when the more moderate items are presented at the end of the list, since it is the end of the event that is weighted most heavily in people's calculations of hedonic value. But, preference for the more moderately ending list could arise because people simply average subjective moments that occur over the entire experience (Fredrickson & Kahneman, 1993; Diener et al., 2001 and see also, Anderson, 1981; 1965 for a contrast of averaging versus additive processing in impression formation). The extended list should always be preferred then since the average will include these more moderate moments as compared to the short list. Recent research supports the idea that the end of an event is weighted more heavily comes from Diener et al., (2001) who demonstrated that a moderate period embedded in the middle of an event did not produce the same strong effects as when it was situated at the end. If people were averaging over the entire event then a moderate middle should have given rise to the same effects as a moderate end. It remains possible however, that a list containing less challenging items would be preferred regardless of where in the list the moderate words were presented. A list with a period of relative ease, wherever it occurs, may be preferred because it provides a mental break, or because it includes items that can be apprehended. Follow-up investigations are directed at varying the position of the added items.

Not all academic challenges can be pleasurable. Indeed, learning tasks that require intense mental effort may even be painful. While muscle tension, pupil dilation, heart rate changes, increased blood pressure, not to mention a decreased sense of well-being and eventual fatigue are all typical responses to pain, they are also responses invoked in tasks that require intense mental effort (Beatty, 1982; Critchley, et al., 2003; Kennedy & Scholey, 2000; Wilkinson, 1962). These responses can impact negatively on affect (Fairclough & Houston, 2004; Hockey, 1993; 1997), which consequently may decrease one's motivation to continue study.

The question of how best to encourage students to engage with material that they find challenging or even aversive to learn has obvious educational relevance. Students are often confronted with challenging learning experiences, where mastery of the material is necessary for academic success. A recent report from the National Survey of Student Engagement (NSEE, 2008) found that students are not spending nearly as much time studying for class as their instructors feel is adequate. While professors expect full time university students to spend in the area of 25 hours per week preparing for class, only about 11% of freshmen achieve that target (NSEE, 2008). An especially disheartening report on high school dropout rates found that there are 6.2 million dropouts per year in the U.S. (Center for Labor Market Studies, 2009) with academic challenge offered by students as one of the top reasons for dropping out of high school (Bridgeland, DiIulio Jr. & Burke Morison, 2006). Obviously many factors contribute to student achievement. However, insofar that students are expected to invest their time in demanding academic tasks, adding a better end to a study episode may be one simple way to increase the utility of study and the motivation for future study of material that is the most effortful to learn.

Happily for students many academic challenges are pleasurable experiences. Evidence from studies investigating the remembered utility of positive experiences finds that the best way to end a pleasurable experience is also on a high note (Diener, Wirtz & Oishi, 2001; Do, Rupert and Wolford, 2008; Fredrickson & Kahneman, 1993). Experiences that end very well are rated as more pleasurable than longer, more moderately pleasant experiences, even though ostensibly the total happiness experienced is greater in the longer case. Do et al., for example, found this pattern in children's retrospective evaluations about positive experiences. Trick or Treaters given a large Hershey bar (very pleasing) and then a piece of bubble gum (mildly pleasing) were less pleased than children who had just received the Hershey bar. Because the second positive was somewhat less positive than the first, the entire experience was diminished. These findings might have a place in the realm of positive learning experiences. When students are taking pleasure in learning, the best place to end the lesson may be when their success rate is high.

A multitude of studies (see Fredrickson, 2000, for a review) have shown that future decisions are informed by memory of previous pain and pleasure. The present study demonstrates that study choices are also guided by remembered utility. There are a number of important questions that arise from this line of research about the metacognitive control involved in self-directed learning. For example, after ending on a high note, are people more likely to select similar material to study for a follow up list? Or when given the opportunity are they more likely to switch to a new topic? We may be able to use memory biases regarding the final stages of an experience to our advantage to shape learning behavior, to increase the value of a learning episode, and to boost motivation for studying challenging material for longer periods of time.

In sum, the present findings extend our understanding about metacognitive evaluations and decisions about learning. In addition, they demonstrate a novel extension of theory about duration neglect in retrospective evaluations. The results here demonstrate that the utility of a learning episode goes beyond simply assessing the value of the information that has been acquired. The means of acquiring knowledge also has value for the student.