Are people really more patient than other animals? Evidence from human discounting of real liquid rewards

Abstract

In previous studies, researchers have found that humans discount delayed rewards orders of magnitude less steeply than do other animals. Humans also discount smaller delayed reward amounts more steeply than larger amounts, whereas animals apparently do not. These differences between humans and animals might reflect differences in the types of rewards studied and/or the fact that animals actually had to wait for their rewards. In the present article, we report the results of three experiments in which people made choices involving liquid rewards delivered and consumed after actual delays, thereby bridging the gap between animal and human studies. Under these circumstances, humans, like animals, discounted the value of rewards delayed by seconds; however, unlike animals, they still showed an effect of reward amount. Human discounting was well described by the same hyperboloid function that has previously been shown to describe animal discounting of delayed food and water rewards, as well as human discounting of real and hypothetical monetary rewards.

Daily life often requires choices between rewards that differ on multiple dimensions. One common situation involves choosing between a larger delayed reward and a smaller reward that is available sooner. In such situations, the subjective value of a reward is typically inversely related to the delay until its receipt—a phenomenon termed delay discounting. Such discounting of delayed rewards has been observed in a wide range of species, including pigeons and rats (Green, Myerson, Holt, Slevin, & Estle, 2004; Richards, Mitchell, de Wit, & Seiden, 1997), monkeys (Stevens, Hallinan, & Hauser, 2005; Woolverton, Myerson, & Green, 2007), and humans (for a review, see Green & Myerson, 2004), suggesting that delay discounting is a fundamental aspect of decision making.

The decrease in the subjective value (V ) of a reward as the delay (D) until its receipt increases is well described by a hyperboloid function:

$$V=A∕{(1+kD)}^s,$$ (1)

where A represents the amount of delayed reward, k is a constant governing how steeply the reward is discounted, and s is a nonlinear scaling parameter (Green & Myerson, 2004). Although Equation 1 describes discounting by both human and nonhuman animals, two notable differences have emerged. First, animals discount delayed rewards very steeply, so that even rewards that are delayed by as little as several seconds are judged to be of much less value than is an immediate reward. In humans, discounting effects are typically observed on time scales that are orders of magnitude longer (e.g., weeks or months vs. seconds). Second, humans show a magnitude effect, discounting smaller reward amounts more steeply than larger amounts, whereas no such magnitude effect has been observed in previous studies with animals (e.g., Freeman, Green, Myerson, & Woolverton, 2009; Green et al., 2004; Mazur, 2000).

These apparent differences between humans and animals, however, might be attributable to the types of rewards studied and/or to the fact that animals actually experienced having to wait for real rewards. Consistent with this interpretation, humans show steeper discounting when making decisions about directly consumable rewards (e.g., candy, soda, beer) as opposed to monetary rewards, perhaps because money is fungible or because it is not a primary reinforcer (Estle, Green, Myerson, & Holt, 2007; Odum & Rainaud, 2003). For example, Estle et al. found that the subjective value of 40 cans of soda decreased by 50% when receiving them was delayed by 6 months, whereas it took three times as long (18 months) for the subjective value of $40 to show a similar decline. Nevertheless, even with consumable rewards, the discounting function for humans in these studies was still much shallower than that observed in animals, suggesting greater patience. However, the rewards and delays in these previous studies were always hypothetical.

Previous studies have shown that humans vary widely in the tendency to make impulsive choices when directly consumable rewards (e.g., juice) follow real delays, and that some humans make much more impulsive choices under such conditions than they do when the rewards are money or points exchangeable for money or juice (see, e.g., Forzano & Logue, 1992, 1994). However, only one previous study using directly consumable rewards presented multiple delays in order to map out a discounting function (McClure, Ericson, Laibson, Loewenstein, & Cohen, 2007). In this study, however, a participant might not receive a chosen reward until after several other choices had been made or until after several other rewards had been received in the interval following the choice, thereby allowing for confusion regarding which choice was associated with which reward.

The present study was designed to bridge the gap between the experimental paradigms that are used to study delay discounting in humans and other animals, with the goal of shedding light on possible species differences. In three experiments, thirsty participants made repeated choices between smaller immediate and larger delayed liquid rewards, and they actually received the rewards at the time they selected. In each experiment, an adaptive, adjusting-amount procedure was used to determine the amount of immediate reward equal in subjective value to a larger, delayed reward, and subjective value was assessed at several delays (ranging from 5 to 60 sec) in order to estimate the discounting function describing the decrease in subjective value as delay increased.

In the first experiment, trial duration was held constant within each of four delay conditions, regardless of whether the immediate or delayed reward was chosen, but it varied across the delay conditions. In the second experiment, trial duration was held constant across all four delay conditions, and trials involving different delays were randomly interleaved (rather than being blocked, as in the first experiment). Trial duration was again held constant in the third experiment, which examined the effect of the amount of delayed reward on the rate of discounting. Across all three experiments, we found that under conditions in which the delays are actually experienced and the rewards are actually consumed, human participants (like other animals) show significant discounting, even over delays of 30 sec or less.

EXPERIMENT 1

Method

Participants

Fifteen undergraduate students received either course credit or $10 for their participation. Participants were instructed not to drink any liquid for 4 h before the experiment. Data from the 1 participant who failed to meet this criterion were discarded.

Apparatus

E-Prime programs (Psychology Software Tools, Inc.) controlled the behavioral task as well as the delivery of liquid rewards via a syringe pump (SP210iw, World Precision Instruments, Inc.). Liquids from two 60-ml plastic syringes mounted on the pump were merged into one tube and then delivered to the participant's mouth through a plastic tube. The simultaneous use of two syringes allowed for a comfortable flow rate of 2.0 ml/sec. The reward was delivered in 0.4-ml squirts but was experienced as a continuous flow. The amount of reward was determined by the number of squirts.

Procedure

Participants were tested individually in a small room. They were informed that the purpose of the study was to examine how people make choices involving delayed rewards, and they were asked to choose one favorite drink that would serve as the reward from a list consisting of apple, orange, grape, grapefruit, and cranberry juices, lemonade, and water. Instructions were read aloud to the participants, and they were informed that the duration of the experiment would not be affected by whether they chose the immediate or the delayed option.

At the beginning of each trial, two alternatives were presented side by side on the computer screen: a fixed 16-ml reward (40 squirts) that was available after a delay, and a smaller amount that was available immediately. We varied whether the larger, delayed amount was displayed on the left or right side from trial to trial. Participants pressed either the “1” or “2” key on the computer keyboard to indicate whether they preferred the left or right alternative, respectively. If the smaller, immediate amount was chosen, a message appeared on the screen indicating that reward delivery could now begin. If the delayed 16-ml reward was chosen, the participant had to wait to receive the reward. During the delay, the time remaining (in seconds) until the reward would be available was indicated on the screen, below which a green horizontal bar whose length was proportional to the number of seconds remaining was displayed.

Regardless of which option was chosen, when the reward became available, participants saw the message, “The reward is ready, press either key to begin.” Reward delivery continued as long as the key was held down; if the key was released, delivery paused, and then resumed when the key was pressed again. During reward delivery, the amount remaining (in squirts) was displayed below a red horizontal bar whose length corresponded to the number of squirts still available.

Four delay conditions (5, 15, 30, and 60 sec) were presented in a blocked design with three trials/block. On the first trial of each block, the choice was between a smaller, immediate reward of 8 ml (20 squirts) and the larger 16-ml delayed amount. On the following two trials, the amount of immediate reward was adjusted on the basis of the participant's preceding choice. If the participant had chosen the smaller, immediate reward on the preceding trial, then the amount of the immediate reward was decreased by half; if the participant had chosen the larger, delayed reward on the preceding trial, then the amount of the immediate reward was increased by half. More specifically, on the second trial, the immediate reward was increased or decreased by 4 ml, and on the third trial, by 2 ml. The subjective value for the delayed reward was estimated to be equal to 1 ml more or less than the amount of immediate reward available on the third trial, depending on whether the delayed or immediate reward had been chosen on that trial. The order in which the delay conditions were presented was counterbalanced across participants.

The time from the occurrence of a choice response until the presentation of the next pair of choice alternatives depended on the delay condition but was fixed within a delay condition at the duration of the delay plus 25 sec, to allow time for reward consumption. Thus, for the 5-, 15-, 30-, and 60-sec delay conditions, the next trial began 30, 40, 55, and 85 sec after a participant's choice response, regardless of whether the immediate or delayed reward was selected.

The experimental session began with a forced choice trial and two practice trials, which familiarized participants with the choice procedure as well as with the rewards and delays. In the forced choice trial, a 16-ml reward was presented after 20 sec. In the practice trials, participants made a choice between a 16-ml reward after 20 sec and either an immediate 8-ml reward (first practice trial) or an immediate 4- or 12-ml reward (second trial, with the amount depending on the choice made in the first practice trial). Additionally, prior to each new delay block, another forced choice trial was provided to allow participants to experience the delay to be studied in the upcoming block of trials. The syringes were refilled after each delay condition. The maximum amount of liquid that could be obtained per session was 304 ml.

Results and Discussion

Figure 1 shows subjective value (as a proportion of the actual reward amount) plotted as a function of the delay to the reward. As may be seen, subjective value decreased as the delay increased. The curve represents a hyperboloid function (Equation 1) fit to the group mean data (k = 1.02, s = 0.18, R² = .957). A planned contrast on the subjective values from individual participants revealed a significant linear decrease as a function of log delay¹ $[F(1,13)=5.92,p<.05,{\eta }_{\mathrm{p}}^2=.33]$. Thus, even though trial duration was constant within each delay condition in the present experiment so that participants could not increase their rate of reward by choosing the immediate over the delayed reward, they discounted the value of the liquid rewards even when they were delayed by only 30 sec or less. Moreover, the same hyperboloid function that describes human discounting of hypothetical rewards on a longer time scale also described the steeper discounting that was observed in the present experiment.

Figure 1.

The mean subjective values of the delayed reward, as a proportion of the actual delayed amount, in Experiment 1. The curve represents the best-fitting hyperboloid function; error bars indicate the standard errors of the means.

EXPERIMENT 2

The participant's choice of the immediate or the delayed reward had no effect on the overall rate of reward in Experiment 1, but the rate of reward did vary across delay conditions. In Experiment 2, we provided a systematic replication in which the rate of reward was held constant across all delays. In addition, choice trials involving different delays were randomly interleaved rather than blocked, as in the previous experiment.

Method

Participants

Sixteen undergraduate students received either course credit or $10 for their participation. Participants were instructed not to drink any liquid for 4 h before the experiment. The data from the 1 participant who chose the immediate option on all practice and test trials were discarded.

Procedure

The apparatus and procedure were identical to those in Experiment 1, except for the following. In Experiment 2, the trial duration (from choice response to the presentation of the next pair of choice alternatives) was held constant at 85 sec, regardless of the delay condition. In addition, the time to the delayed reward alternative (5, 15, 30, or 60 sec) varied randomly from trial to trial. Finally, after completing the three trials that made up each delay condition, there was a replication trial in which participants were given a choice between the same alternatives as those on the first trial of that condi tion (i.e., a delayed 16-ml reward or an immediate 8-ml reward) in order to assess preference reliability.

The experimental session began with two forced choice trials, one in which a 16-ml reward was presented after 20 sec, and another in which an 8-ml reward was presented immediately. The syringes were refilled after every six trials, and the maximum amount of liquid that could be obtained per session was 312 ml.

Results and Discussion

As may be seen in Figure 2, subjective value decreased as a function of delay, and the hyperboloid discounting function provided an excellent fit to the data (k = 0.066; s = 0.563; R² = .988). A planned contrast indicated a linear decrease in subjective value as log delay increased $[F(1,14)=58.9,p<.00001,{\eta }_{\mathrm{p}}^2=.79]$.

Figure 2.

The mean subjective values of the delayed reward, as a proportion of the actual delayed amount, in Experiment 2. The curve represents the best-fitting hyperboloid function; error bars indicate the standard errors of the means.

Recall that after completing the three trials that made up each delay condition, participants were given a choice between the same alternatives as those on the first trial of that condition (i.e., a delayed 16-ml reward or an immediate 8-ml reward). The concordance between the first trial and the replication trial for the 15 participants across the four delay conditions was 76.7% (46 cases out of 60)—significantly greater than chance [t(14) = 5.87, p < .0001]—attesting to the relative stability of their preferences.

As in Experiment 1, participants discounted delayed rewards on the order of seconds, and their data were well described by a hyperboloid function. The results of Experiment 2 confirm that this pattern holds when the rate of reward is held constant across all delay conditions. The subjective value of the liquid reward decreased by 50% when it was delayed by only 30–40 sec. This is comparable to the results of Richards et al. (1997)—who found that, in rats, the subjective value of a 100-μl water reinforcer decreased by 50% when it was delayed by about 6 sec—particularly when one considers that the rats had not had any liquid in over 23 h. Taken together, the findings strongly suggest that the apparent species differences in discounting rates that one sees when comparing previous studies of discounting by humans and other animals were a result of procedural differences rather than a reflection of fundamental differences between species.

EXPERIMENT 3

The first two experiments ruled out one supposed difference between discounting by humans and other animals—namely, the apparent difference in rate of discounting. However, there is another, equally important possible species difference. Humans have been shown to discount larger delayed rewards less steeply than smaller ones, and such effects of reward magnitude are extremely robust, at least with delayed monetary rewards (Green & Myerson, 2004). Previous studies with nonhuman animals, in contrast, have not found a magnitude effect (Green et al., 2004; Richards et al., 1997), even in monkeys (Freeman et al., 2009). Again, however, most of the human studies that compared the discounting of different reward amounts did not involve directly consumable rewards, and in those that did, the rewards and the delays were hypothetical. Accordingly, Experiment 3 revisited this issue using two different amounts of actual liquid rewards that the participants had to wait to receive.

Method

Participants

Sixteen undergraduate students received either course credit or $10 for their participation. Participants were instructed not to drink any liquid for 4 h before the experiment. The data from 1 participant who failed to meet this criterion were discarded.

Procedure

The apparatus and procedure were identical to those in Experiment 2, except for the following. There were three delay conditions (10, 25, and 60 sec) in which the delayed reward was 16 ml (40 squirts), and two delay conditions (10 and 25 sec) in which the delayed reward was 8 ml. Within each delay/amount condition, the procedure used to adjust the amount of the immediate reward was similar to that used previously. There were three choice trials per delay when the delayed reward was 16 ml, but only two choice trials per delay when the delayed reward was 8 ml. Both the amount of time until receipt of the delayed reward and the amount of the delayed reward varied randomly from trial to trial. The syringes were refilled after every seven trials, and the maximum amount of liquid that could be obtained per session was 296 ml.

Results and Discussion

As may be seen in Figure 3, the subjective value decreased as a function of delay, for both the 16- and 8-ml rewards. The hyperboloid function was fit simultaneously to the data for both amounts using a separate k parameter for each amount and a single s parameter. The hyperboloid again provided a very good fit (16 ml; k = 0.034; 8 ml; k = 0.085; s = 0.635; R² = .956). As in Experiments 1 and 2, a planned contrast indicated a linear decrease in subjective value of the 16-ml reward as log delay increased $[F(1,14)=13.0,p<.005,{\eta }_{\mathrm{p}}^2=.48]$. The concordance between participants’ choices on the first trial of each delay condition and on the replication trial for the 16-and 8-ml amounts was 73.3%, significantly greater than chance [t(14) = 4.06, p < .005].

Figure 3.

The mean subjective values of the delayed rewards, as a proportion of their actual amounts, in Experiment 3. The curves represent the best-fitting hyperboloid function fit to the data for both amounts simultaneously; error bars indicate the standard errors of the means.

Although estimates of subjective value were based on three choice trials at each of three delays for the 16-ml amount, there were only two choice trials at two delays (10 and 25 sec) for the 8-ml amount (not counting the replication trials). Therefore, when we compared subjective values at both amounts, we used only the first two trials at each amount for the 10- and 25-sec delay conditions. A 2 (amount) × 2 (delay) repeated measures ANOVA revealed significant main effects of delay $[F(1,14)=6.7,p<.05,{\eta }_{\mathrm{p}}^2=.32]$ and amount $[F(1,14)=37.2,p<.001,{\eta }_{\mathrm{p}}^2=.73]$, as well as an amount × delay interaction $[F(1,14)=6.3,p<.05,{\eta }_{\mathrm{p}}^2=.31]$, reflecting the fact that the difference between the two amount conditions was greater at 25 sec than at 10 sec. The amount effect was apparent also at the individual level: When the subjective value of each reward amount (as a proportion of its actual amount) was averaged across the two delays for each participant, the subjective value for the 8-ml amount was less than that for the 16-ml amount for 13 of the 15 participants, reflecting steeper discounting of the smaller amount.

The results replicate those from the first two experiments showing that humans, like animals, discount the value of directly consumable rewards that are delayed by seconds. However, unlike animals, the human participants showed a magnitude effect similar to that observed in previous studies in which humans discounted delayed hypothetical rewards. That is, the smaller liquid reward was discounted significantly more steeply than was the larger liquid reward.

GENERAL DISCUSSION

In three experiments, thirsty participants made repeated choices between smaller immediate and larger delayed liquid rewards, and they actually consumed the rewards at the time they had selected. In Experiment 1, participants discounted the value of rewards delayed by only 30 sec or less. This occurred despite the fact that trial duration was held constant within each delay condition so that rate of reward was not affected by participants’ choices and (as they were told) so that the duration of the experiment was not affected by whether they chose the immediate or the delayed option. In Experiment 2, the trial duration was held constant, not only within each delay condition, but across them as well, and delay conditions were randomly interleaved rather than blocked, as in the first experiment. Despite these methodological changes, participants again discounted rewards that were delayed by a matter of seconds. Finally, in Experiment 3, we compared discounting of 16-ml rewards, like those used in the previous two experiments, with discounting of 8-ml rewards, and these smaller delayed rewards were discounted even more steeply than were the larger rewards.

It has long been assumed that humans show greater patience and self-control than do other animals, and results showing that humans discount delayed rewards orders of magnitude less steeply than do animals have been taken as support for this view. Nevertheless, the appropriateness of this comparison is open to question, given the many methodological differences between the human and animal studies. Chief among these differences is the fact that research with humans, for the most part, has involved participants who make choices on the basis of hypothetical scenarios, whereas in animal studies, the subjects have had to wait for real, directly consumable rewards.

Previous human studies have shown that it makes surprisingly little difference whether monetary rewards are real or hypothetical (e.g., Lagorio & Madden, 2005), whereas the type of reward (i.e., monetary vs. directly consumable) makes a big difference, even when the rewards are hypothetical (e.g., Estle et al., 2007; Odum & Rainaud, 2003). The present study, however, is the first to report discounting functions from human participants who (like subjects in animal experiments) made their choices and then—on trials in which they chose the delayed outcome—actually had to wait in order to consume their reward without other events intervening. Moreover, as in the animal studies, participants were mildly deprived of the kind of rewards (i.e., liquids) that they would consume in the experiment.

Thus, in the present experiments, we demonstrated that when tested under relatively comparable conditions, in terms of actual delays and directly consumable rewards, the difference between humans and other animals in how steeply rewards are discounted is markedly reduced. The similarity in delay discounting between humans and other animals tested under relatively comparable conditions suggests that similar decision-making processes may be involved. Nevertheless, one important behavioral difference remains: Humans—but not animals—show robust magnitude effects, discounting larger delayed rewards less steeply than smaller rewards. It is still possible, of course, that one of the remaining procedural differences (e.g., the fact that reward amounts and delays were symbolically signaled in the present experiments) may explain the apparent lack of a magnitude effect in animals.

There are, of course, differences in how humans and animals make decisions under some circumstances, and there are also differences between the circumstances in which they typically make decisions. We would emphasize, however, that when tested under relatively comparable circumstances, the differences in how steeply humans and animals discount the future consequences of their choices are much smaller than people have previously thought.