Abstract
In humans, the order of receiving sequential rewards can significantly influence the overall subjective utility of an outcome. For example, people subjectively rate receiving a large reward by itself significantly higher than receiving the same large reward followed by a smaller one (Do, Rupert, & Wolford, 2008). This result is called the peak-end effect. A comparative analysis of order effects can help determine the generality of such effects across primates, and we therefore examined the influence of reward-quality order on decision making in three rhesus macaque monkeys (Macaca mulatta). When given the choice between a high–low reward sequence and a low–high sequence, all three monkeys preferred receiving the high-value reward first. Follow-up experiments showed that for two of the three monkeys their choices depended specifically on reward-quality order and could not be accounted for by delay discounting. These results provide evidence for the influence of outcome order on decision making in rhesus monkeys. Unlike humans, who usually discount choices when a low-value reward comes last, rhesus monkeys show no such peak-end effect.
Keywords: Decision making, Peak-end rule, Order preferences, Contrast effects, Rhesus monkeys
According to utility maximization theory, individuals should prefer options that maximize overall utility regardless of the order of outcome events (Kahneman & Tversky, 1979). However, Kahneman and colleagues showed that people preferred a longer duration of a painful water bath that ends slightly less painfully to a shorter one that does not change in pain intensity, even when the former option entailed more cumulative pain overall (Kahneman, Fredrickson, Schreiber, & Redelmeier, 1993). Similarly, Redelmeier and colleagues (Redelmeier & Kahneman, 1996; Redelmeier, Katz, & Kahneman, 2003) showed that colonoscopies were rated less unpleasant when they lasted longer but ended more gently to those that were significantly shorter but did not change in intensity. Aversive loud noises were also rated as less negative when ending on a less aversive note (Schreiber & Kahneman, 2000). These findings run counter to the principle of maximizing overall utility and suggest a peak-end rule in humans whereby alternatives that end less negatively are preferred, even at the expense of overall utility maximization.
The examples mentioned so far involve negative, aversive stimuli, but positive stimuli show similar effects. Both adults and children (elementary and middle school) subjectively rated their experience higher when receiving a high-valued reward by itself (a highly-rated DVD movie for adults, candy for children) against a higher valued reward followed by a lower valued reward (a lower quality movie for adults and a less desired candy for children; Do, Rupert, & Wolford, 2008). In another study, people rated advertisements more positively when the interval length between their highest positive feelings for the advertisement and the end of the advertisement was shorter, and they rated them more negatively when the interval length was longer (Baumgartner, Sujan, & Padgett, 1997). These results suggest a peak-end preference whereby people rated a subjective experience based on how well the experience ended, even at the expense of overall utility. Although significant differences have been found between testing paradigms that use choice preferences versus subsequent outcome evaluations (see Wilson & Gilbert, 2005), the peak-end effect appears to hold for both ratings (Baumgartner et al., 1997; Do et al., 2008; Redelmeier et al., 2003; Schreiber & Kahneman, 2000) and choices (Kahneman et al., 1993; Redelmeier et al., 2003).
To determine the generality of order preferences among primates, it is important to conduct a comparative analysis with other species. Some chimpanzees have been found to exhibit a selective-value effect whereby individuals appeared to value equally a higher valued food item in isolation and the same higher valued item in conjunction with a lower valued one. The suggestion was that the lower valued food item was of no value in the presence of the better one (Silberberg, Widholm, Bresler, Fujita, & Anderson, 1998). However, it was unclear whether there was any order preference since the food items were offered simultaneously. Furthermore, chimpanzees switched their preferences to the food mixture when the study was conducted with equalized trial lengths and longer delays between trials (Beran, Ratliff, & Evans, 2009), suggesting utility maximization trial by trial when the chimpanzees were unable to receive the preferred item in quick succession across trials. Notably, however, when the chimpanzees chose the food mixture, they tended to eat the preferred food item first the majority of the time (Beran et al., 2009).
The result for rhesus monkeys have been inconsistent, although related studies have shown them to be relatively impulsive and also subject to cognitive biases such as the selective-value effect. Evans and Beran (2007) showed that rhesus macaques rarely exceed a 30-s delay in a delay-maintenance task, while great apes (chimpanzees and an orangutan) averaged up to 180-s delays in the same task (Beran, 2002) and upwards to 300 s in some chimpanzees (Beran, Savage-Rumbaugh, Pate, & Rumbaugh, 1999). Thus, rhesus monkeys appear to be more impulsive than the great apes. In additionHayden Heilbronner, Nair, and Platt (2008) found that rhesus monkeys were significantly more sensitive to changes in high-reward outcomes than to changes in low-reward outcomes; and they may also show a selective-value effect (Silberberg et al., 1998) in which a larger reward may draw attention away from a smaller one.
Heightened impulsiveness and sensitivity to the higher valued rewards might imply a peak-first preference when goods are obtained sequentially, and this effect could be due to the preferred food dropping in value the longer a subject has to wait for it (Hayden & Platt, 2007). However, it is possible that there is something special about the order of receiving outcomes that is independent of delayed discounting and risk assessment. Humans use aphorisms such as “save the best for last” or “all’s well that ends well”, which reflects the effect of order independent of time or the risk of losing the final reward.
Overall, then, there is evidence for order preferences in humans and some nonhuman primates, but the effects are mixed, and the number of species tested so far is relatively small. In the current study, we tested rhesus monkeys (Macaca mulatta) to determine whether they would exhibit an order preference. Due to heightened impulsiveness and sensitivity to large rewards, we hypothesized that the rhesus monkey choices would be affected by order and, furthermore, that they would more likely exhibit a peak-first preference—that is, preferring to receive the best item first rather than last.
GENERAL METHOD
If A represents the higher valued item, and B represents the lower valued one, we represented the sequence of receiving A then B as A → B, and the sequence of receiving B then A as B → A. We tested A → B versus B → A directly to determine whether order of outcomes influenced rhesus monkey choice behaviour. Given the comparative literature discussed in the introduction suggesting a peak-first preference, for the current experiment, we hypothesized that all monkeys would prefer A → B over B → A.
Subjects
We tested three male rhesus monkeys, denoted as monkey T, P, and H, ages 9, 7, and 7 years, respectively. All monkeys were maintained at approximately 95% of their ad libitum weights. The monkeys were housed in a homeroom and were rolled to the testing room in our laboratory in custom-made chairs. All three subjects were used in one response time and one forced-choice behavioural experiment (Knight, Klepac, & Kralik, 2011). All experiments in this report complied with all current laws and regulations of the United States, the United States Department of Agriculture (USDA), and the Institutional Animal Care and Use Committee (IACUC) of Dartmouth College for the use of animals in research.
Materials
In the test room the monkey’s chair was placed across a table (30 inches) from the experimenter. The monkey sat in the chair, with his left arm loosely restrained (using a custom-made metal ring attached to the chair around both the upper and lower arms) and the right arm free to reach. The custom-made chairs were used for tighter experimental control over behaviour. Two grey opaque canisters were used as stimuli. Using construction paper covered by clear tape, we changed each canister’s lid to a particular colour that was easily viewable by the monkey. The colour corresponded to each reward contingency described below (see specific methods and tables for all experiments). In Experiments 1–4 for monkeys P and H, and Experiments 1 and 4 for monkey T, a button panel consisting of two unlit buttons was centred in front of the monkey. We removed the button panel for monkey T in experiments where he exhibited protracted side biases during learning of the reward contingencies (removing the panel relieved these biases). We removed the button panel for all monkeys in Experiment 5. The monkeys made selections by touching one of the two buttons corresponding to the canister lid presented across from it. When the button panel was removed, the monkeys made selections by directly touching the canister’s coloured lid. All experimental sessions were videotaped.
Procedure
We tested each monkey on one session per day. Each session consisted of 10 familiarization trials and 30 experimental trials. With this total number of trials, there was no evidence of satiation effects (all monkeys ate the food items in all sessions). The purpose of familiarization was to allow the monkeys to learn the coloured lid and reward contingencies, which did not change within an experiment. During familiarization, one canister was presented per trial, randomized among the two different colours (for colour and reward contingencies see tables within each individual experiment) in the experiment and two different positions (left or right). This procedure was modified for monkey T in Experiments 2, 3, and 5 to facilitate the learning of the reward contingencies: Two canisters were used in familiarization, with the second one having a red lid and yielding no reward upon choice. Upon choice in both procedures, the canister’s lid was pointed back towards the experimenter so the food within it was not viewable to the monkey. The lid was then removed, and the monkey received the contents based on the experimental contingencies. The intertrial interval was approximately 8 s.
All experimental trials were conducted using a two-alternative forced-choice procedure. Each trial began when the experimenter held two closed opaque canisters 9 inches from the edge of the button panel, or 18 inches from the monkey if the button panel was not used. We held the canisters at an angle so that the coloured lids were easily viewable to the monkeys, held them in place until the monkeys had looked at the alternatives for approximately 3 s, then brought them forward to the edge of the button panel or within reaching distance of the monkey if the button panel was not used. Canister positions were pseudorandomized so that the same positions did not occur more than twice in a row.
To minimize potential inadvertent cues from the experimenter, several procedures were implemented: (a) White noise was played to minimize any auditory distractions; (b) the experimenter wore a white lab coat, mask, safety goggles, and medical gloves throughout the session; (c) five different experimenters tested the monkeys, following a stereotyped procedure, and we obtained no differences between experimenters.
Three significant sessions (two-tailed binomial tests per session, p < .05) or 10 consecutive non-significant sessions were required from each monkey before moving to the next experiment. The three significant sessions were not necessarily consecutive. We therefore considered the overall preference to be the average over all trials from the first significant session to the last significant session. We started with the first significant session because in some cases the monkeys needed extra time to learn the reward contingencies associated with each colour. Once there was one significant session, we obtained two more to verify stability. Ten consecutive nonsignificant sessions were taken to mean there was no preference by the monkey.
Food preferences were conducted to determine the superior/inferior food items. It was important for the inferior item to be consumed when presented alone so that it was not aversive and was of positive value. Because monkey P did not eat sugar snap peas as readily as monkeys T and H, green beans were used for that subject. After informal preference testing to determine the best food items to use (selecting a highly valued and lower valued item) prior to the study, we conducted one formal 30-trial session per monkey to verify preferences. All monkeys selected one grape over a half vegetable (green bean for monkey P, sugar snap pea for monkeys H and T) on every trial. We denoted the superior item (the grape) with the variable “A” and the inferior item (a half vegetable) “B”.
EXPERIMENT 1
Method
Subjects, materials, and procedure
These were as listed under “General Method”.
Specific procedure
Selection of one coloured lid resulted in the monkey receiving a grape, followed 4 s later by a half vegetable, denoted as A → B. The 4-s delay was added to provide time for the monkey to consume the food item. The other coloured lid gave a half vegetable, followed 4 s later by a grape, denoted as B → A. Table 1 shows lid colours and contingencies for each monkey.
Table 1.
Canister colours/contingency | ||
---|---|---|
Monkey | B → A | A → B |
P | Purple | Yellow |
H | Green | Blue |
T | Purple | Blue |
Note: A monkey selected from two canisters with coloured lids. Choice of one canister resulted in a half vegetable followed 4 s by a grape (B → A), and choice of the other resulted in a grape followed 4 s later by a half vegetable (A → B). Monkeys were given different colours in the experiments to avoid colour confounds.
Results
As shown in Figure 1, all three monkeys (P, H, and T) exhibited a significant preference for A → B over B → A, consistent with a peak-first rather than peak-end preference. All three sessions for monkey P were significant, selecting A → B on 70%, 86.7%, and 86.7% of the trials (two-tailed binomial tests: p < .05, p < .0001, p < .0001). Overall, monkey P selected A → B on 81.1 ± 9.6% of trials across the three sessions (χ2: p < .001). All three sessions for monkey H were also significant, selecting A → B on 70%, 73.3%, and 70% of the trials (two-tailed binomial tests: p < .05, p < .05, p < .05). Overall, monkey H selected A → B on 71.1 ± 1.9% of trials across the three sessions (χ2: p < .02). Monkey T took four sessions to have three significant ones, selecting A → B on 70%, 76.7%, and 70% of the trials (two-tailed binomial tests: p < .05, p < .01, p < .05). Overall, monkey T selected A → B on 70 ± 5.4% of trials across the four sessions (χ2: p < .02).
Discussion
Rather than displaying a possible peak-end or recency effect by preferring the outcome with the last and most recent best experience, the monkeys clearly preferred the alternative with the best item received first. This result potentially contrasts with humans, in which we appear to have a peak-end, recency preference. Given that rhesus monkeys may display the selective value effect and ignore the lower valued item (B; Silberberg et al., 1998), it is not clear how much the choice behaviour of any of the monkeys can be explained by order versus simply the delay of the preferred food item (A). In Experiments 2 and 3, we attempted to determine whether the lower valued item (B) affects choice in both the second and the first positions.
EXPERIMENT 2
In Experiment 1 we tested two different outcome orders against each other directly: A → B versus B → A. However, it was important to determine whether both food items in the sequence actually affected choice. Thus, in Experiments 2 and 3, we attempted to determine whether the lower valued item affects choice both in the second and in the first positions. In Experiment 2, we determined the value of B in the second position by comparing A → B to A alone. If B is of positive value, A → B should be preferred to A alone. If B is of zero value and not affecting choice, removing it should have no effect. That is, the monkeys should be indifferent to the choice alternatives. If B has negative value, A alone should be preferred. This last finding would be comparable to that of Do et al. (2008), in which people preferred A alone to A → B.
Method
Subjects, materials, and procedure
These were as listed under “General Method”.
Specific procedure
Choice of one coloured lid resulted in the monkey receiving one grape, followed 4 s later by a half vegetable (sugar snap pea or green bean), denoted A → B. Again, the 4 s allowed the monkey to eat the grape before receiving the vegetable. The other coloured lid led to the monkey receiving one grape, followed by a 4-s delay, denoted as A → [ ]. We added the delay to maintain comparable trial lengths for both choice alternatives. Table 2 shows the lid colours and contingencies for each monkey.
Table 2.
Canister colours/contingency | ||
---|---|---|
Monkey | A → [ ] | A → B |
P | Blue | Yellow |
H | Yellow | Blue |
T | Yellow | Blue |
Note: Choice of one canister resulted in a grape followed by a 4-s delay (A → [ ]), and choice of the other resulted in a grape followed 4 s later by half a vegetable (A → B).
Results
Two out of three monkeys (P and H) showed a significant preference for A → B over A → [ ] (see Figure 2). All of monkey P’s sessions in this experiment were significant, selecting A → B on 90%, 83.3%, and 73.3% of the trials (two-tailed binomial tests: p < 1.0 × 10−5, p < .001, p < .05, respectively). Overall, monkey P selected A → B on 82.2 ± 8.4% of all trials across the 3 sessions (χ2: p < .001). All of monkey H’s sessions were also significant, selecting A → B on 76.7%, 93.3%, and 86.7% of the trials (two-tailed binomial tests: p < .01, p < 1.0 × 10−5, p < .0001). Overall, monkey H selected A → B on 85.6 ± 8.4% of trials across the 3 sessions (χ2: p < .001). Because monkey T did not have even 1 significant session in 10 sessions, we concluded that he did not have a preference. Monkey T selected A → B on average 51.1 ± 10.2% of trials over his last 3 nonsignificant sessions (χ2: p > .5).
Discussion
For two monkeys, P and H, the preference for A → B over A → [ ] showed that B influenced the choice behaviour positively (or at least that the value of B was greater than the value of the short delay). And since A was more highly valued than B, both A and B (A → B) influenced the behaviour of these two monkeys positively, whereas, monkey T’s behaviour appeared to be influenced only by A, with B being essentially ignored and thus having zero value. This result appears to provide more evidence for the selective value effect, in which a lower valued food item is ignored and holds zero value when in the presence of a higher valued item (Beran et al., 2009; Silberberg et al., 1998), although in the current case, the effect occurred with the items presented sequentially rather than simultaneously.
Our results contrast with the peak-end findings in people in which participants significantly rated the experience higher when it was the highly valued reward in isolation (e g., Do et al., 2008). In the current experiment, none of the monkeys showed a preference for A alone. This difference could be due to a true species difference in order preferences. However, it is possible that differences in the testing procedures from that of the Do and colleagues’ experiment could underlie the opposing results. In Do and colleagues’ experiment, participants did not have a choice, and it is not certain that the subjects would choose the option that they would rate higher, especially with knowledge of both contingencies and experience selecting both alternatives. Conducting a choice paradigm in humans with repeated trials should help illuminate the factors influencing decision making and subjective value. In fact, interesting work with humans has shown that different results may be obtained between rating choice alternatives in isolation versus comparing them directly (Hsee, 1998; Hsee, Loewenstein, Blount, & Bazerman, 1999; Hsee & Zhang, 2004; List, 2002). For example, Hsee (1998) found that a 24-piece dinnerware set was rated more highly than one with the same 24 pieces, plus several more pieces, including ones that were broken. However, this preference reversed when the two options were directly compared. One therefore might predict that when given a choice, people will tend to select the alternative with the largest overall utility when making a direct comparison between A → B and A alone, especially when a fixed delay is included to eliminate receiving a quick succession of As over trials (see Beran et al., 2009). However, in the Kahneman et al. (1993) study, in which a peak-end effect was found when placing one’s hand into painfully cold ice water, and in the Redelmeier et al. (2003) study, in which people received colonoscopies, the participants were given a choice between two alternatives, and they still chose the option with more overall pain (less overall utility) that ended better. Thus, our results may also suggest a possible difference between humans and another primate species in that rhesus monkeys do not show a clear peak-end preference, at least under our test conditions.
EXPERIMENT 3
We next attempted to determine the value of B in the B → A alternative, whether being positive, zero, or even negative. To do this, we presented the monkeys with a choice between B → A and A alone, with A being presented at the same delay. If the value of B was positive, B → A should be preferred; if zero, the monkeys should be indifferent to the two alternatives; if negative, A alone should be preferred.
Method
Subjects, materials, and procedure
These were as listed under “General Method”.
Specific procedure
The selection of one coloured lid resulted in the monkey receiving a half vegetable, followed 4 s later by a grape, denoted B → A. Selection of the other coloured lid resulted in a 5-s delay, followed by a grape, denoted [ ] → A. The 5-s delay ensured that the grape was given at the same delay for either choice alternative (1 s to hand the first food item and the 4-s delay between items). Table 3 shows lid colours and contingencies for each monkey.
Table 3.
Canister colours/contingency | ||
---|---|---|
Monkey | B → A | [ ] → A |
P | Purple | Green |
H | Green | Purple |
T | Black | Grey |
Note: Choice of one canister resulted in a half vegetable followed 4 s by a grape (B → A), and choice of the other resulted in a delay of 5 s followed by a grape ([ ] → A).
Results
Two of the three monkeys (H and T) showed significant preferences for B → A over [ ] → A. Monkey H took 5 sessions to have 3 significant ones, selecting B → A on 80%, 76.7%, and 86.7% of the trials in the 3 significant sessions (two-tailed binomial tests: p < .01, p < .01, p < .0001). Again, taking overall preference across all sessions from the first significant session to the last, monkey H selected B → A on 74 ± 10.4% of trials across 5 sessions (χ2: p < .001; see Figure 3). Monkey T required 12 sessions to have 3 significant ones. In 3 consecutive sessions, monkey T selected B → A on 70%, 80%, and 83.3% of the trials (two-tailed binomial tests: p < .05, p < .01, p < .001). Overall, monkey T selected B → A on 77.8 ± 6.9% of trials across the 3 sessions (χ2: p < .001). Monkey P took 6 sessions to have 3 significant ones and showed the opposite effect, selecting B → A on 30%, 23.3%, and 16.7% of trials in the 3 significant sessions (two-tailed binomial tests: p < .01, p < .01, p < .0001). Overall, monkey P selected B → A on 26.7 ± 8.6% of all trials across the final 4 sessions (χ2: p < .01). Thus, monkey P preferred [ ] → A over B → A.
Discussion
Since two of the monkeys (H and T) preferred B → A over [ ] → A, for them the food item represented by B, the half vegetable, was of positive value when in the first position (or at least some value higher than the delay). Thus, Experiments 2 and 3 showed that for monkey H, the value of B was positive (or at least greater than the corresponding delay value in the other alternative) whether in the first or last position, and thus preferences were based on the assessment of both food items. For monkey T, however, the value of B was dependent on position, with B having no value in the last position (Experiment 1), but positive value in the first position.
For monkey P, when B was in the second position in Experiment 2, it had positive value; however, when switched to the first position here, he preferred [ ] → A over B → A, showing that the impact of B actually became negative. Thus, for this set of food items, monkey P clearly preferred receiving the best item first (rather than last), even with A delayed and an overall lower amount of food in that alternative. This negative rendering of the value of B is comparable to the findings obtained by Beran et al. (2009) in which the value of a lower valued food item (apple pieces) became negative for chimpanzees when the item was offered simultaneously with a higher valued item (banana pieces). A difference with that finding is that in their study the negative value reverted to positive once the experimenters equalized the trial lengths between choice alternatives and lengthened the intertrial interval (ITI). Here, the result was obtained with equal trial lengths, although longer ITIs would be required to make a more direct comparison with their study. Another difference between studies is that our reward items were presented sequentially rather than simultaneously. Thus, the negation of the value of B due to A following it may also result from a strongly negative anticipatory contrast effect (Flaherty, 1996; Williams, 2002), although this effect was typically found when responding during the B component decreased, which is a different testing paradigm from that in our study. A related possibility is that monkey P could be showing a specific flavour–flavour order preference whereby the consumption of the first item can affect the actual taste of the second one since he preferred receiving B if it came after A (Experiment 2) but not before (Experiment 1). Nonetheless, the preference found for monkey P does appear to be due to the order of the reward items and provides further evidence that there are contexts—in this case the particular order of certain food items—in which a positive outcome can be rendered negative for rhesus monkeys.
EXPERIMENT 4
To test how well the peak-first preference found in Experiment 1 generalized to other food items, we repeated Experiment 1 with two different food items. Because Experiment 1 showed that the monkeys preferred A → B over B → A, we hypothesized that the same preference for receiving the best item first would hold with the new food items.
Method
Subjects, materials, and procedure
These were as listed under “General Method”, except as described in the following section.
Specific procedure
Grapes were replaced with mini marshmallows, hereafter called marshmallows and denoted A2, and the half vegetables were replaced with a single Cheerio®, denoted B2. For all monkeys, the marshmallows were strongly preferred over Cheerios, and the Cheerios were readily eaten when offered alone. To verify monkey preferences, we first conducted one 10-trial session per monkey. All monkeys selected one marshmallow over one Cheerio on every trial. In addition, we conducted 3 to 4 trials of preference testing on monkey P between Trials 15 and 16—that is, in the middle of the session, during each of his three significant sessions to verify that his preference did not change: He selected the marshmallow on every trial.
Selection of one coloured lid resulted in the monkey receiving a marshmallow immediately, followed 4 s later by a Cheerio, denoted A2 → B2. The other coloured lid gave one Cheerio immediately, followed 4 s later by a marshmallow, denoted B2 → A2. Table 4 shows lid colours and contingencies for each monkey.
Table 4.
Canister colours/contingency | ||
---|---|---|
Monkey | B2 → A2 | A2 → B2 |
P | Brown | Pink |
H | Orange & blue cross | Black & white chequerboard |
T | Orange & blue cross | Black & white chequerboard |
Note: Selection of one canister resulted in a Cheerio followed 4 s later by a mini marshmallow (B2 → A2), and selection of the other resulted in a mini marshmallow followed 4 s later by a Cheerio (A2 → B2).
Results
Two of the three monkeys (H and T) showed a significant preference for A2 → B2 over B2 → A2 (see Figure 4). All three sessions for monkey H were significant, selecting A2 → B2 on 100%, 73.3%, and 93.3% of the trials (two-tailed binomial tests: p < 1.0 × 10−7, p < .05, p < 1.0 × 10−5). Overall, monkey H selected A2 → B2 on 88.9 ± 13.9% of trials across the three sessions (χ2: p < .001). All three sessions for monkey T were also significant, selecting A2 → B2 on 90%, 100%, and 96.7% of the trials (two-tailed binomial tests: p < 1.0 × 10−5, p < 1.0 × 10−7, p < 1.0 × 10−6). Overall, monkey T selected A2 → B2 on 95.6 ± 5.1% of trials across the three sessions (χ2: p < .001). All three sessions for monkey P were also significant and showed the opposite effect, selecting A2 → B2 on 20%, 13.3%, and 26.7% of trials in three consecutive sessions (two-tailed binomial tests: p < .01, p < .0001, p < .05). Overall, monkey P selected A2 → B2 on 20 ± 6.7% of trials across the three sessions (χ2: p < .001). Thus, monkey P’s preference was for B2 → A2 over A2 → B2.
Discussion
Overall, all three monkeys showed clear preferences. Monkey H and T once again exhibited a potential peak-first preference, suggesting that either (a) their choices were being influenced by only the first item received (thus, simply preferring A over B); or (b) receiving the best item first may be typically preferred by them, at least under these testing conditions. At the same time, with the two new food items, monkey P in fact showed a clear peak-end preference, which we find quite interesting. There was no evidence that a Cheerio was ever valued more highly than a marshmallow for any of the monkeys (we in fact conducted three to four trials of preference testing on monkey P between Trials 15 and 16—that is, in the middle of the session, during each of his three significant sessions, and he continued to select the marshmallow on every trial). Thus this finding appears to reflect a true peak-end preference for this monkey. The result therefore suggests that not all monkeys are compelled to obtain the best item first and/or immediately, and that the actual preferred order depends on the food items, at least for some monkeys.
Two possible factors underlying this preference reversal for monkey P are flavour–flavour order preferences and the differential value between the two food items. First, as in Experiment 3, it was possible that the consumption of the first item affected the actual taste of the second one, with the Cheerio being preferred before the marshmallow. Second, the differential value between the two food items might have caused the preference reversal. The difference in value between grapes and green beans might be much larger than that between marshmallows and Cheerios, causing a peak-first preference in the former case, but not in the latter. It would be important to determine in the future whether the relative differences between the two food items can reliably predict a peak-first or peak-end preference. Flaherty, Turovsky, and Krauss (1994) did find that the effect of the second sweetened solution on the licking response of rats to the first solution depended on the relative value of the two rewards; however, their results appeared to be the opposite of ours. In their study, when the first reward was significantly lower than the second, a facilitation effect occurred on the first, whereas, as the reward value of the first increased and became closer to the level of the second, a negative contrast occurred. We, however, appeared to obtain the opposite effect with monkey P, such that when the first item was significantly lower than the second, the negative effect of the second item on the first increased. Further research will need to characterize the effects of the relative value of the two items on choice behaviour.
EXPERIMENT 5
In Experiment 1, all three monkeys exhibited a peak-first preference; however, it was not yet clear whether that preference was due to order per se, or whether it was driven by an impulsive tendency to receive the best item as soon as possible. Put differently, delay discounting theory would predict a preference for A → B over B → A, because the higher value A would be significantly reduced when delayed (Ainslie, 1975; Green, Fry, & Myerson, 1994; Logue, 1995; Rachlin & Green, 1972). We therefore conducted our final experiment to test for the influence of order itself, independent of potential effects due to delay discounting. For this test, we again offered the choice between A → B and B → A, but with a delay added before receiving the A → B outcome, so that the delay to receiving A was identical in both alternatives. Delay discounting theory should now predict a preference for the B → A alternative, since B is received immediately, and A is received at the same time as the identical first item A in the alternative. If, in contrast, the monkeys have a true order preference that is separable from the effects of delay discounting, then even with the added delay, A → B would still be preferred.
Method
Subjects, materials, and procedure
These were as listed under “General method”.
Specific procedure
Selection of one coloured lid resulted in the monkey receiving a half vegetable, followed 4 s later by a grape, which was then followed by a 4-s delay, denoted B → A → [ ]. The final delay was necessary to ensure a comparable trial length for both contingencies. The second coloured lid gave an initial 5-s delay to match the exact overall delay to receive the grape in the first choice alternative, followed by a grape, followed 4 s later by a half vegetable, denoted [ ] → A → B. Table 5 shows lid colours and contingencies for each monkey.
Table 5.
Canister colours/contingency | ||
---|---|---|
Monkey | B → A → [ ] | [ ] → A → B |
P | Purple | Yellow |
H | Green | Blue |
T | Yellow | Striped (green and white) |
Note: Selection of one canister resulted in a half vegetable, followed 4 s later by a grape, followed by a 4-s delay (B → A → [ ]), and selection of the other resulted in an initial 5-s delay, followed by a grape, followed 4 s later by a half vegetable ([ ] → A → B).
Results
As shown in Figure 5, two of the three monkeys (P and H) revealed a significant preference for [ ] → A → B over B → A → [ ]. All 3 sessions for monkey P were significant, selecting [ ] → A → B on 86.7%, 90%, and 73.3% of the trials (two-tailed binomial tests: p < .0001, p < 1.0 × 10−5, p < .05). Overall, monkey P selected [ ] → A → B on 83.3 ± 8.8% of trials across the 3 sessions (χ2: p < .001). Monkey H took 4 sessions to have 3 significant ones. In 3 consecutive sessions, monkey H selected [ ] → A → B on 73.3%, 76.7%, and 76.7% of the trials (two-tailed binomial tests: p < .05, p < .01, p < .01). Overall, monkey H selected [ ] → A → B on 75.6 ± 1.9% of trials across the last 3 sessions (χ2: p < .01). Because monkey T did not have a significant session in 10 sessions, we concluded that he did not have a preference Monkey T selected [ ] → A → B on average 48.9 ± 9.6% of trials over his last 3 nonsignificant sessions (χ2: p > .2).
Discussion
If the monkeys’ choices had been based on how quickly they could obtain the rewards, they would have preferred B → A → [ ], because the overall outcome was delivered sooner than [ ] → A → B. This was not the case for any of the monkeys. Two of them (P and H) showed a clear preference for [ ] → A → B. Although it is possible that the delay in the last position could have had a stronger, negative influence on choice than that in the first position, causing the two monkeys to avoid B → A → [ ], it seems unlikely since Experiment 1 showed that the first position was more important than the last. Thus, monkeys P and H again exhibited a clear peak-first preference even when the items were received later in time. These results cannot be fully explained by delay discounting or risk aversion, and they provide evidence that order itself influenced the choices of these two monkeys.
Monkey T’s indifference to the two alternatives suggests that the value of both Bs were equivalent for him, being either zero (or at least equal to the delay values) or some positive value. Because monkey T’s results in Experiment 3 showed that the value of B in B → A was positive, they suggest that both Bs may be equally positive. Thus, for monkey T, it appeared that when A was immediate, it controlled his behaviour, such that B was in fact ignored (Experiment 2). However, when A was delayed (both Experiments 3 and 5), monkey T was then influenced by B, when in either first or last position. His indifference between B → A → [ ] and [ ] → A → B, however, makes it unclear whether he was influenced by order, other than being exceptionally impulsive with the higher valued food item.
GENERAL DISCUSSION
People appear to show a general peak-end preference, and we tested rhesus monkeys to see whether they showed an order preference. Based on our limited sample of 3 individuals, we found no evidence for a consistent peak-end preference. Most often, rhesus monkeys chose the option that provided the greatest good first: a peak-first preference. In a series of tests that explored this finding, we could rule out a simple account of our result in terms of delay discounting (Experiment 5), a consistent neglect of the lesser good (Experiment 2 and 3), or a specific relationship between the two goods tested in the main experiment (Experiment 4). Such factors seem to contribute to individual variation. One monkey (T), for example, neglected the lesser good when it was presented after a greater good and was indifferent to order when the delay of the greater good was the same. Another monkey (P) preferred the greater good first for familiar goods, but had the opposite preference for the more novel goods. These factors produce individual variation in the experiments, but they do not account for the main result: the lack of a consistent peak-end preference. These findings suggest that humans and rhesus monkeys value sequences of goods differently and point potentially to the importance of subsequent adaptations in food order preferences after apes diverged from other catarrhine primates roughly 25 million years ago.
Although a species difference may exist between humans and rhesus monkeys for valuation of sequential rewards, more studies are necessary both with humans and with nonhuman animals to clarify potential differences. As opposed to the Do et al. (2008) experiments, our study was conducted with the same subjects over repeated trials. It is possible that the human findings would change if they were given more experience with the outcomes and tested in a repeated-choice paradigm (Hayden & Platt, 2009). As discussed above, there is evidence that forecasted preferences do not always match up with the subjective experience of the outcomes (Wilson & Gilbert, 2005), and preferences may change depending on particular test conditions (see, e.g., Mantonakis, Rodero, Lesschaeve, & Hastie, 2009; Rode, Rozin, & Durlach, 2007), such as whether the alternatives are considered in isolation or in direct comparison (Hsee, 1998; Hsee et al., 1999; Hsee & Zhang, 2004; List, 2002). Nonetheless, the peak-end phenomenon has been obtained with humans using a choice paradigm, thus resembling the present experiment (Kahneman et al., 1993; Redelmeier et al., 2003). In any case, it is clear that future experiments will need to test a diversity of species under more similar conditions to determine exactly how and when peak-end effects arise.
A peak-first preference appears to be similar to other phenomena found with nonhuman animals, such as relative impulsiveness and contrast effects in which the value of the food items influence each other. Indeed, monkey T in our study appeared to be exceptionally impulsive, such that when the higher valued food item (A) was received immediately, the other item appeared to be ignored, mimicking the selective-value effect obtained with other primates (Beran et al., 2009; Silberberg et al., 1998). Interestingly, when A was delayed in Experiments 3 and 5, B then influenced monkey T’s behaviour because he preferred B → A over [ ] → A and was indifferent to B → A → [ ] and [ ] → A → B, respectively. This strong preference for receiving A immediately relates to findings that animals tend to prefer immediate over future rewards by devaluing or discounting the future goods (Ainslie, 1975; Hayden & Platt, 2007; Heilbronner, Rosati, Stevens, Hare, & Hauser, 2008; Logue, 1995; Rachlin & Green, 1972; Rosati, Stevens, Hare, & Hauser, 2007; Stevens, Hallinan, & Hauser, 2005).
Although monkey T’s results can be mostly explained by general impulsiveness for the higher valued food item, the results of all three monkeys in Experiment 5 cannot be fully explained by delay discounting because none of the monkeys showed a preference for the alternative that gave both reward items earliest in time (B → A → [ ]). Thus, apart from delay discounting, the order of the outcomes influenced choice.
The results do not appear to be the result of memory limitations that might cause a primacy or recency effect, whereby the first (primacy) or last (recency) items in a serial list might be more easily remembered than others (Castro & Larsen, 1992; Wright, Santiago, & Sands, 1984; Wright, Santiago, Sands, Kendrick, & Cook, 1985). Wright and colleagues found that when the delay period was between 1–10 s, rhesus monkeys showed comparable primacy and recency effects (Wright et al., 1984, 1985) on memory. Since our intertrial intervals were approximately 8 s and never more than 10 s, neither effect should have outweighed the other. Thus, our results do not appear to be an artefact of memory limitations.
In general, contrast effects could underlie order preferences (see Flaherty, 1996; Williams, 1997, 2002; Zentall & Singer, 2007). For example, a negative anticipatory effect of A on B in B → A may lower the value of B, resulting in the lowering of the value of the entire alternative; likewise, a positive anticipatory effect (also called facilitation or positive induction) of B on A in A → B may increase the value of A and perhaps the entire alternative, resulting in a preference for [ ] → A → B over B → A → [ ]. However, because anticipatory contrast effects are typically obtained with different testing paradigms, dependent measures, and species, more work will be necessary to determine the degree to which contrast effects underlie order preferences in rhesus monkeys.
The order effects found here may be due to a similar phenomenon found in people in which we often appear to neglect duration in our retrospective evaluations (Fredrickson & Kahneman, 1993; Kahneman et al., 1993). Our monkeys may have also tended to neglect time as a factor, and instead focused more attention on the quality of the items and preferring the sequence with the best item first. People, on the other hand, may tend to neglect time for a different reason: when the experience results in a better end (peak-end).
Evolutionarily, it might be advantageous to follow a strategy such as “a fruit in the hand is better than two in the bush” when faced with the uncertainties of a dynamically changing and competitive environment (Heilbronner et al., 2008; Rosati et al., 2007; Stevens et al., 2005). This logic would predict that if a human peak-end preference is at least partly an evolved adaptation, it would have developed as ecological resources for humans became more certain and predictable. However, more research is needed to clarify the differences between humans and other animals in decision making given that other studies have found similarities in choice behaviour under uncertainty (Hayden & Platt, 2007, 2009; Hertwig, Barron, Weber, & Erev, 2004; Weber, Shafir, & Blais, 2004).
Overall, we have shown that the sequential ordering of reward items influences choice behaviour. Future work should continue to determine the conditions under which factors such as reward order, risk, and delay discounting differentially influence decision making.
Acknowledgments
Our funding was provided by the National Institute of Mental Health’s Research Career Award for Transition to Independence (K22) grant (1 K22 MH071756–01), under the title “Neurophysiology of Inhibitory Control in Decision Making” with principal investigator Jerald D. Kralik. The funding source was not involved with any aspect of this study or its submission. We thank Sara Kahn, William Levine, and William Sampson for helping with data collection. We would also like to thank Steven Wise for helpful discussions and comments on an earlier version of the manuscript.
Footnotes
Publisher's Disclaimer: Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions
This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden.
The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.
REFERENCES
- Ainslie G. Specious reward: A behavioral theory of impulsiveness and impulse control. Psychological Bulletin. 1975;82(4):463–496. doi: 10.1037/h0076860. [DOI] [PubMed] [Google Scholar]
- Baumgartner H, Sujan M, Padgett D. Patterns of affective reactions to advertisements: The integration of moment-to-moment responses into overall judgments. Journal of Marketing Research. 1997;34(2):219–232. [Google Scholar]
- Beran MJ. Maintenance of self-imposed delay of gratification by four chimpanzees (Pan troglodytes) and an orangutan (Pongo pygmaeus) Journal of General Psychology. 2002;129(1):49–66. doi: 10.1080/00221300209602032. [DOI] [PubMed] [Google Scholar]
- Beran MJ, Ratliff CL, Evans TA. Natural choice in chimpanzees (Pan troglodytes): Perceptual and temporal effects on selective value. Learning and Motivation. 2009;40(2):186–196. doi: 10.1016/j.lmot.2008.11.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Beran MJ, Savage-Rumbaugh ES, Pate JL, Rumbaugh DM. Delay of gratification in chimpanzees (Pan troglodytes) Developmental Psychobiology. 1999;34(2):119–127. doi: 10.1002/(sici)1098-2302(199903)34:2<119::aid-dev5>3.0.co;2-p. [DOI] [PubMed] [Google Scholar]
- Castro CA, Larsen T. Primacy and recency effects in nonhuman-primates. Journal of Experimental Psychology: Animal Behavior Processes. 1992;18(4):335–340. doi: 10.1037//0097-7403.18.4.335. [DOI] [PubMed] [Google Scholar]
- Do AM, Rupert AV, Wolford G. Evaluations of pleasurable experiences: The peak-and-end rule. Psychonomic Bulletin & Review. 2008;15(1):96–98. doi: 10.3758/pbr.15.1.96. [DOI] [PubMed] [Google Scholar]
- Evans TA, Beran MJ. Delay of gratification and delay maintenance by rhesus macaques (Macaca mulatta) Journal of General Psychology. 2007;134(2):199–216. doi: 10.3200/GENP.134.2.199-216. [DOI] [PubMed] [Google Scholar]
- Flaherty CF. Incentive relativity. Cambridge UK: Cambridge University Press; 1996. [Google Scholar]
- Flaherty CF, Turovsky J, Krauss KL. Relative hedonic value modulates anticipatory contrast. Physiology & Behavior. 1994;55(6):1047–1054. doi: 10.1016/0031-9384(94)90386-7. [DOI] [PubMed] [Google Scholar]
- Fredrickson BL, Kahneman D. Duration neglect in retrospective evaluations of affective episodes. Journal of Personality and Social Psychology. 1993;65(1):45–55. doi: 10.1037//0022-3514.65.1.45. [DOI] [PubMed] [Google Scholar]
- Green L, Fry AF, Myerson J. Discounting of delayed rewards: A life-span comparison. Psychological Science. 1994;5(1):33–36. [Google Scholar]
- Hayden BY, Heilbronner SR, Nair AC, Platt ML. Cognitive influences on risk-seeking by rhesus macaques. Judgment and Decision Making Journal. 2008;3(5):389–395. [PMC free article] [PubMed] [Google Scholar]
- Hayden BY, Platt ML. Temporal discounting predicts risk sensitivity in rhesus macaques. Current Biology. 2007;17(1):49–53. doi: 10.1016/j.cub.2006.10.055. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hayden BY, Platt ML. Gambling for Gatorade: Risk-sensitive decision making for fluid rewards in humans. Animal Cognition. 2009;12(1):201–207. doi: 10.1007/s10071-008-0186-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Heilbronner SR, Rosati AG, Stevens JR, Hare B, Hauser MD. A fruit in the hand or two in the bush? Divergent risk preferences in chimpanzees and bonobos. Biology Letters. 2008;4(3):246–249. doi: 10.1098/rsbl.2008.0081. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hertwig R, Barron G, Weber EU, Erev I. Decisions from experience and the effect of rare events in risky choice. Psychological Science. 2004;15(8):534–539. doi: 10.1111/j.0956-7976.2004.00715.x. [DOI] [PubMed] [Google Scholar]
- Hsee CK. Less is better: When low-value options are valued more highly than high-value options. Journal of Behavioral Decision Making. 1998;11(2):107–121. [Google Scholar]
- Hsee CK, Loewenstein GF, Blount S, Bazerman MH. Preference reversals between joint and separate evaluations of options: A review and theoretical analysis. Psychological Bulletin. 1999;125(5):576–590. [Google Scholar]
- Hsee CK, Zhang J. Distinction bias: Misprediction and mischoice due to joint evaluation. Journal of Personality and Social Psychology. 2004;86(5):680–695. doi: 10.1037/0022-3514.86.5.680. [DOI] [PubMed] [Google Scholar]
- Kahneman D, Fredrickson BL, Schreiber CA, Redelmeier DA. When more pain is preferred to less: Adding a better end. Psychological Science. 1993;4(6):401–405. [Google Scholar]
- Kahneman D, Tversky A. Prospect theory: Analysis of decision under risk. Econometrica. 1979;47(2):263–291. [Google Scholar]
- Knight EJ, Klepac KM, Kralik JD. Expectation and surprise in primate decision making. 2008 Manuscript in preparation. [Google Scholar]
- List JA. Preference reversals of a different kind: The “more is less” phenomenon. American Economic Review. 2002;92(5):1636–1643. [Google Scholar]
- Logue AW. Self-control: Waiting until tomorrow for what you want today. Englewood Cliffs, NJ: Prentice Hall; 1995. [Google Scholar]
- Mantonakis A, Rodero P, Lesschaeve I, Hastie R. Order in choice: Effects of serial position on preferences. Psychological Science. 2009;20(11):1309–1312. doi: 10.1111/j.1467-9280.2009.02453.x. [DOI] [PubMed] [Google Scholar]
- Rachlin H, Green L. Commitment, choice and self-control. Journal of the Experimental Analysis of Behavior. 1972;17(1):15–22. doi: 10.1901/jeab.1972.17-15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Redelmeier DA, Kahneman D. Patients' memories of painful medical treatments: Real-time and retrospective evaluations of two minimally invasive procedures. Pain. 1996;66(1):3–8. doi: 10.1016/0304-3959(96)02994-6. [DOI] [PubMed] [Google Scholar]
- Redelmeier DA, Katz J, Kahneman D. Memories of colonoscopy: A randomized trial. Pain. 2003;104(1–2):187–194. doi: 10.1016/s0304-3959(03)00003-4. [DOI] [PubMed] [Google Scholar]
- Rode E, Rozin P, Durlach P. Experienced and remembered pleasure for meals: Duration neglect but minimal peak, end (recency) or primacy effects. Appetite. 2007;49(1):18–29. doi: 10.1016/j.appet.2006.09.006. [DOI] [PubMed] [Google Scholar]
- Rosati AG, Stevens JR, Hare B, Hauser MD. The evolutionary origins of human patience: Temporal preferences in chimpanzees, bonobos, and human adults. Current Biology. 2007;17(19):1663–1668. doi: 10.1016/j.cub.2007.08.033. [DOI] [PubMed] [Google Scholar]
- Schreiber CA, Kahneman D. Determinants of the remembered utility of aversive sounds. Journal of Experimental Psychology: General. 2000;129(1):27–42. doi: 10.1037//0096-3445.129.1.27. [DOI] [PubMed] [Google Scholar]
- Silberberg A, Widholm JJ, Bresler D, Fujita K, Anderson JR. Natural choice in nonhuman primates. Journal of Experimental Psychology: Animal Behavior Processes. 1998;24(2):215–228. doi: 10.1037//0097-7403.24.2.215. [DOI] [PubMed] [Google Scholar]
- Stevens JR, Hallinan EV, Hauser MD. The ecology and evolution of patience in two New World monkeys. Biology Letters. 2005;1(2):223–226. doi: 10.1098/rsbl.2004.0285. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Weber EU, Shafir S, Blais AR. Predicting risk sensitivity in humans and lower animals: Risk as variance or coefficient of variation. Psychological Review. 2004;111(2):430–445. doi: 10.1037/0033-295X.111.2.430. [DOI] [PubMed] [Google Scholar]
- Williams BA. Varieties of contrast: A review of incentive relativity by Charles F Flaherty. Journal of the Experimental Analysis of Behavior. 1997;68(1):133–141. doi: 10.1901/jeab.1997.68-133. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Williams BA. Behavioral contrast redux. Animal Learning & Behavior. 2002;30(1):1–20. doi: 10.3758/bf03192905. [DOI] [PubMed] [Google Scholar]
- Wilson TD, Gilbert DT. Affective forecasting: Knowing what to want. Current Directions in Psychological Science. 2005;14(3):131–134. [Google Scholar]
- Wright AA, Santiago HC, Sands SF. Monkey memory: Same different concept-learning, serial probe acquisition, and probe delay effects. Journal of Experimental Psychology: Animal Behavior Processes. 1984;10(4):513–529. [PubMed] [Google Scholar]
- Wright AA, Santiago HC, Sands SF, Kendrick DF, Cook RG. Memory processing of serial lists by pigeons, monkeys, and people. Science. 1985;229(4710):287–289. doi: 10.1126/science.9304205. [DOI] [PubMed] [Google Scholar]
- Zentall TR, Singer RA. Within-trial contrast: Pigeons prefer conditioned reinforcers that follow a relatively more rather than a less aversive event. Journal of the Experimental Analysis of Behavior. 2007;88(1):131–149. doi: 10.1901/jeab.2007.27-06. [DOI] [PMC free article] [PubMed] [Google Scholar]