Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Dec 16.
Published in final edited form as: J Exp Anal Behav. 2013 Jan 14;99(2):150–158. doi: 10.1002/jeab.11

Effects of Reward Bundling on Male Rats' Preference for Larger-later Food Rewards

Jeffrey S Stein 1, Rochelle R Smits 2, Patrick S Johnson 3, Kennan J Liston 4, Gregory J Madden 5
PMCID: PMC4267752  NIHMSID: NIHMS647687  PMID: 23319442

Impulsive choice is defined as selecting a smaller-sooner (SS) over a larger-later (LL) reward, and self-control choice as the opposite (e.g., Ainslie, 1974). An impulsive choice reveals that the subjective value of the LL reward is less than that of the SS; a devaluation process referred to as delayed-reward discounting (for overview see Madden & Johnson, 2010). When the discounted value of a reward is assessed across a wide range of delays, the nonlinear function that describes the discounted values of the LL rewards most often takes a hyperbolic shape as shown in the top panel of Figure 1, and as quantified by Mazur (1987):

Figure 1.

Figure 1

Open in a new tab

The top panel illustrates how the value of a larger-later (LL) reward declines as the delay to its delivery increases. At time T1 the discounted value of the LL reward falls below the smaller-sooner (SS) reward and so the impulsive choice for the SS reward is made. The middle panel illustrates how three SS or three LL rewards may be bundled together. A single choice at T1 results in delivery of either the SS or LL bundle. The bottom panel shows hyperbolic delay discounting curves obtained using the same parameters used in the top panel. At T1 the sum of the discounted values of the rewards in the SS bundle is given by the height of the SS reward bar to the left of T1 (the filled portion shows that portion contributed by the sum of the delayed SS rewards). The discounting curve appearing above T1 is the sum of the LL discounting functions (dashed curves) at the times shown. At T1, when the first reward in the SS bundle may be obtained now, the discounted value of the LL bundle exceeds that of the SS bundle. Thus, by bundling rewards, choice should theoretically shift from impulsivity (top panel) to self-control (bottom panel).

V=A1+kD (1)

Where V is the discounted value of the LL reward of amount A delivered following delay D. The free parameter k quantifies the steepness of the discounting curve. Given the steepness of the discounting curve shown in the top panel of Figure 1 the value of the LL reward dips below that of the SS reward which is chosen at T1.

The relation between steep delayed-reward discounting and problem drug use is well established (see MacKillop et al., 2011 for meta-analysis) and converging evidence suggests that steep discounting precedes and predicts acquisition of drug taking in rats (for review see Stein & Madden, in press) and substance-abuse treatment outcomes in humans (e.g., Stanger et al., 2011). As a result, there has been increased interest in procedures that may be used to decrease impulsive choice (e.g., Bickel, Yi, Landes, Hill, & Baxter, 2011; Radu, Yi, Bickel, Gross, & McClure, 2011). One such procedure was outlined by Ainslie (1975) who suggested that by bundling a sequence of LL rewards together, their summed discounted value would exceed the value of a bundle of SS rewards; as a result, impulsive choice would decline.

This is illustrated in the middle panel of Figure 1 which illustrates two reward bundles that a decision maker, situated in time at T1, might choose between. The first bundle is composed of three SS rewards; the first delivered now (at T1) and the others in the future (following F1 and F2). When the individual chooses this reward bundle, he/she obtains the immediate benefits of the SS reward and is committed to receiving the remaining two smaller rewards in the future (and not receiving any of the LL rewards). The second reward bundle is composed of three LL rewards, one delivered following the delay (D1) and the others in the future (following F1 + D2 and F2 + D3).

When the three separate rewards in the SS and LL bundles shown in Figure 1 are combined into the SS and LL bundles, the value of each bundle is determined by the sum of the discounted values of all rewards in the bundle:

V=∑i=1n(A1+kDi) (2)

where the remaining parameters are as in Equation 1 (for findings consistent with this model see Andrade & Hackenberg, in press; Brunner, 1999; Brunner & Gibbon, 1995; Kirby, 2006; Mazur, 1984, 1986, 1989; Mitchell & Rosenthal, 2003; Shull, Mellon, & Sharp, 1990). The summed values of the bundles are illustrated in the bottom panel of Figure 1. The value of the SS bundle is the sum of the immediate reward value (the open portion of the smaller-sooner reward proximal to T1) and the discounted values of the other two rewards in the bundle. The sum of these two discounted values was added to the height of the SS bar nearest T1 (the filled portion of this bar).

The elevated dashed discounting curve above the value of the SS bundle at T1 shows the value of the LL bundle (i.e., the sum of the discounted value sat T1 of the three LL rewards). According to Equation 2, the value of the LL bundle exceeds that of the SS bundle and the impulsive choice is predicted to be avoided, an outcome confirmed in studies of human decision-making (Kirby, 2006; Kirby & Guastello, 2001). As a result of this prediction, reward bundling has been viewed as a means by which to shift impulsive choice towards self-control (e.g., Ainslie, 1974, 1975, 2001; Ainslie & Haslam, 1992; Hofmeyer, Ainslie, Charlton, & Ross, 2011; Ross, Sharp, Vuchinich, & Spurrett, 2008). Much of the discussion surrounding this shift involves verbally mediated bundling (e.g., viewing one's present choice to smoke a cigarette, or not, as the first in a bundled series in which the present choice is highly predictive of future choices; Monterosso & Ainslie, 1999).

One rationale for the present study was to conduct the first parametric manipulation of bundle size on impulsive choice in rats. Equation 2 predicts that each additional reward in the bundle will have an incremental contribution to the value of the bundle, albeit a diminishing contribution with incrementally longer delays. Thus, we conducted a between-subjects comparison of the effects of an extended history of making choices when SS and LL rewards were delivered in bundles of 1 (unbundled reward), 3, or 9 rewards. The second rationale for conducting the present study was to determine if exposure to reward bundling would shift rats' choice toward greater self-control when rewards were no longer bundled. Ainslie and Monterosso (2003) reported that when rats were exposed to 36 sessions in which three rewards were bundled as in Figure 1, subsequent unbundled choices trended toward increased self-control (p = .06); however, these authors' use of a within-subject design confounded the post-bundling shift with maturational factors. Thus, given the previously cited interest in procedures designed to experimentally manipulate impulsive choice, we wondered if exposure to reward bundling would increase self-control for unbundled rewards in an experiment designed explicitly to test such a possibility.

Method

Subjects

Twenty-four male, experimentally naïve Long-Evans rats (Harlan, Indianapolis, IN) served as subjects. Rats were approximately 75 days old at the start of the experiment and were housed individually in a temperature- and humidity-controlled colony room providing a 12-h light cycle (lights on at 7am). Rats were weighed daily and were maintained at approximately 85% free-feeding weight via supplemental feedings approximately two hours after experimental sessions. Water was available ad libitum in their home cages. Animal use was in accordance with the Institutional Animal Care and Use Committee of Utah State University. Two rats (one each in the Bundle-size 1 and 3 groups) developed receptacle biases during the reward-bundling phase (5-s delay condition). Additional training did not ameliorate the bias and so these rats were dropped from the study.

Apparatus

Twelve identical operant chambers (Med Associates, St. Albans, VT) were used. Each was equipped with two pellet receptacles on the front wall (6 cm apart, 1 cm from the floor, and 2 cm from each side wall). Each receptacle was equipped with a photocell to detect head entries and a stimulus light capable of either bright (20 lux) or dim (5 lux) illumination. Two separate feeders (Coulbourn, Allentown, PA) delivered 20-mg food pellets (BioServ, Frenchtown, NJ) into the two receptacles. A non-retractable lever (11 cm above the floor) and a white, 28-V DC stimulus light were centered on the rear wall. Each chamber was enclosed in a sound and light attenuating cubicle (Med Associates, St. Albans, VT) equipped with a 2.5-kHz Sonalert® tone generator, a fan, and a white-noise speaker. Experimental events were programmed via MED-PC IV software and controlled by a PC in an adjacent room.

Procedures

Initial training

Throughout the experiment one session was completed per day, 7 days a week. Auto shaping procedures were used to train head entry into the two food receptacles. Trials strictly alternated between receptacles and training continued until 90 of 100 reinforcers were obtained by an operant response. Lever pressing was also auto shaped but the reinforcer was the illumination of a food receptacle which, when entered, presented a food pellet.

Next, 80-trialchoice-training sessions were arranged, each divided into four blocks of 20 trials. Trials began with the illumination of the stimulus light over the lever. A single lever press extinguished the light and illuminated both receptacles; one was lit bright and the other dim. Entering either receptacle extinguished both receptacle lights and resulted in either one or three food pellets being delivered immediately to the chosen receptacle. For half the rats (randomly assigned), the one- and three-pellet rewards were assigned, respectively, to the left and right receptacles. For the remaining rats, this reward assignment was reversed. Within each receptacle-assignment grouping, the bright receptacle light signaled the location of the three-pellet reward for half of the rats, whereas the dim light signaled this reward for the other half of the rats.

The first six trials in each 20-trial block were forced-exposure trials in which only one receptacle was lit following a lever press (three on each receptacle, order selected randomly without replacement). Entering the unlit receptacle had no programmed consequence. A 15-s limited-hold was in effect during each trial, such that if a lever and a head-entry response were not made within 15 s of trial onset, the trial terminated and was scored as an omission. If a forced-exposure trial was omitted, it was repeated until completed. A variable intertrial interval (ITI), in which no lights or tones were presented, ensured that trials were initiated every 70 s regardless of the rat's behavior. A 180-s blackout separated trial blocks.

Pre-bundling test of impulsive choice

Next, impulsive choice was assessed using a within-session, increasing-delay procedure (Evenden & Ryan, 1996). These sessions were identical to the choice-training sessions except that the delay to the three-pellet reward increased across trial blocks from 0, 5, 15, and then to 45 s. Rats completed 42 of these sessions with five no-delay sessions (i.e., the delay to the three-pellet reward remained at 0-s throughout) pseudorandomly interspersed among the first 35 pre-test sessions (1 per 7 pre-test sessions). The order was identical for all rats. No-delay sessions were omitted from the final seven sessions of the pre-test. Terminal data are taken from the final six sessions.

Reward-bundling phase

Rats were matched into triads based on the area under their percent choice of the LL reward data (AUC) in the terminal impulsive-choice pre-test sessions. Rats from each triad were then randomly assigned to one of three reward-bundling groups: Bundle-size 1, 3, or 9. There were no significant differences in pre-test AUC between bundle-size groups (F(2,19) = 0.03, p = .97). The Bundle-size 1 group (panel A in Figure 2) served as a control, as each choice resulted in a single outcome – either one pellet now or three pellets after a delay. A 35-s ITI ensured that trials began at regular intervals regardless of the reward selected. A single choice in the Bundle-size 3 and 9 groups (Figure 2B & 2C) resulted, respectively, in three and nine discrete deliveries of the chosen reward. If the SS reward was selected, one pellet was delivered immediately and either two (Bundle-size 3 group) or eight (Bundle-size 9 group) additional pellets were delivered at regular inter-pellet intervals; this interval was equal to the delay to the LL reward. For the latter groups, an ITI ensured that trials began at regular intervals regardless of the choice made. The ITI was three and nine times the ITI of the control group for the Bundle-size 3 (ITI = 105 s) and Bundle-size 9 (ITI = 315 s) groups, respectively.

Figure 2.

Figure 2

Open in a new tab

Trial structure used during the reward-bundling phase for Bundle-size 1 (panel A), 3 (panel B), and 9 (panel C) groups. Solid arrows represent post-choice reward-delays or inter-pellet intervals, whereas dashed arrows represent time spent in the ITI.

The number of choices made by rats assigned to each group was held constant at 18 trials per session. The first four trials were forced-exposure trials. These trials began with the illumination of the rear-lever light. Pressing it once extinguished the lever light and illuminated one receptacle. Two forced-exposure trials were completed in each receptacle with order selected randomly without replacement. Entering the receptacle extinguished the light in the receptacle and initiated the reward-bundling sequence depicted in Figure 2. Assignment of bright or dim receptacle lights to the one- and three-pellet rewards remained unchanged from the impulsive-choice pre-test. The remaining 14 trials were choice trials in which both receptacles were illuminated following a lever press. A 15-s omission criterion was in effect on all trials. Forced-exposure trial omissions resulted in presentation of the same trial after the ITI.

The delay to the LL reward was parametrically manipulated across conditions. Each delay condition lasted 40 sessions (20 each with the LL reward assigned to left and right receptacles). Delays decreased across conditions (17.5, 12.5, 10, 7.5, 5, and 0 s) followed by a replication at two delays in an ascending order (5 s and17.5 s).

Post-bundling test of impulsive choice

Following the reward bundling phase, impulsive choice was reassessed using the same procedures described above for the pre-bundling test of impulsive choice.

Data Analysis

Paired-samples t-tests were used to determine if choice was affected by the receptacle to which the SS and LL rewards were assigned, or was different between the initial assessment and reassessment at delays of 5 and 17.5s. Independent-samples t tests were used to determine if choice differed depending on stimulus assignment (bright or dim) to the choice alternatives. During the reward-bundling phase, the main dependent measure was area under the curve (AUC, Myerson, Green & Warusawitharana, 2001). This measure, expressed as a proportion of the maximum AUC, can vary from 0 (exclusive preference for the smaller reward) to 1.0 (exclusive preference for the larger reward). A secondary dependent measure was percent LL choice at each delay; the relation between LL choice and delay was analyzed using ordinary least squares regression.

To evaluate the effects of reward-bundling on impulsive choice without bundled rewards, the main dependent measure was a difference score calculated by subtracting pretest AUC (before reward bundling) from post-test AUC (after reward bundling) for each rat. Two separate one-way ANOVAs were used to compare reward-bundling AUCs and impulsive-choice difference scores between groups, with post-hoc comparisons and linear contrasts used in each ANOVA. Difference scores were also subjected to one-sample t tests to determine if they differed significantly from 0 (i.e. no change in impulsive choice). Finally, Pearson r correlations were used to examine the relation between all bivariate combinations of pre-test, reward-bundling, and post-test AUCs, as well as pre-test vs. post-test difference scores. All analyses were conducted in SPSS (ver. 19.0, SPSS Inc., Chicago, IL). Unless otherwise noted, pair wise comparisons in post-hoc tests in ANOVA, as well as multiple pair wise comparisons between tests of a similar type (e.g., the one-sample t-tests or correlations just described) were implemented using Bonferroni correction.

Results

Because there was no significant effect of reward assignment to receptacle location (uncorrected p >.19, in all cases) or assignment of bright or dim stimuli to the choice alternatives in any condition (uncorrected p > .45, in all cases) data were analyzed without respect to these assignments. In addition, the reassessment of percent LL choice at both the 5-s (uncorrected p > .17, in all cases) and 17.5-s (uncorrected p > .12, in all cases) delays did not differ significantly from choice when initially assessed in any group; therefore, data were averaged across assessments.

The upper panel of Figure 3 shows a significant linear decline in the average percent LL choice as delays increased from 0 to 17.5 s (F(1,19) = 116.13, p < .0001). The middle panel shows the distribution of AUC values obtained from individual rats' percent LL choice from the terminal sessions during the reward-bundling phase. A significant main effect of group (F(2,19) = 4.98, p < .01) was detected, as was a significant linear increase in AUC values with bundle size (F(1,19) = 9.40, p < .01). Post-hoc comparisons revealed that only the Bundle-size 1 and 9 groups were significantly different (p < .05). AUC values obtained in the impulsive-choice pre-test were not correlated with AUC values obtained in the reward-bundling phase in any group (p > .66 in all cases), nor when data were collapsed across groups (p = .53).

Figure 3.

Figure 3

Open in a new tab

Mean percent larger-later (LL) choice (top panel) and AUC values (middle panel) in the reward-bundling phase for Bundle-size 1, 3, and 9 groups. The bottom panel depicts AUC difference scores from pre- to post-test of impulsive-choice (i.e., when rewards were not bundled).

Individual rats' pre-test percent LL choice was fit with a reverse sigmoid function1 to interpolate the delay at which choice of the LL reward was predicted to be 50% (i.e., indifference). The fits provided were excellent (median R2= 99.9).We then solved for the value of k in Equation 1 at which the value of three pellets delivered after this indifference delay was equal to one pellet delivered now (i.e., the value of k at which the LL and SS rewards were of equal value). These estimated k-values, shown in Table 1, were normally distributed and did not differ across groups (p = .64). These values were input separately into Equation 2 to predict the delay at which each rat should be indifferent between the SS and LL bundles (i.e., the indifference delay). In the Bundle-size 3 and 9 groups, Equation 2 always predicted larger indifference delays than those obtained in the reward-bundling phase (the latter obtained using the same reverse sigmoid function as above; median R2= 93.5). The mean difference between predicted and obtained indifference delays was -13.7 (SD = 8.0) seconds in the Bundle-size 3 group and -17.7 (SD = 10.4) seconds in the Bundle-size 9 group. Thus, when rewards were bundled, rats made more impulsive choices than predicted by Equation 2.

Table 1.

Rat k Predicted Indifference Delay (sec) Obtained Indifference Delay (sec)
BS1-1 0.13 15.4 2.7
BS1-2 0.13 14.8 3.1
BS1-3 0.15 12.1 4.9
BS1-4 0.15 12.9 2.5
BS1-5 0.22 9.2 2.9
BS1-6 0.24 8.2 8.7
BS1-7 0.38 5.2 6
BS3-1 0.09 36.8 6.2
BS3-2 0.15 22.22 7.3
BS3-3 0.15 22.22 4.9
BS3-4 0.16 20.8 8.4
BS3-5 0.21 15.85 10.9
BS3-6 0.21 15.85 5.9
BS3-7 0.37 9 2.9
BS9-1 0.12 43.5 20.1
BS9-2 0.14 37.5 11.2
BS9-3 0.15 35 3.3
BS9-4 0.18 29 5.9
BS9-5 0.16 33 10.8
BS9-6 0.27 19.5 11.2
BS9-7 0.38 13.9 9.5
BS9-8 0.56 9.4 7.4
Open in a new tab

The bottom panel of Figure 3 showschanges in AUC values from the pre-test of impulsive choice that was completed before the reward-bundling phase and the post-test completed after this phase, when rewards were no longer bundled. A one-way ANOVA revealed a significant effect of group on AUC difference scores (F(1,19) = 3.54, p < .05), and also revealed a significant linear increase in difference scores with bundle size (F(1,19) = 6.49, p < .05). In post-hoc comparisons, only the difference between Bundle-size 1 and 9 groups was significant (p < .05). Likewise, difference scores of the Bundle-size 9 group differed significantly from zero (t(7) = 4.2, p < .01), whereas there was no difference in pre- to post-training AUC values in the Bundle-size 1 (t(6) = 0.60, p = 1.0) or the Bundle-size 3 (t(6) = 2.2, p = .21) groups.

Consistent with the analysis of difference scores above, the correlation between pre-test and post-test AUC values was significant (and positive) only for the Bundle-size 1 group (r = .86, p < .05). Thus, rats in the control (no-bundling) group made consistent choices throughout the experiment, whereas rats in the bundling groups did not. To determine if choices made during the reward-bundling phase were predictive of post-test choices made when rewards were no longer bundled, we examined the correlations between AUC values obtained in the reward-bundling phase (middle panel of Figure 3) and the pre-post AUC difference scores (bottom panel of Figure 3). These values were uncorrelated in any group (p = 1.0, in all groups). Likewise, AUC in the bundling phase was not predictive of post-test AUC values (p > .60, in all groups).

Discussion

The present experiment sought to answer two questions. First, if the number of SS and LL rewards that are bundled together is increased, does this increase preference for the LL reward bundle (as predicted by Equation 2)? A significant linear increase in AUC value with increased bundle sizes provides an affirmative answer, although post-hoc tests revealed that AUC differed only between the Bundle-size 9 and 1 (control) groups. With respect to specific predictions, when baseline k-values were input into Equation 2 to predict the effects of reward bundling on choice, the obtained increase in LL choice was much smaller than the predicted increase. For example, at the 17.5-s delay, Equation 2 predicted that Bundle-size 1 and 3 groups would prefer the SS alternative, whereas rats in the Bundle-size 9 group would prefer the LL alternative. Instead, all groups strongly favored the SS rewards at this delay. At shorter delays, where Equation 2 predicted that the Bundle-size 3 and 9 groups would prefer the LL reward it was clear that the predicted effects of reward bundling fell short of obtained shifts in self-control choice.

Although this smaller than predicted effect of reward bundling questions the quantitative predictions of Equation 2, the underestimation may reflect an incorrect assessment of steady-state k-values in the pre-bundling test of impulsive choice. In that test, we used a common procedure for assessing impulsive choice — that developed by Evenden and Ryan (1996) in which delays increase within-session across trial-blocks. Madden and Johnson (2010) suggested that this procedure may underestimate k-values because choices made in later trial blocks (long delays) may be influenced by carryover effects from earlier trial-blocks (brief delays). That is, rats may persist in selecting the LL reward at a long delay because, earlier in the session, the delay to this reward was shorter. In the present experiment we endeavored to decrease the probability of such carryover effects with a number of procedural variables. For example, we arranged more forced-exposure trials at the beginning of each trial block than is typical in the literature (6 vs. the typical 2-4). We also used a strict geometric progression of non-zero delays (5, 15, and 45 s vs. the typical 10, 20, 40, 60 s), and imposed a 180-s blackout period between trial blocks, in order to increase the discriminability of within-session changes in delay. These procedures would appear to have increased our rats' sensitivity to the delay changes between blocks because steady-state percent LL choice at the longest delays (24.1% LL choice at the 15-s delay and 2.4% LL choice at the 45-s delay), were well below percent LL choice reported in studies using Long-Evans rats without these procedures (e.g., interpolated percent LL choice at 15 and 45 s delays in the Harty, Whaley, Halperin & Ranaldi, 2011 study were > 40% and > 20%, respectively). Thus, although it is possible that our pre-bundling assessment of impulsive choice still underestimated the degree of delay discounting, the present findings suggest that Equation 2 overestimates the effects of reward bundling on percent LL choice.

The second question that we sought to answer was if an extended history of reward bundling would increase choice of the LL reward above baseline levels when rewards were no longer bundled together. The answer to this question was a qualified yes. Post-bundling preference for the LL reward increased above baseline levels for the Bundle-size 9 group. Although we did not observe this effect in any other group, we did observe a significant linear increase in AUC difference scores as a function of bundle size. Our findings are consistent with those of Ainslie and Monterosso (2003) who reported that when rats completed 36 sessions in a Bundle-size 3 phase (average of 18.6 trials in each), subsequent unbundled choices trended toward self-control (p = .06). Our rats completed 320 sessions (18 trials each) in which rewards were bundled and the finding of a lasting effect of this experience was replicated to the degree noted above. However, as noted in the Introduction, Ainslie and Monterosso used a within-subjects experimental design which confounded the post-bundling shift in choice with maturational factors. The between-groups manipulation of bundle size in the present study can rule out these maturational factors as accounting for the increased self-control.

Having said this, we must note that the lasting effect of reward bundling on subsequent choices involving unbundled-rewards is qualified by the observation that AUC in the bundling phase did not predict AUC in the post-bundling test of impulsive choice. Thus, sensitivity to the reward-bundling arrangement was not predictive of increased self-control when rewards were no longer bundled. Said another way, the self-control demonstrated in post-bundling tests of impulsivity appears to have had little to do with bundling per se. To explore other possibilities we examined the correlation between AUC values obtained at pre- and post-bundling tests; perhaps the increases in LL choice following the bundling phase were predicted by baseline delay discounting. This was not the case; the only significant pre-post AUC correlation was in the Bundle-size 1 (control) group, r = .86, p < .05; indicating that test-retest reliability of delay discounting was excellent after a 320-day period separating impulsive-choice assessments. However, pre-bundling tendency toward self-control (pre-test AUC) was not predictive of behavior following the bundling phase (post-test AUC) in any other group (all other p values > .26).

One possible explanation for the lack of correlation between choices made in the bundling phase and post-bundling impulsive choice is that rats in the Bundle-size 9 group were exposed to far more reward delays than were rats in the other groups. A recent study in our lab (Stein et al., submitted) demonstrated significant increases in LL reward preference in an experimental group of rats given a prolonged history of pressing a lever for food rewards delayed by 17.5 s. In the present study, rats in the Bundle-size 9 group were exposed to sequential reward delays on every trial in the reward-bundling phase, regardless of the alternative chosen (see Figure 2). The same was true for rats in the Bundle-size 3 group, although to a much lesser extent. On average, rats in the Bundle-size 1 group never preferred the LL reward and therefore, rarely experienced delays to reward. The lack of a positive correlation between choice in the bundling phase and choice in the post-test of impulsive choice suggests that the lasting effects of the bundling condition on LL choice might be better accounted for by delay exposure rather than to experience with reward bundling.

In sum, the present study demonstrates significant increases in self-control in rats when multiple rewards are bundled together. While reward-bundling AUC values only differed significantly between Bundle-size 9 and 1 (control) groups, a parametric influence of bundle size was apparent in the significant linear relation between AUC and bundle size. This finding extends the data reported by Ainslie and Monterosso (2003) by exploring the effects of a larger bundle size and by demonstrating a significant increase in LL choice when rewards were no longer bundled together. Question for future research are how long these changes in impulsive choice last and, given the positive relation between naturally-occurring impulsive choice and heightened vulnerability to drug reward in rats (e.g., Perry, Larsen, German, Madden, & Carroll, 2005; Yates, Marusich, Gipson, Beckmann, & Bardo, 2011), does this reduction in impulsive choice influence subsequent drug self-administration. The latter investigation may help to resolve the nature of the robust relation between steep delay discounting and human substance abuse.

Acknowledgments

This research was supported by a grant from the National Institute on Drug Abuse (DA029605) awarded to the last author.

Footnotes

1

The authors thank Paul L. Soto for supplying the equation and spreadsheet used to interpolate indifference points.

Contributor Information

Jeffrey S. Stein, Department of Psychology, Utah State University

Rochelle R. Smits, Department of Psychology, Utah State University

Patrick S. Johnson, Department of Psychology, Utah State University

Kennan J. Liston, Department of Psychology, Utah State University

Gregory J. Madden, Department of Psychology, Utah State University

References

  1. Ainslie GW. Impulse control in pigeons. Journal of the Experimental Analysis of Behavior. 1974;21:485–489. doi: 10.1901/jeab.1974.21-485. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Ainslie GW. Specious reward: A behavioral theory of impulsiveness and impulse control. Psychological Bulletin. 1975;82:463–496. doi: 10.1037/h0076860. [DOI] [PubMed] [Google Scholar]
  3. Ainslie GW. Break down of Will. New York, NY: Cambridge University Press; 2001. [Google Scholar]
  4. Ainslie GW, Haslam N. Hyperbolic discounting. In: Loewenstein G, Elster J, editors. Choice Over Time. New York, NY: Russell Sage Foundation; 1992. pp. 57–92. [Google Scholar]
  5. Ainslie G, Monterosso JR. Building blocks of self-control: Increased tolerance for delay with bundled rewards. Journal of the Experimental Analysis of Behavior. 2003;79:37–48. doi: 10.1901/jeab.2003.79-37. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Andrade LF, Hackenberg TD. Saving the best for last? A cross-species analysis of choices between reinforcer sequences. Journal of the Experimental Analysis of Behavior. doi: 10.1901/jeab.2012.98-45. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bickel WK, Yi R, Landes RD, Hill PF, Baxter C. Remember the future: Working memory training decreases delay discounting among stimulant addicts. Biological Psychiatry. 2011;69:260–265. doi: 10.1016/j.biopsych.2010.08.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Brunner D. Preference for sequences of rewards: Further tests of a parallel discounting model. Behavioural Processes. 1999;45:87–99. doi: 10.1016/s0376-6357(99)00011-x. [DOI] [PubMed] [Google Scholar]
  9. Brunner D, Gibbon J. Value of food aggregates: Parallel versus serial discounting. Animal Behavior. 1995;50:1627–1634. [Google Scholar]
  10. Evenden JL, Ryan CN. The pharmacology of impulsive behavior in rats: The effects of drugs on response choice with varying delays to reinforcement. Psychopharmacology. 1996;128:161–170. doi: 10.1007/s002130050121. [DOI] [PubMed] [Google Scholar]
  11. Harty SC, Whaley JE, Halperin JM, Ranaldi R. Impulsive choice, as measured in a delay discounting paradigm, remains stable after chronic heroin administration. Pharmacology, Biochemistry and Behavior. 2011;98:337–340. doi: 10.1016/j.pbb.2011.02.004. [DOI] [PubMed] [Google Scholar]
  12. Hofmeyr A, Ainslie G, Charlton R, Ross D. The relationship between addiction and reward bundling: An experiment comparing smokers and non-smokers. Addiction. 2011;106:402–409. doi: 10.1111/j.1360-0443.2010.03166.x. [DOI] [PubMed] [Google Scholar]
  13. Kirby KN. The present value of delayed rewards are approximately additive. Behavioural Processes. 2006;72:273–282. doi: 10.1016/j.beproc.2006.03.011. [DOI] [PubMed] [Google Scholar]
  14. Kirby KN, Guastello B. Making choices in anticipation of similar future choices can increase self-control. Journal of Experimental Psychology: Applied. 2001;7:154–164. doi: 10.1037//1076-898x.7.2.154. [DOI] [PubMed] [Google Scholar]
  15. MacKillop J, Amlung MT, Few LR, Ray LA, Sweet LH, Munafò MR. Delayed reward discounting and addictive behavior: A meta-analysis. Psychopharmacology (Berlin) 2011;216:305–321. doi: 10.1007/s00213-011-2229-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Madden GJ, Johnson PS. A delay-discounting primer. In: Madden GJ, Bickel WK, editors. Impulsivity: The behavioral and neurological science of discounting. Washington, DC: American Psychological Association; 2010. pp. 11–37. [Google Scholar]
  17. Mazur JE. Tests of an equivalence rule for fixed and variable reinforcer delays. Journal of Experimental Psychology: Animal Behavior Processes. 1984;10:426–436. [PubMed] [Google Scholar]
  18. Mazur JE. Choice between single and multiple delayed reinforcers. Journal of the Experimental Analysis of Behavior. 1986;46:67–77. doi: 10.1901/jeab.1986.46-67. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Mazur JE. An adjusting procedure for studying delayed reinforcement. In: Commons ML, Mazur JE, Nevin JA, Rachlin H, editors. Quantitative analyses of behavior: vol 5 The effect of delay and of intervening events on reinforcement value. Hillsdale, NJ: Erlbaum; 1987. pp. 55–73. [Google Scholar]
  20. Mazur JE. Theories of probabilistic reinforcement. Journal of the Experimental Analysis of Behavior. 1989;51:87–99. doi: 10.1901/jeab.1989.51-87. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Mitchell SH, Rosental AJ. Effects of multiple delayed rewards on delay discounting in an adjusting amount procedure. Behavioural Processes. 2003;64:273–286. doi: 10.1016/s0376-6357(03)00144-x. [DOI] [PubMed] [Google Scholar]
  22. Monterosso J, Ainslie GW. Beyond discounting: Possible experimental models of impulsive choice. Psychopharmacology. 1999;146:339–347. doi: 10.1007/pl00005480. [DOI] [PubMed] [Google Scholar]
  23. Myerson J, Green L, Warusawitharana M. Area under the curve as a measure of discounting. Journal of the Experimental Analysis of Behavior. 2001;76:235–243. doi: 10.1901/jeab.2001.76-235. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Perry JL, Larson EB, German JP, Madden GJ, Carroll ME. Impulsivity (delay discounting) as a predictor of acquisition of IV cocaine self-administration in female rats. Psychopharmacology. 2005;178:193–201. doi: 10.1007/s00213-004-1994-4. [DOI] [PubMed] [Google Scholar]
  25. Radu PT, Yi R, Bickel WK, Gross JJ, McClure SM. A mechanism for reducing delay discounting by altering temporal attention. Journal of the Experimental Analysis of Behavior. 2011;96:363–385. doi: 10.1901/jeab.2011.96-363. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Ross D, Sharp C, Vuchinich RE, Spurrett D. The Pico economics and Neuroeconomics of Disordered Gambling. Cambridge, MA: MIT Press; 2008. Midbrain Mutiny. [Google Scholar]
  27. Shull RL, Mellon RC, Sharp JA. Delay and number of food reinforcers: Effects on choice and latencies. Journal of the Experimental Analysis of Behavior. 1990;53:235–246. doi: 10.1901/jeab.1990.53-235. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Stanger C, Ryan SR, Fu H, Landes RD, Jones BA, Bickel WK, Budney AJ. Delay discounting predicts adolescent substance abuse treatment outcome. Experimental and Clinical Psychopharmacology. 2011 doi: 10.1037/a0026543. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Stein JS, Madden GJ. Delay discounting and drug abuse: empirical, conceptual, and methodological considerations. In: Mackillop J, de Wit H, editors. The Wiley-Blackwell Handbook of Addiction Psychopharmacology. Wiley-Blackwell; Oxford, UK: in press. [Google Scholar]
  30. Stein JS, Johnson PS, Smits RR, Renda CR, Liston KJ, Shahan TA, Madden GJ. Early and prolonged exposure to reward delay in rats: Effects on impulsive choice and alcohol consumption. Psychopharmacology (Berlin) doi: 10.1037/a0031245. submitted. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Yates JR, Marusich JA, Gipson CD, Beckmann JS, Bardo MT. High impulsivity in rats predicts amphetamine conditioned place preference. Pharmacology, Biochemistry and Behavior. 2011;100:370–376. doi: 10.1016/j.pbb.2011.07.012. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES