Reducing Impulsive Choice: V. The Role of Timing in Delay-Exposure Training

Abstract

Impulsive decision-making is common in addiction-related disorders, with some research suggesting it plays a causal role in their development. Therefore, reducing impulsive decision-making may prevent or reduce addiction-related behaviors. Recent research shows that prolonged experience with response-contingent delayed reward (delay exposure [DE] training) reduces impulsive choice in rats, but it is unclear what behavioral mechanisms underlie this effect. The present study evaluated whether improvements in interval timing mediate the effects of DE training on impulsive choice. Thirty-nine Long-Evans rats were randomly assigned to groups completing DE, immediacy-, or no-exposure training, followed by impulsive-choice and timing tasks (temporal bisection). Despite replicating the DE effect on impulsive choice, timing accuracy and precision were unaffected by DE training and unrelated to impulsive choice. The present findings did not replicate previous reports that timing precision and impulsive choice are related, which may be due to between-laboratory differences in impulsive choice tasks. Continued research to identify candidate behavioral mechanisms of DE training may assist in improving training efficacy and facilitating translational efforts.

1. Introduction

Addictions are characterized by habitual preferences for immediate outcomes over more optimal, delayed outcomes (Bickel and Marsch, 2001), which reveals steep devaluation (discounting) of future outcomes (Odum, 2011). Indeed, individuals with obesity (Amlung et al., 2016; Jarmolowicz et al., 2014), substance misuse (Kirby, Petry, & Bickel, 1999; Petry, 2001), and gambling disorder (Reynolds, 2006) steeply discount delayed rewards (see Amlung et al., 2017; and MacKillop et al., 2011, for reviews). Some longitudinal evidence suggests steep discounting predicts future cigarette smoking (Audrain-McGovern et al., 2009) and alcohol use (Fernie et al., 2013). Thus, reducing delay discounting may prove useful in preventing or reducing addictions (Gray and MacKillop, 2015; Volkow and Baler, 2015).

Prolonged experience with delayed reinforcement (delay-exposure [DE] training) reduces impulsive choice in rats (Stein et al., 2013). The effect is reliable, producing significant and large (d = 1.19 to 3.30) reductions in impulsive choice in Long-Evans and Wistar rats (Renda et al., in press; Renda and Madden, 2016; Stein et al., 2015, 2013). The behavioral mechanisms by which DE training works are unknown, but their identification may aid in improving training efficacy and/or in guiding effective translation.

One possible mechanism of DE training is interval timing. Marshall, Smith, and Kirkpatrick (2014) found that precision of rats' interval timing was strongly correlated (r = -.73) with impulsive choice, but timing accuracy was not. Subsequently, Smith et al. (2015) found that experimentally improving timing precision reduced impulsive choice. The present research evaluated the hypothesis that DE training reduces impulsive choice via increases in timing precision.

2. Methods

2.1. Subjects

Sample size was based on effect sizes from previous DE studies and the correlation between timing precision and impulsive choice (Marshall et al., 2014). Subjects were 39 Long-Evans rats (Harlan, Indianapolis, IN) individually housed under a controlled 12 hr light/dark cycle. At PND 26, rats were food restricted to their projected 85% weight (based on vendor-supplied growth curves) but had continuous access to water in the home cage. Sessions were conducted daily at the same time. Rats received supplemental food 2-hours post-session.

2.2. Apparatus

Ten identical operant chambers (Med Associates, St. Albans, VT) within sound-attenuating, ventilated cabinets were used. Each chamber contained a white-noise generator, two low-profile levers (10.5 cm above the chamber floor) on either side of a food receptacle on the front wall, and one low-profile lever on the back wall. A 28-V stimulus light was above each lever and a house light above the food receptacle; the latter was connected to a pellet dispenser delivering 45 mg food pellets (Bio-Serv, Frenchtown, NJ).

2.3. Procedures

At intake, rats were randomly assigned to one of three groups¹: delay-exposed (DE, n = 12), immediacy-exposed (IE, n = 12), or no-exposure control (CONT, n = 12). First, rats were trained to lever-press using an auto-shaping procedure (see Stein et al., 2013). Next, rats completed exposure-(DE/IE) or no-training (CONT) (section 2.3.1). Then, rats completed an impulsive choice assessment (2.3.2), a temporal bisection task (a measure of interval timing; 2.3.3), and a re-test of impulsive choice.

2.3.1. Exposure Training

Rats assigned to DE and IE groups completed an average of 125 (range 123-127) sessions in which lever pressing produced two pellets immediately (IE) or following a delay (DE). Trials began with insertion of the rear lever and illumination of its cue light. Pressing the lever caused it to retract; then, either the cue light was extinguished and two food pellets were delivered immediately (IE), or the cue light remained on for 17.5 s until food delivery (DE). Failure to lever press within 20 s terminated the trial; these were recorded as “omissions” and the trial was repeated. Trials began every 50 s. Sessions ended after 80 food deliveries or 2 hours, whichever occurred first. CONT rats were transported to and from the laboratory with DE/IE rats, but did not complete sessions.

Side-lever training commenced after completion of DE, IE, or the equivalent duration of no-training (CONT). This consisted of 40 presentations each of the left and right levers (randomly determined) on the front wall. The same contingencies for lever pressing, cue light illumination, and omissions during DE/IE training were active during side-lever training. CONT rats completed this training with the same contingencies as IE rats. Side-lever training terminated when rats completed ≥ 90% of trials on each lever for two consecutive days.

2.3.2. Impulsive Choice (IC) Assessment

The impulsive choice (IC) assessment was modeled after that in Evenden and Ryan (1996) and previous DE studies (e.g., Renda & Madden, 2016). Sessions consisted of repeated choices between a 1-pellet and 3-pellet alternative (randomly assigned across rats to left and right levers), presented across three blocks of 20 trials each. The first 6 trials within a block were single-lever forced-choice trials (3 left and 3 right, order randomly determined) and the remaining 14 were two-lever free-choice trials. A centering response on the rear lever initiated all trials. Omissions (as previously defined) were only repeated on forced-choice trials. Trials occurred every 70 s, with a 7-minute blackout between trial blocks.

Prior to the IC assessment, rats chose between 1 and 3 pellets with equal delays across the side levers. These amount discrimination sessions followed the structure outlined above, but the reward delay was 0 s for IE and CONT rats, and 17.5 s for DE rats (cue light above the retracted lever stayed on for the duration of the delay). Amount discrimination continued until the 3-pellet reward was chosen on ≥ 90% trials for two consecutive days.

During the IC assessment, the one-pellet reward was delivered immediately while the delay (or lack thereof) to the three-pellet alternative increased across the three blocks (from 0 s, 15 s, and 30 s; as before, cue light on during the delay). This phase lasted at least 15² sessions and until choices stabilized over 6 sessions. Choice was considered stable when (a) the average area under the curve (see section 2.4) for the last three sessions deviated from that of the previous three by ≤ 0.15, and (b) average preference for the larger reward in the first trial block (when both rewards were immediate) was ≥ 80%.

2.3.3. Temporal Bisection (TB) Task

Temporal bisection (TB) task procedures were based on Marshall et al. (2014). Rats were first trained to discriminate between short (4.00 s) and long (12.00 s) house light illuminations. Trials began with house-light illumination (4.00- or 12.00-s duration randomly determined), followed by insertion of the left and right levers. One lever was assigned as correct following short illuminations, and the other following long (counterbalanced across rats). Correct lever selections produced two food pellets; when incorrect selections occurred, the trial was repeated until correct following a 5 s ITI. Sessions consisted of 80 trials (half of each stimulus duration) and training continued until accuracy was ≥ 80% for both durations across two consecutive days.

During testing, 66 training trials (as above, but trials not repeated following incorrect selections) and 35 non-reinforced probe trials were randomly intermixed. Probe trials consisted of 4.00, 4.80, 5.70, 6.93, 8.32, 9.99, and 12.00 s house-light illuminations. One training session separated every three testing sessions to maintain accurate discriminations, with alternations continuing until 12 testing sessions were completed.

2.4. Data Analysis

Group differences in non-choice measures (e.g., days to acquire lever pressing, omissions during DE/IE training) were evaluated using non-parametric tests due to non-normal distributions (Kruskall-Wallis and Wilcoxon rank-based tests). Impulsive choice was quantified for each rat as the average area under the proportion-choice curve (plotted as a function of delay, the AUC; see Myerson, Green, & Warusawitharana, 2001) in the stable sessions. Smaller AUCs correspond to greater impulsive choice. Exploratory correlations between AUC and non-choice measures are reported in supplemental materials because most significant relations appeared spurious and/or driven by outliers (see Figures S1-S3). As in Marshall et al. (2014), the following cumulative logistic distribution was fit to individual subject timing data (average proportion long-interval responses over the final 5 days) to estimate timing precision:

$$p(\mathit{\text{Long}})=\frac{1}{(1+e^{\frac{-(x-\mu )}{\sigma }})}$$ Eq. (1)

where x is the stimulus duration, μ is the mean of the function (accuracy) and σ is the standard deviation (precision); smaller values of σ indicate greater precision.

The effects of training on impulsive choice and the mediating role of timing precision were evaluated using beta regressions. Beta regression is a generalized linear model that assumes the dependent measure is beta-distributed and bound between 0 and 1 (Ferrari and Cribari-Neto, 2004; Simas et al., 2010). All analyses were conducted in R (R Core Team, 2013); beta regressions were conducted using the betareg package (Cribari-Neto & Zeileis, 2010). Typical regression diagnostics were performed and problematic observations removed (details in Supplemental Materials). These removals improved model predictions but did not change the results. Mediation analysis was conducted following the steps outlined by Baron and Kenny (1986), but beta regression could not be used for the second step (evaluating if training impacted timing precision) because timing precision is not expressed as a proportion. Instead, a Mann-Whitney U test was used because no regression techniques were appropriate for both AUC and timing precision as outcomes. Because IE and CONT did not significantly differ on dependent measures and using CONT rats as the comparison did not change the results, the mediation analysis uses the DE and IE groups.

For the regression and mediation-related analyses, we computed Bayes factors, which compared the likelihood of the null hypothesis relative to the alternative. Following an approach similar to that in Gallistel (2009), Bayes factors for evaluating group differences in timing precision were calculated using the BayesFactor package (Morey and Rouder, 2015). Bayes factors for comparing regression models were calculated using the Bayesian Information Criterion (see Jarosz and Wiley, 2014 for detail). Note that computation of the Bayes factor for the former is based on a t-test; values for timing precision in the DE group did not pass a test of normality, although the departure was small (see Figure 1) and the results of a t-test and the Mann-Whitney U test for these data produced the same conclusions. Thus, the absolute value of the Bayes factor may be slightly biased, albeit the relative weight of evidence representative.

Figure 1.

Top row: Proportion larger-later reward (LLR) choices as a function of delay, paneled by Control (CONT), Immediacy Exposure (IE) and Delay Exposure (DE) groups. Data represent individual rats' average proportion LLR choice over the stable sessions of the initial test of impulsive choice. The inset graph in each panel shows the individual-subject values for timing precision, derived from the cumulative logistic distribution fits. Bottom row: Average proportion long-lever selections as a function of probe stimulus duration, paneled by group. The proportions are the averages for each rat over the stable testing sessions and the black curve is the cumulative logistic distribution fitted to the group medians. The inset graph in each panel shows the individual-subject values for timing accuracy (mean), derived from the cumulative logistic distribution fits.

3. Results and Discussion

Table 1 shows non-choice measures for each phase. The only significant group difference was the number of days to pass TB training criteria (χ²[2] = 6.00, p = .049): DE rats tended to take 4-5 fewer days than CONT rats, W = 114, p = .02. DE and IE rats performed similarly in the final 5 sessions of exposure training (e.g., trials completed, omissions).

Table 1.

Medians and 1^st and 3^rd quartiles for non-choice (e.g., days to complete autoshaping) and choice (AUC) measures during each experimental phase for each group, as well as p-values from statistical tests of group differences in measures.

Measure	Group			p
	CONT (n = 13)	IE (n = 14)	DE (n = 12)
Days in Autoshaping	12 (6–14)	12 (10–12)	13 (10–14)	.71
Training Trials Completed#	NA	80 (80–80)	80 (80–80)	>.99
Training Trials Omitted#	NA	0.00 (0–0)	0 (0–0.25)	.06
Latency (s) to Press in Training#	NA	0.63 (0.44–1.04)	0.64 (0.52–1.12)	.55
Total Training Trials Completed	NA	9913 (9729–9967)	9860 (9757–9999)	>.99
Days of Side-Lever Training	2 (2–2)	2 (2–2)	2 (2–2)	>.99
Days to Discrimination (Initial, IC)	3 (2–4)	3 (2–3)	4 (3–4)	.09
Days to Stability (Initial, IC)	20 (15–30)	15 (15–21)	18 (16–24)	.32
AUC (Initial, IC)	.25 (.24-.27)	.29 (.25-.46)	.62 (.41-.87)	<.001
Days to Discrimination (TB)	12 (9–14)	10 (8–14)	8 (6–10)	.05*
Days to Discrimination (Re-test, IC)	3 (2–3)	2 (2–3)	4 (3–5)	.09
Days to Stability (Re-test, IC)	15 (15–16)	15 (15–16)	15 (15–16)	.84
AUC (Re-test, IC)	.26 (.25-.28)	.27 (.25-.64)	.61 (.33-.86)	.02

Note: CONT, IE, and DE indicate control (no-exposure), immediacy-, and delay-exposed, respectively. IC and TB indicate impulsive choice and temporal bisection, respectively.

^#Measures marked with an are averages over the final 5 days of exposure training, and descriptive statistics for AUC include all rats.

^*The exact p-value for this test was .0497

The DE training effect was replicated: AUCs and proportion larger-later choice by group and delay are shown in Table 1 and Figure 1, respectively. Group was a significant predictor of AUC, χ²(2) = 16.32, p < .001. DE rats had significantly higher AUCs than both IE (z = 2.72, p = .01) and CONT rats (z = 4.30, p = .0001).

The DE and IE groups did not significantly differ in timing precision, χ²(1) = 0.06, p = .80 (see Figure 1); the estimated Bayes factor (null/alternative) suggested these data were 2.26 times (p = .69) more likely under the null hypothesis than the alternative³. Timing precision was also not a significant predictor of choice when added to the regression model (z = -0.29, p = .77); the Bayes factor indicated these data were 4.17 times (p = .81) more likely to occur under a model absent of timing precision, rather than a model that included it (i.e., the probability that timing precision is related to impulsive choice is low). Thus, no evidence supported the hypothesis that changes in timing precision mediated the DE effect on impulsive choice. These conclusions were unchanged when using a mean-standardized measure of timing precision (σ/μ; to account for the correlation between precision and accuracy). The lack of mediation is unlikely due to experience in the TB interfering with training effects: there were no significant within-subject changes in choice across IC assessments, ps > .69 (see Figure S4). At retest, group remained a significant predictor of choice, χ²(2) = 8.18, p = .02; although the DE-IE difference no longer reached significance (z = 1.49, p = .18). Exploratory analyses also revealed no DE-IE differences in timing accuracy t(17) = 0.48, p = .64, nor was there a significant relation between timing accuracy and choice (z = -1.68, p = .09).

4. Conclusions

No evidence supported the hypothesis that the reduction in impulsive choice following DE training was mediated by improvements in interval timing; moreover, the data favored null hypotheses indicating that (a) DE training had no effect on timing precision and (b) there was no relation between timing precision and impulsive choice. The latter, which is at odds with Marshall et al. (2014), might be due to differences in IC assessments. In their study, the smaller-sooner reward was delayed by 2.5-30 s (larger-later reward delayed by 30 s), whereas a constant, 0-s smaller-sooner delay was used herein. The between-experiment difference might also be explained by differences in the age of the rats (∼30-50 day younger at study onset in our experiment). Future research should explore other candidate mechanisms of DE training; e.g., reductions in delay aversion. Understanding how DE-training reduces impulsive choice may help to develop interventions for reducing/preventing addictions.

Highlights.

• Replicated prior reports that exposure to delayed outcomes reduces impulsive choice in rats.

• Improved timing precision does not mediate delay-exposure effects on impulsive choice.

• Determining mechanisms of delay exposure training may help improve its efficacy.