Reducing Impulsive Choice VII: Effects of Duration of Delay-Exposure Training

Abstract

Impulsive choice is related to substance use disorders, obesity, and other behaviors that negatively impact human health. Reducing impulsive choice may prove beneficial in ameliorating these maladaptive behaviors. Preclinical research in rats indicates that one reliable method for producing large and lasting reductions in impulsive choice is delay-exposure (DE) training. In all six of the prior DE-training experiments, rats were given extensive experience (~120 training sessions) with a delayed reinforcement contingency. The present experiment evaluated if similar large and lasting reductions in impulsive choice could be achieved with less training. The duration of DE training between groups of male Wistar rats was 0-sessions (training ended after a lever-pressing acquisition criterion was met), 30, 60, or 120 sessions. Comparison groups were given the same durations of training with immediate reinforcement. A post-training assessment of impulsive choice was completed using an increasing-delay procedure. For rats assigned to the 60-session condition, impulsive choice was reassessed at a 120-day follow-up. DE training reduced impulsive choice but, contrary to expectation, reductions in impulsive choice did not increase with DE-training duration (no significant training-duration by group interaction). Importantly, 60 sessions of DE training produced reductions in impulsive choice that were comparable to prior published findings and this effect remained significant at the 120-day follow-up. Procedural elements that may be responsible for the DE-training effect, and how they could be improved in future experiments, are discussed.

Impulsivity is a multifaceted construct implicated in a number of problematic behaviors including risk taking, novelty seeking, inattention, impulsive action, and impulsive choice (Evenden 1999). Operationally defined, impulsive choice is the preference for smaller-sooner rewards (SSRs) in lieu of larger-later rewards (LLRs). Delay discounting, one process that may underlie impulsive choice, characterizes the subjective devaluation of delayed reinforcers (Mazur 1987; Rachlin et al. 1991; Madden et al. 1999; Green et al. 2007).

In humans, delay discounting is associated with a number of health-impacting problem behaviors. For example, excessive delay discounting (i.e., rapid devaluation of delayed reinforcers) is observed in almost all types of substance use (for meta-analyses, see MacKillop et al. 2011; Amlung et al. 2017), with more pronounced effects in clinical populations (MacKillop et al. 2011). In nonhumans, similar relationships are observed, though these findings are not without exceptions (Stein and Madden 2013).

Excessive delay discounting is also correlated with compulsive gambling (Petry 2001; Alessi and Petry 2003), obesity (Amlung et al. 2016), criminal activity (Vedelago et al. 2019), and risky behaviors (Odum et al. 2000; Chesson et al. 2006), leading some researchers to argue that excessive delay discounting is a trans-disease process (Bickel et al. 2012, 2019); i.e., a temporally extended pattern of choice that increases the probability of meeting diagnostic criteria for addictive and other disorders. If true, then reducing delay discounting may be useful in preventing a number of behavioral maladies (Volkow and Baler 2015; Gray and MacKillop 2015; Bickel et al. 2019).

Although biological variables undoubtedly play a role in impulsive choice (Helms et al. 2006; Stein et al. 2012), learning is demonstrably relevant. Indeed, a variety of learning-based approaches have successfully changed delay discounting in humans and nonhumans (Koffarnus et al. 2013; Gray and MacKillop 2015; Rung and Madden 2018; Scholten et al. 2019; Smith et al. 2019). One replicable approach to reducing nonhuman impulsive choice involves providing rats with prolonged exposure to delayed reinforcement contingencies (Stein et al. 2013, 2015a; Renda and Madden 2016; Renda et al. 2018; Rung et al. 2018; Peck et al. 2019). That is, delay-exposed (DE) rats learn to press a lever, which is then retracted during a cue-light signaled delay to food. Because the lever-retraction + cue-light signals a delay reduction to food, relative to the interfood interval, that signal may function as a conditioned reinforcer (Williams 1994; Shahan and Cunningham 2015). Comparison groups receive either extended exposure to immediate rewards (IE group) or are fallow during the training phase (control group). In prior published experiments, this training has lasted ~120 sessions. While IE and control groups are undifferentiated, DE rats, on average, make fewer impulsive choices (Renda et al. 2018; Rung et al. 2018; Peck et al. 2019). This effect lasts when reassessed following intervening experiences (Stein et al. 2013, 2015b; Renda and Madden 2016), with the longest test-retest interval assessed thus far being 4 months (Renda and Madden 2016).

The present experiment was conducted to parametrically manipulate the duration of DE training, and in particular, to evaluate if less extensive exposure to delayed reinforcement contingencies is adequate to reduce impulsive choice. The rationale for this study was threefold. First, from a practical perspective, laboratory resources could be saved and experiments more expediently conducted if less DE training produced similar reductions in impulsive choice, both immediately after training and at 4-month follow-up. Second, because learning-based interventions discovered with nonhumans have been successfully translated to reduce human impulsive choice (Mazur and Logue 1978; Schweitzer and Sulzer-Azaroff 1988), mapping the relation between DE training-duration and efficacy would be useful in evaluating the likely feasibility and appropriate dose of this intervention if adapted to humans. Third, from a theoretical perspective, Killeen (2011) has suggested that learning the contingent relation between a response and its delayed-reinforcing consequence may be critical in nonhuman self-control choice. If so, then simply acquiring an operant lever-press under delayed-reinforcement contingencies, with no additional exposure to delays, should suffice to reduce impulsive choice. To these ends, a between-groups design was used to assess impulsive choice following 0, 30, 60, or 120 sessions of DE or IE training. The 0-session condition (no further training after a response-acquisition criterion was met) served to evaluate the effects of learning to lever press with delayed- vs. immediate-reinforcement on subsequent impulsive choice.

Method

Subjects

One-hundred sixty-two experimentally naïve male Wistar rats (Harlan Laboratories, Indianapolis, IN), approximately 21 days old at intake, served as subjects. This study was conducted over 22 months in cohorts of 15 to 40 rats, with rats randomly assigned to DE and IE groups. Sessions were mostly conducted 7 days per week, with some exceptions on Sundays. The DE and IE groups that completed 120 sessions of training are the same rats (and data) reported in Peck et al. (2019). In the latter study, rat strain and supplier, age at intake, and training procedures are very similar to those used with the other groups. Rats were individually housed in a temperature- and humidity-controlled colony room and maintained at 85% of their dealer-supplied free-feeding growth curves. Supplemental feeding was provided approximately 2 h after experimental sessions. Access to water was freely available in the home cage. Procedures were conducted in accordance with protocols approved by the Institution of Animal Care and Use Committee at Utah State University (IACUC protocols 2602 & 2232).

Apparatus

Thirty-six Med-Associates operant chambers were used. Each chamber was housed within a sound-attenuating cubicle outfitted with a ventilation fan. Experimental manipulanda were positioned on the front and rear chamber walls. Two retractable levers were positioned on the front wall (6.5 cm above the grid floor). A food receptacle was centered between these levers (2.5 cm above the grid floor). A pellet dispenser, positioned outside the chamber, delivered 45-mg pellets (Bio-Serv, Frenchtown, NJ) to the food receptacle. A third lever was centered on the rear wall of the chamber (6.5 cm above the grid floor). Above each lever was a 28-V cue light. A white-noise generator was positioned in the upper right-hand corner of the rear wall (13 cm above the grid floor).

Procedures

Table 1 provides an overview of the sequential phases completed by rats in each group; it also lists their approximate age at phase onset. Rats were randomly assigned to DE- or IE-training groups, with the constraint that group sizes be equal at 20 rats. All groups completed lever training followed by either 0, 30, 60 or 120 sessions of DE or IE training. Post-training assessments of impulsive-choice followed. Details of each phase are discussed below.

Table 1.

Approximate Age of Rats, in Days, at the Beginning of Each Phase and Analytic N for Each Group

		Phase			Analytic N
Condition	Lever Training	DE/IE Training	Impulsive-Choice Test	Imp-Choice Reassess	DE	IE
0-session	35	-	40	-	16	16
30-session	35	41	71	-	20	17
60-session	35	42	102	241	17	17
120-session	26	48	184	-	20	20

Note: DE and IE represent delay- and immediacy-exposure training, respectively.

Lever training.

A modified autoshaping procedure was used to establish lever pressing (e.g. Stein et al. 2013, 2015a; Renda and Madden 2016). Each 2-hr session consisted of 20 trials. In the first five trials, the rear-wall lever and corresponding cue light were activated 15 s prior to a scheduled food delivery (two pellets). Pressing the lever during this 15-s interval produced food either immediately (IE rats) or following a 17.5-s delay (DE rats); during the delay period, the cue light remained illuminated. If the lever was not pressed within 15 s, the lever and light were deactivated and response-independent food was provided immediately (IE) or following the signaled 17.5-s delay (DE). For the remaining 15 trials, the cue light and center lever remained in the chamber until a lever press occurred and food events were response-dependent. As before, these food deliveries were either immediate (IE) or delayed (DE). For DE rats, a cue-light was always presented throughout the delay to food. All stimuli except white noise were deactivated during the “blackout” intertrial interval (ITI).

To encourage the same rate of response acquisition across groups, a cycle to trial (C:t) ratio of 11:1 was programmed during the first 5 trials (see Gibbon et al. 1977). The inter-food-interval served as the cycle time (C); the maximum duration that the cue light could be illuminated served as the trial time (t). During the initial 5 trials, if the lever was not pressed the C:t ratio for IE rats was 165-s: 15-s (C:t= 11) and for DE rats it was 357.5-s:32.5-s (C:t= 11). Lever training continued until the rats pressed the lever on at least 18 of the 20 trials for two consecutive sessions.

Delay- and immediacy-exposure training.

Next, rats in the 30, 60, and 120 session groups completed either DE or IE training for the number of sessions indicated in the group names. During each training session, the insertion of the rear-wall lever and illumination of the cue light marked the beginning of a trial. For DE rats, a lever-press immediately retracted the lever and initiated a 17.5-s delay. At the end of the delay, the cue light was extinguished, and two food pellets were delivered. For IE rats, a lever press retracted the lever, extinguished the cue light, and two food pellets were delivered immediately. For both groups, failure to press the lever within 15 s was scored as an omission; omitted trials were repeated. An adjusting ITI ensured that trials began every 60 s, regardless of response latencies. Sessions ended after 80 completed (non-omitted) trials or 2 h, whichever came first.

Amount-discrimination training.

Following lever training (0-session groups) or DE/IE training (30-, 60-, and 120-session groups), amount-discrimination training sessions were conducted to ensure rats could discriminate different reinforcer amounts. These sessions were composed of three 20-trial blocks separated by a 7-min intercomponent blackout. In each trial block, there were 6 forced-choice trials followed by 14 free-choice trials. Trials began by presenting the rear-wall lever and cue light. A response retracted the lever, extinguished the cue light, and either one (force-choice trials) or both (free-choice trials) front-wall lever(s) and cue light(s) were presented. A press to a front-wall lever retracted the lever(s). For IE rats, the cue light(s) were darkened, and the corresponding food reinforcer (either one or three pellets) was delivered immediately; for DE rats, a 17.5-s delay, accompanied by the cue light on the selected lever, preceded the corresponding reinforcer delivery. The assignment of reinforcer amount to the front-wall levers was counterbalanced across subjects. An adjusting ITI ensured new trials began every 60 s. Omitted forced-choice trials (no response within 30 s) were repeated. Sessions ended after 60 completed trials or 2 hrs elapsed. Amount-discrimination training continued until percent larger-reward choice during the free-choice trials was ≥ 90% for two consecutive sessions.

Impulsive-choice assessment.

A within-session increasing-delay procedure was used to assess impulsive choice (Evenden and Ryan 1996). Trial structure was the same as in amount-discrimination training except that the delay to the larger reward increased across successive trial blocks (0, 15, and 30 s). Remedial amount-discrimination sessions were conducted if percent LLR choice in the 0-s delay block dropped below 60% for two consecutive sessions (see Renda et al. 2018 for details). The impulsive-choice assessment continued for at least 14 sessions and until the following stability criteria were met: 1) ≥ 80% choice of the three-pellet alternative in the 0-s delay block for 3 consecutive sessions, 2) area under the curve (AUC; see in each of the final 3 sessions did not deviate by more than 20% from the mean of the final 3 sessions, and 3) no increasing or decreasing trend in AUC over the final 3 sessions.

Impulsive-choice reassessment.

For rats in the 60-session group, DE training produced a large reduction in impulsive choice. Therefore, impulsive choice was reassessed 120 days after the initial assessment was completed. During the test-retest interval, these rats remained in their home cages and had ad-libitum food access; rats were returned to 85% of their current free-feeding weights prior to beginning the reassessment. Reassessment included amount-discrimination training as described above, with the exception that the levers to which the SSR and LLR were assigned were reversed and the delay to both food options was delivered immediately in the DE group. If a lever bias was suspected (percentage of LLR choices across all delays was ≥ 90%, and this did not replicate a preference for that reward in the initial impulsive-choice assessment), an additional lever reversal(s) was conducted.

Data Analysis

Nonchoice measures were evaluated using nonparametric tests (i.e., Mann–Whitney U and Kruskal-Wallis tests) because some distributions violated normality. Specifically, number of days to acquire lever pressing and number of sessions to pass amount discrimination were compared across groups and conditions (training durations). Latency to respond and omissions were averaged across the final 5 sessions of DE/IE training and compared across groups and conditions. Pearson’s r correlations were used to assess the relation between the number of days to meet the lever-training criterion and the summary measure of impulsive choice (AUC) for each group across conditions. AUC was calculated as the proportion of the area under the choice curve formed by the larger-later choice percentages plotted as a function of their delay (see Myerson et al. 2001). For this and subsequent analyses, p-values <.05 were considered statistically significant. All groups completed nearly all programmed trials, so this measure was not included for analysis.

A beta regression was used to examine the effects of DE/IE training duration on impulsive choice (AUC). Beta regression better accommodates the bounded nature of AUC and non-normal distributions of data, the latter of which often occur with AUC under this experimental manipulation (see Rung et al. 2018; Peck et al. 2019). Independent variables of Group (DE/IE), Training Duration (0-/30-/60-/120-session), and their interaction were included in the model. In the absence of a significant interaction, subsequent pairwise comparisons were used to evaluate overall group differences in AUC for zero and non-zero training durations (e.g., 0- vs. 30-sessions), and cumulative training durations (e.g., 30 vs. 60-sessions). Some rats had AUC values of 1.0 (i.e., exclusive choice of the large-reward alternative, regardless of delay), which cannot be accommodated in the beta regression; thus, all AUCs were transformed by subtracting a small, constant value (.0001) prior to analysis.

To ensure model results were not biased by overly influential values, diagnostic screening was conducted by evaluating Cook’s D values and standardized residuals. Observations producing Cook’s D values exceeding a cut-off of 4/N, or standardized residuals greater than or equal to 3 (i.e., three standard deviations) were excluded and the model was re-run. Eight rats’ data were screened out for meeting these criteria (4 for large residuals, 4 for high Cook’s D values) in the primary analyses of AUC. These exclusions made the predicted AUC values closer to the median for each group and increased precision (model parameter associated with the variance of the dependent variable). The results of models with and without these exclusions are reported below; unless otherwise stated, predicted values are from the model without influential/outlying values.

An additional beta regression was conducted comparing AUC values between the 60-session rats’ impulsive choice reassessment and the 120-session rats’ initial test of impulsive choice; the age of rats across these two groups were approximately the same at the time of these tests. This analysis included independent variables of Group (DE/IE), Training Duration (60-session/120-session), and their interaction, which served to clarify the degree of and the longevity of this shorter-trained DE effect. If the DE effect after 60 sessions deteriorated below that seen after the typical duration of training (or if the additional 60 sessions substantially increased the training effect), this would be revealed by a significant interaction. Following the same diagnostic steps described above, two rats’ data were screened out for large Cook’s D values. Results from the initial and post-diagnostics models are reported below. To facilitate comparisons across training conditions, Cohen’s d was calculated for DE/IE group differences in AUC at each training duration. Wilcoxon signed-rank tests were also conducted within the DE and IE groups in the 60-session condition to evaluate the stability of the training effect on a within-subjects basis.

Analyses were conducted in R (R Core Team, 2013); data and analysis code are available in the Open Science Framework: 10.17605/OSF.IO/C7BAM. Beta regressions were conducted using the “betareg” package (Cribari-Neto and Zeileis 2009); test statistics for overall Group-level effects (Wald Chi-squared) from the beta regressions were obtained using the “car” package (Fox and Weisberg 2011); and pairwise comparisons of categorical variable levels in the beta regression were conducted using the “lsmeans” package (Lenth 2016). Correlations and nonparametric analyses for nonchoice outcomes were conducted using the “furniture” package (Barrett and Brignone 2017), and all other tests were conducted using functions available in the base R software.

Results

One hundred forty nine of the 162 rats comprised the final study sample; the final, analytic N is shown in Table 1. One rat was excluded from the experiment for failing to acquire lever pressing. Nine rats were excluded for failing to discriminate differences in reward amount in the initial impulsive-choice assessment, and three rats in the 60-session condition were excluded at reassessment either for failing to discriminate reward differences or for an intractable lever bias.

Across all rats, the number of sessions to acquire lever pressing was significantly correlated with AUC in the test of impulsive choice, r = .18, p < .03. However, the correlation was not significant within any training duration condition (ps ≥ .16). In the final 5 training sessions, DE rats’ latencies to press the rear-wall lever were significantly longer that IE latencies among rats completing 30 (medians [IQR]: DE = 1.8 s [1.1-2.9], IE = 0.6 s [0.5-1.0]; p < .001) and 60 sessions (DE = 1.4 s [0.7-1.8], IE = 0.6 s [0.6-1.2]; p < .05), but following 120 sessions of training the difference was not significant. There were no significant differences in rear-lever omissions in the 30, and 60-session groups, there being virtually none, but a nominal difference in the 120-session condition achieved significance (medians [IQR]: DE = 0.5 [0-3], IE = 0.0 [0-0]; p < .001). During the amount-discrimination phase that followed DE/IE training, DE rats in the 30- and 60-session conditions required more sessions than IE rats to meet the amount discrimination criteria (ps ≤ .02).

Individual-subjects’ stable percent LLR choices in the impulsive-choice test, separated by training group and duration, are shown in Figure 1; the corresponding AUC values (individuals, group medians, ± IQR) are shown in Figure 2. The upper portion of Table 2 provides test statistics for the independent variables included in the initial and post-diagnostic beta regressions evaluating primary training and group effects. Group and training duration were significant predictors of AUC (ps < .001); that is, AUC values were larger (less impulsive choice) following DE training relative to IE training, and AUC values increased with training duration. Cohen’s d effect sizes across DE and IE groups were .57 (0-session condition), .74 (30-session), .84 (60-session) and .78 (120-session). While the main effects of the independent variables were significant, the Group x Training Duration interaction was not—neither in the initial model nor in the model post-diagnostics.

Fig. 1.

Individual subjects’ average percent larger-later reward choices made at each delay over the final five sessions. Columns and rows of graphs correspond to the DE (black circles) and IE groups (white circles) and training durations, respectively.

Fig. 2.

Degree of impulsive choice, quantified as area under the impulsive choice curve (AUC) for rats assigned to DE or IE groups at four different training durations. Values closer to 1.0 reflect less impulsive choice. Data are from the stable sessions completed during the initial impulsive-choice assessment. The height of the shaded bars corresponds to group medians, with error bars showing the 1^st and 3^rd quartiles.

Table 2.

Wald Chi-Squared Tests for Independent Variables in the Beta Regression Predicting Area Under the Curve (AUC) Across Training Durations and Groups

Model/Effect	χ2	df	p
AUC Across All Training Durations/Groups
Initial Model
Group	18.57	1	<.0001
Training Duration	21.45	3	<.0001
Group x Training Duration	1.84	3	.61
Post-Diagnostics Model
Group	32.34	1	<.0001
Training Duration	49.85	3	<.0001
Group x Training Duration	2.56	3	.46
Initial vs. Retest of AUC in 120- and 60-session Rats, respectively
Initial Model
Group	15.77	1	<.0001
Training Duration	15.71	1	<.0001
Group x Training Duration	0.05	1	.83
Post-Diasnostics Model
Group	19.80	1	<.0001
Training Duration	19.70	1	<.0001
Group x Training Duration	0.01	1	.91

To investigate specific differences in AUC across Groups and Training Durations, pairwise comparisons were corrected for multiple comparisons using the False Discovery Rate method (Benjamin and Hochberg 1995). Compared to other methods, the False Discovery Rate controls for Type I error while balancing the likelihood of a Type II error. Estimated mean differences, test statistics, and p-values for Group and Training Duration comparisons are provided in Table 3. The main effect of Group revealed that overall, DE rats had significantly higher AUC values than IE rats (for DE and IE groups, predicted AUCs from the model post-diagnostics were, respectively, .72 [SE = .03] and .51 [SE = .04]).

Table 3.

False-Discovery Rate Corrected Comparisons of Area Under the Curve (AUC) across Rats as a Function of Group or Training Duration in the Initial and Post-Diagnostic Models

Comparison			Estimated Difference in AUC	Std. Error	z	p
Initial Model
	Group
		IE vs. DE	−0.17	0.04	−4.29	<.0001
	Training Duration
		0 vs. 30	.11	.06	1.85	.09
		30 vs. 60	−.16	.06	−2.67	.02
		60 vs. 120	−.09	.05	−1.79	.09
		0 vs. 60	−.05	.06	−0.79	.43
		0 vs. 120	−.14	.05	−2.57	.02
Post-Diagnostics Model
	Group
		IE vs. DE	−0.21	.04	−5.63	<.0001
	Training Duration
		0 vs. 30	.04	.06	0.71	.48
		30 vs. 60	−.22	.05	−4.14	.0001
		60 vs. 120	−.10	.04	−2.18	.04
		0 vs. 60	−.18	.05	−3.24	.002
		0 vs. 120	−.27	.05	−5.39	<.0001

Note. Negative estimated differences in AUC indicate that the group listed second in the comparison had a greater estimated AUC. All comparisons are conducted collapsed across all levels of the other independent variable (i.e., IE vs. DE represents group differences across all training durations).

Significant differences as a function of Training Duration differed slightly across the initial and post-diagnostic models. In the initial model, 120 but not 60 sessions of training produced significantly greater AUCs relative to 0 sessions (collapsed across DE and IE groups due to the lack of a significant interaction above). In comparing incremental effects of training durations (i.e., 0 vs. 30 sessions, 30 vs. 60, and so on), reductions in impulsive choice emerged at 30 vs. 60 sessions of training (p = .02), but the subsequent increase (from 60 to 120 sessions) produced no additional, significant reduction in impulsive choice (p = .09). In the post-diagnostics model, training effects emerged earlier, beginning at 60 sessions of training relative to 0 sessions p = .002); and there were additional, significant reductions in impulsive choice when the training duration was increased from 60 to 120 sessions p = .04).

Figure 3 shows individual subjects’ and group median AUC values obtained in the groups that completed 60-sessions of DE (left panel) or IE (right panel) training, the only groups that completed both an initial (post-training) test of impulsive-choice and, 120-days later, a reassessment of choice. There were no significant test-retest changes in AUC in either group (DE: V = 104, p = .74; IE: V = 104, p = .21).

Fig. 3.

Left and right panels show individual-subject and group median area under the curve (AUC) values of 60-session delay-exposure (DE) and immediacy-exposure (IE) groups, respectively. Initial values are from the first, post-training assessment of impulsive-choice, and retest values are from the reassessment conducted 120-days later.

A comparison of AUCs between 60-session rats at reassessment, and 120-session rats at initial testing (a comparison that approximately equated the age of all rats in the analysis), revealed significant effects of Group and Training Duration, but no significant Training Duration by Group interaction. Model results for this analysis (initial and post diagnostics) are provided in the lower portion of Table 2. Thus, these DE rats had higher AUCs than IE rats (estimated ${\stackrel{‒}{x}}_{\mathrm{diff}}=.22[\mathit{SE}=.05]$, z = 4.50, p <.0001), and rats receiving 120 sessions of training (collapsed across DE/IE training type) had higher AUCs (estimated ${\stackrel{‒}{x}}_{\mathrm{diff}}=-.22[\mathit{SE}=.05]$, z = −4.49, p <.0001). The lack of a significant interaction in the presence of these main effects indicates (a) the effect of Group (DE vs. IE) remained significant in 60-session rats at the reassessment and (b) the additional 60 sessions of training received by the 120-session DE group yielded no additional reductions in impulsive choice relative to the long-term pattern of choices made by 60-session DE rats.

Discussion

The results of the current experiment replicate, for the seventh time, prior studies showing that DE-trained rats make significantly more self-controlled choices than IE-trained rats (e.g., Stein et al. 2015a; Renda and Madden 2016). However, the efficacy of DE training did not increase significantly with the duration of that training, which was evidenced by the lack of a significant Group by Training Duration interaction. The long-term effects of 60 sessions of DE training proved robust to time, as impulsive choice in this group was not significantly different from test to retest (4 months later), and, at retest, the choices of the 60-session DE rats were comparable to those of 120-session DE rats. Thus, short- and long-term reductions in impulsive choice may be achieved following 60-sessions of DE training; the long-term effects of briefer training durations must await future research.

The lack of a significant interaction was a surprising outcome, likely due in part to within-group variability and to a visually apparent increase in self-control in the IE groups as training duration increased. While the latter increase in self-control choices could be a product of maturation, it might simply be an outlier sample of IE rats. In prior published experiments, rats given 120-sessions of IE training had median AUC values ranging from .26 to .31 (Stein et al. 2015a; Renda and Madden 2016; Renda et al. 2018; Rung et al. 2018), well below the median of .75 in the present experiment. Further supporting the outlier-sample hypothesis, this prior range is comparable to the range of AUC values among rats given 0-60 sessions of IE training in the current experiment (see Figure 2).

The within-group variability that may be observed in Figures 1 and 2 also played a role in the lack of a statistically significant effect of DE training-duration on self-control outcomes. This level of variability is typical of past studies, including rats undergoing DE training. Factors predicting individual differences in responsiveness to DE training are unknown. One candidate predictor, pre-training levels of impulsive choice, has been explored in only one study (Renda et al. 2018), but homogeneity of impulsive choices at pre-training precluded a meaningful evaluation of this possible predictor. Other candidate factors that deserve empirical evaluation include individual differences in (a) sign- vs. goal-tracking during signaled delays to the LLR, (b) baseline levels of interval timing (accuracy and precision), and (c) the extent to which the delay-signaling stimulus acquires a conditioned-reinforcing function.

Although the present findings do not allow us to conclude that the efficacy of DE training improves with training duration, they do support the conclusion that 60- and 120-sessions of DE training produce comparable reductions in impulsive choice, both immediately after training and, for 60-session DE rats, when reassessed 120 days later. Other learning-based approaches to reducing impulsive choice have found these reductions to be long lasting. For example, Logue and Mazur (1981) reported that pigeons trained to choose LLRs remained less impulsive than controls when retested 11 months later. Similarly, Bailey et al. (2018) reported that the impulsivity-reducing effects of temporal training under fixed-interval schedules were retained when retested 9 months later (Bailey et al. 2018). In addition, alcohol consumption (Stein et al. 2013, 2015b) and experience with a novel instrumental-learning task (Renda and Madden 2016) intervening between DE training and the reassessment of impulsive choice did not compromize the impulsivity-reducing effects of that training. Such findings suggest a role for long-term memoiy processes, and associated structural changes (Olds et al. 1990). That said, it is clear that 60-sessions of DE training was adequate to produce large and lasting reductions in impulsive choice. This finding increases the practicality, and hence the translational potential of this impulsivity-reducing technique.

Returning to another rationale for the present experiment, we sought to evaluate if learning the contingent relation between a response and the delayed-reinforcer it produces was a sufficient condition for reducing impulsive choice (Killeen 2011). Comparing DE and IE rats in the 0-session condition offers the best opportunity to test this hypothesis. DE rats learned this if response then delayed-reinforcer contingency during lever-training, whereas IE rats were given no experience with delayed reinforcers. In an exploratory analysis comparing 0-session DE and IE groups, DE rats showed significantly higher AUCs (Mann-Whitney U = 76.5, p < .05; d = 0.57). Other supporting evidence comes from Stein et al. (2015) who reported that, on forced-choice LLR trials, DE rats spent more time during delays to food (than IE rats) with their head in the feeder area, thereby suggesting that they had learned the contingent relation between their response and the delayed food reward.

The significant reduction in impulsive choice produced in the 0-session DE group is an intriguing finding that deserves future attention. The training provided to rats assigned to the 0-session groups was a modified Pavlovian autoshaping procedure. Specifically, during the first 5 trials, the conditioned stimulus (CS) onset was a compound stimulus – inserted lever and illuminated cue light – signaling that, in the absence of a lever press, food would be delivered 11 times sooner than the inter-food interval (C:t = 11 for both groups; see Balsam and Gallistel 2009). For DE rats, when the lever was pressed (or 15 s elapsed without a press), the lever was retracted but the cue-light remained lit for 17.5 s; this event signaled a further delay-reduction to food (C:t = 375.5/17.5 = 20.4). For IE rats, food was delivered immediately after a lever press (or 15 s without a press), so they had no experience with a cue-light alone CS. Such Pavlovian training may have played a role in DE rats’ stronger preference for LLRs in the test of impulsive choice. That is, presenting the cue-light alone CS throughout the LLR’s 17.5-s delay to food may have reduced the aversiveness of the the delay (see also Peck et al. 2019), particularly if that CS functioned as a conditioned reinforcer (Williams 1994). Such Pavlovian contingencies, and the consolidated long-term memory common in associative learning (Olds et al. 1990), might usefully be exploited to improve the efficiency, efficacy, and durability of reductions in impulsive choice. For example, a more Pavlovian-inspired approach would arrange a highly salient CS which uniquely signals a large delay-reduction to food, while maintaining the 17.5-s delay to food; e.g., C:t = 350/17.5 = 20. Following acquisition across an extended span of time (Gallistel and Papachristos 2020), the CS would, during the test of impulsive choice, be arranged as the delay-bridging stimulus delivered immediately upon choosing the LLR. For control rats with no experience with this CS, its onset would not function as a conditioned reinforcer. Of interest is if this Pavlovian training alone would more efficiently produce lasting reductions in impulsive choice than DE training.

Future experiments should also address three limitations of the present study. First, we did not evaluate if 0- or 30-sessions of DE training produced lasting reductions in impulsive choice when assessed at a 120-day follow-up. If reductions in impulsive choice following minimal DE training are replicable, it would be important to evaluate if the effects last as long as those produced by more lengthy intervals of DE training. Second, comparing impulsive choice between DE rats assigned to the 0- to 120-session conditions is complicated by an age confound; i.e., the latter group was 4 months older than the former when the impulsive-choice test was conducted. Impulsive choice is known to decrease with age in rodents (Pinkston and Lamb 2011; Doremus-Fitzwater et al. 2012). Although Renda et al. (2018) reported only small maturation-related reductions in impulsive choice over the time spans of the present experiment, if future research is to evaluate effects of different training durations, better maturational controls are needed. One approach would be to begin shorter-duration training (e.g., 0-session groups) at a later age, such that all groups may be tested at the same chronological age. This, however, introduces a different confound – age during training – and no studies have evaluated if training is more effective when begun earlier in the lifespan. Replicating the DE effect in longer-living species such as pigeons or monkeys could address the maturation issue if impulsive choice is otherwise stable over the interval needed for training and testing; such a replication would simultaneously assesses the inter-species generality of the DE effect. Finally, the data from the 120-session condition are those previously reported by Peck et al. (2019); readers should be careful not to mistake those data as another replication of the DE effect following 120 days of training.

Conclusions.

In sum, the present data support three conclusions. First, the data replicate, in a seventh published paper, the finding that DE training reduces impulsive choice relative to training with immediate reinforcement (IE training). Second, periods of DE training shorter than 120 sessions can produce significant reductions in impulsive choice. Third, 60 sessions of DE training were adequate to produce a significant reduction in impulsive choice, and that reduction was maintained when reassessed 120 days later. Future studies should evaluate if shorter periods of training, when combined with optimized Pavlovian contingencies, can produce large and lasting reductions in impulsive choice.