Generalizability of time-based interventions: Effects of choice procedure and smaller-sooner delay

Abstract

Interventions exposing rats to delayed-reward contingencies attenuate suboptimal impulsive choices, a preference for a smaller-sooner (SS) over a larger-later (LL) reward. Interventions may potentially improve delay-tolerance, timing of delays, and/or discrimination of reward magnitudes. Generalization from the intervention to impulsive choice under different procedures can provide insights into the processes that underlie the intervention effects. Experiment 1 tested intervention effects on systematic-delay (SYS) and adjusting-delay (ADJ) procedures, predicting that intervention effects would be more effective on the SYS procedure with predictable delays. The ADJ procedure did not benefit significantly from intervention, but the SYS procedure, unexpectedly, showed greater impulsive choices following intervention. Experiment 2 tested whether short (5 s) SS intervention delays may have promoted greater impulsivity in the SYS impulsive choice procedure in Experiment 1. Short SS delays in choice and intervention procedures increased impulsive choices in comparison to longer (10 s) delays. Incongruent SS delays in the intervention/choice procedures resulted in negative intervention effects. The results suggest that short SS delays are detrimental to self-control and that specific temporal information generalizes from the intervention to the SYS choice task, but not the ADJ choice task.

1. Introduction

Impulsive decision-making is often assessed using an impulsive choice procedure that requires an individual to choose between two options: a large and delayed reward (larger-later, LL) or a small but relatively sooner reward (smaller-sooner, SS). All other things being equal, impulsive (versus self-controlled) decision-making is defined as a preference for an SS option over an LL option when the SS option is the suboptimal choice for long-term reward maximization. The value of a reward declines as a hyperbolic function of the delay to its receipt (Mazur, 1987, 2000; Odum, 2011). The degree to which LL preference decreases as a function of increased LL delay or decreased SS delay (i.e., sensitivity to delay) is a good index of impulsive decision-making (Bickel et al., 2019; Bickel & Mueller, 2009). Greater preference for the SS option has been observed in individuals with attention-deficit/hyperactivity disorder (Barkley et al., 2001; Solanto et al., 2001; Sonuga-Barke, 2002; Sonuga-Barke et al., 1992), substance use disorders (MacKillop et al., 2011; Verdejo-García et al., 2008), obesity (Rasmussen et al., 2010), and gambling (Dixon et al., 2006). The close ties between these maladaptive conditions and impulsive choices encourages the study of procedures to attenuate impulsive choices in animal models that can potentially translate to clinical interventions in humans.

Impulsive choice appears to have stable, but malleable, trait-like properties in humans (Kirby, 2009; Odum & Baumann, 2010) and rats (Peterson et al., 2015). This demonstrates correspondence between human and nonhuman animal research. Rodent models are a useful tool to investigate the mechanisms that underlie impulsive choice and some of the aspects that promote maladaptive decision-making (Galtress et al., 2012; Stein et al., 2015). Furthermore, the discovery of the mechanisms underlying impulsive choice opens the door to develop interventions to promote self-control (Rung & Madden, 2018; Smith et al., 2019). To the extent that maladaptive decision-making is a product of impulsive decision-making, interventions that treat impulsive choices may concomitantly treat associated impulse-control disorders.

The catalog of studies showing that learning/experiential-based interventions successfully reduce impulsive choices in preclinical research is substantial and collectively supports the claim that experience with delayed reinforcement increases self-control in impulsive choice tasks (e.g., Bailey et al., 2018; Fox et al., 2019; Mazur & Logue, 1978; Panfil et al., 2020; Peck et al., 2019; Rung et al., 2018; Smith et al., 2015). Translational research where experience with delays increases self-control in human participants has shown similar support (Binder et al., 2000; Dixon & Holcomb, 2000), but the mechanisms underlying these experience-dependent interventions are not yet fully understood. Revealing the mechanism(s) of interventions for self-control would allow clinicians to design more effective treatment protocols. If multiple mechanisms are involved, then treatment protocols can be tailored for individual needs.

In an investigation of time-based interventions, Smith et al. (2015; Experiment 2) reported that exposing rats to fixed-interval (FI; rats exposed to consistent delays across trials) and variable-interval (rats exposed to variable delays across trials) schedules of reinforcement increased self-controlled choices with corresponding increases in timing precision (i.e., decreased varience in the peak-interval procedure; Roberts, 1981). This outcome demonstrated that improvements in timing correspond with improvements in self-control, consistent with the observed correlation between individual differences in timing abilities and LL choices (Marshall et al., 2014; McClure et al., 2014; however see, Rung et al., 2018). These time-based interventions may have improved self-control by improving timing, or by increasing delay tolerance (Smith et al., 2019). Peck et al. (2019) supported the delay tolerance hypothesis. Rats were trained to associate a delay with a light cue. A subsequent observing procedure showed that the time-based intervention reduced the rats’ tendency to turn off the stimulus correlated with the delay, but the control intervention did not. This suggested that the stimulus representing the delay was aversive, and that rats exposed to the intervention had greater tolerance to the aversive properties of the delay-stimulus.

Delay is not the only important dimension in impulsive decision-making; sensitivity to reinforcer amount/quality (or subjective reinforcer magnitude) can also affect preference. Marshall and Kirkpatrick (2016) demonstrated that rats that showed better reward discrimination were more LL-preferring and that reward-discrimination interventions improved self-control. Stein et al. (2013) used a reward bundling procedure, in rats, where a single choice led to a chain of either several SS or LL outcomes experienced in a single trial. The experience with “bundled” delay-reward outcomes increased LL choices and this suggested that it increased sensitivity to reward magnitude (Smith et al., 2019). Just as time-based interventions may improve self-control by improving the subjective judgment of delays, the reward-discrimination intervention improved reward magnitude discrimination.

Previous research showed timing-choice and reward discrimination-choice correlations and that these variables changed together following the intervention (Marshall & Kirkpatrick, 2016; Smith et al., 2015). A complementary approach to testing the mechanisms of the interventions is to determine the conditions under which the effects of an intervention do or do not generalize to the choice task. In a systematic investigation of how time-based interventions affect impulsive decision-making, Bailey et al. (2018) demonstrated that intervention-driven improvements in self-control generalized to different choice procedures and the effects lasted for at least 9 months (also see: Renda & Madden, 2016). They demonstrated generality of an FI intervention by showing that it increased self-controlled choices in procedures that systematically varied SS delay, LL delay, and LL reward magnitude across sessions. Thus, the FI intervention improvement of self-control does not appear to be a product of the specific delays or magnitudes used in the choice procedure.

The present experiment sought to further evaluate the generality of the time-based interventions and interrogate the mechanisms governing choices. Traditionally, the effects of interventions on impulsive choice have been evaluated using systematic impulsive choice procedures (SYS). The SYS procedures assess impulsive choices by varying the LL delay across sessions (Green & Estle, 2003) or across blocks of trials within sessions (Evenden & Ryan, 1996) independent of previous choices. Another commonly used choice procedure is the adjusting-delay procedure (ADJ; e.g., Mazur, 1987) where the LL delays vary within-session in response to recent choices. For example, if the rat chooses the LL option, then the LL delay may increase (i.e., to make the LL less attractive), but if the rat chooses the SS, then the LL delay may decrease (i.e., to make the LL more attractive). This process repeats until choices stabilize around a given LL delay, the point of subjective equivalence (PSE). High PSE values are taken as an index of greater self-control.

Given that the reward delays are inconsistent across trials during the ADJ procedure, it is possible that the FI intervention may not generalize to the ADJ task as well as the SYS task. Consistent with this idea, the ADJ procedure produced weaker test-retest reliability and poorer temporal tracking than SYS procedures (Peterson et al., 2015). Moreover, Cardinal et al. (2002) used an ADJ procedure and reported that rats’ choices were poorly sensitive to the adjusted changes in the delay at a trial-by-trial level. They suggested that a degree of random (or “uninformed”) choices was involved when using the ADJ procedure. The results from Peterson et al. (2015) and Cardinal et al. (2002) suggest that the ADJ procedure should result in poor sensitivity to delays, which may imply poor timing of the delays that would lead to less “informed” choices. This would suggest that an intervention that improves timing cognition would not strongly benefit self-controlled choices using an ADJ procedure. Intervention benefits that rely on other processes, such as improving delay tolerance, may increase LL choices in the SYS and ADJ procedures similarly.

Experiment 1 was designed to test the hypothesis that FI interventions would generalize better to improvements in LL choices for a SYS impulsive choice procedure (with reliable delays) over an ADJ impulsive choice procedure (with dynamic delays). Our alternative competing hypothesis was that the ADJ procedure might show increases in LL choices due to delay exposure effects improving delay tolerance. Thus, if we observed improvements in the ADJ procedure, that would indicate a potential role of delay tolerance in the FI intervention, but if not, then that suggests that specific timing of delays may be more important. Experiment 1 also tested timing abilities using the peak-interval procedure. If improvements in timing cognition are involved in improved self-controlled choices, then rats in the intervention group should show better timing performance.

2. Method

2.1. Subjects

Eighty-four male Sprague-Dawley rats (Charles River, Portage, MI; N = 36, Experiment 1; N = 48, Experiment 2) participated in the two experiments. The rats were 21 days of age upon arrival and 35 days of age when initial training started (the rats arrived on separate dates for the two studies), were pair-housed, maintained in a 12:12 reversed light:dark cycle (7am-7pm), tested during the dark cycle, and had ad libitum water access in the home cage and testing chambers. The rats were maintained at a target of 85% of their free-feeding weights (determined by the growth curve information provided by Charles River) and sustained by receiving 45-mg pellets (BioServ, Flemington, NJ) delivered in the testing session and supplemental chow diet (LabDiet 5001, Brentwood, MO) provided in the home cage after the testing session. The amount of post-session supplemental chow was adjusted based upon the number of pellets received to maintain a constant daily caloric intake.

2.2. Apparatus

Experimental sessions were conducted in 24 identical operant chambers (Med Associates, St. Albans, VT). Operant chamber dimensions were 25 × 30 × 30 cm and were placed inside a ventilated noise-attenuated chamber (74 × 38 × 60 cm). The front of each chamber included a centrally located house-light placed above a pellet receptacle (45-mg pellets delivered by a magazine dispenser and recorded head entries via an LED photocell) flanked to the left and right with identical stations that included two key-lights above two response levers. The two levers recorded responses via microswitches. The back of the chamber provided free water access from a bottle located outside of the testing chamber. MedPC-IV controlled the experimental contingencies and recorded all events at a 2-ms resolution for Experiment 1, and MedPC-V controlled the experimental contingencies and recorded all events at a 1-ms resolution for Experiment 2.

2.3. Procedure

Experiments 1 and 2 both use the procedures described below. Specific details of the individual tasks are described separately.

2.3.1. Initial training

Rats received initial training to collect pellets from the receptacle and lever-press for pellets. Rats started the experiment with one session of magazine training where pellets were delivered according to a random-time 60-s schedule until 30 food pellets were delivered. The following two sessions trained the rats to lever press by requiring them to respond on both levers to earn pellets and to progress the schedule requirements across three blocks within the session. The rats were trained to lever press in three blocks within each session where they earned 20 pellets for responding on each lever with successive blocks delivering a concurrent fixed-ratio 1 schedule, a concurrent random ratio (RR) 3 schedule, and a concurrent RR 5 schedule. The two lever training sessions lasted until the rats earned 120 pellets or 120 min had passed.

2.3.2. Behavioral Testing

The Intervention, Impulsive Choice, and Peak procedures were all modifications of a common trial contingency (Figure 1). All trials started with the illumination of the house-light and the extension of one or both lever(s). The first lever press resulted in the house-light extinguishing, the illumination of the key-light above the lever, and the start of an FI schedule. Lever presses during the FI schedule did not produce any programmed outcomes until after the interval elapsed. The first press after the reinforcer was primed resulted in pellet delivery, lever retraction, chamber blackout, and an inter-trial interval (ITI). Lever location assignment to SS and LL was counterbalanced across subjects. The LL option delivered two pellets and the SS option delivered one pellet after a delay that was less than or equal to the LL option.

Fig. 1.

Illustration of trial progression. Trials started with a blackout period and proceeded to (A) a forced-LL trial with only the LL associated lever available (e.g., left lever), (B) free-choice trial with both levers available or (C) a forced-SS trial with only the SS lever available (e.g., right lever). (D) After an LL lever press or (E) an SS lever press, the alternative lever retracted, the light above the lever illuminated, and a response initiated FI delay started. If the LL lever was chosen (D), then the next response after the LL delay resulted in (F) the light above the lever extinguishing and two pellets being delivered. If the SS lever was chosen (E), then the next response after the SS delay resulted in (G) the light above the lever extinguishing and one pellet being delivered. After a pellet delivery, (H) an ITI in blackout (both levers retracted) proceeded until the start of a new trial.

2.3.3. Data Analysis

Impulsive choice data were modeled in R using linear mixed-effects models (lme4 package; Bates et al., 2018) that included fixed (group level variables) and random effects (individual differences). An intercept parameter for individual rats was included as a random effect in each model. Because the slope and intercept parameters were significantly correlated in the random effects, we did not include slope as a random effect to avoid overparameterization (Bates et al., 2015). Impulsive choices using the SYS procedure were analyzed with a mixed-effects model with the lever choice (0=SS, 1=LL) as the dependent variable and a binomial error distribution (i.e., logistic regression). The independent variables included delay (SS or LL, depending upon task) and group assignment. Impulsive choices using the ADJ procedure were analyzed using a mixed-effects model that used the average adjusted LL delay as the dependent variable and specified a Poisson error distribution (appropriate for count data). The independent variables included session and group. Post-hoc analyses (emmeans package; Lenth et al., 2018) were used to test group differences in slope parameters (emtrends) and group differences at the middle LL or SS delay for the SYS tasks (emmeans).

Peak interval data were evaluated using a nonlinear mixed-effects model (nlme package; Pinheiro et al., 2017) fitting a Gaussian function with a linear component (Buhusi & Meck, 2000). All sessions were included in the model for each rat. The peak model predicted responses per second, R(t), as a function time in the interval (t) and had five free parameters,

$$\mathrm{R}(t)=m\times \text{exp}(\text{-}0.5\times [(t\text{-}a)∕v{]}^2)+l\times (t\text{-}a)+s$$

where m represented the maximum response rate at the peak, v was the variance (standard deviation) of the Gaussian (i.e., timing precision estimate), l was a linear component determining the slope of the right-sided tail, (s) represented the starting response rate, and (a) represented the location of the peak (i.e., timing accuracy estimate).

3. Experiment 1

Experiment 1 integrated an intervention approach used previously (Bailey et al., 2018; Panfil et al., 2020) with SYS and ADJ procedures that were previously compared (Peterson et al., 2015). This experiment was designed to test the hypothesis that the intervention would disproportionately improve LL choices in the SYS procedure where the delays are constant within a session. Timing was also assessed using an abbreviated peak-interval procedure (Kaiser, 2009) to determine whether the FI intervention and/or choice task altered timing of the LL delay. Finally, to ensure that the choice functions observed using the LL delay procedures generalized to other tasks, we tested all rats on a systematic SS delay procedure where the LL delay was constant at 30 s and the SS delay increased across blocks of sessions.

3.1. Design and Procedure

The rats progressed through a series of experimental phases (Figure 2) that evaluated impulsive choice and timing performances. After the rats were trained to lever press for pellets (see section 2.3.1), they were randomly assigned to a fixed-interval intervention (FI, n = 18) or no-delay control (ND, n = 18) condition. Following the intervention, impulsive choice and timing behaviors were evaluated across experimental phases. In the first choice phase, pairs of rats from each intervention group (n = 18) were randomly assigned to receive either an ADJ LL-Delay (n = 18, n = 11/FI intervention, n = 7/ND control) or SYS LL-Delay (n = 18, n = 7/FI intervention, n = 11/ND control) impulsive choice procedure (25 sessions) in a counterbalanced order. However, due to a programming error, one FI rat and one ND rat received their SYS and ADJ tasks in a different order from originally planned. Following impulsive choice testing, all rats were re-assessed in the peak-interval procedure (6 sessions) and then proceeded to the second phase of impulsive-choice testing. In the second impulsive-choice phase the rats were assigned to the alternative choice task (SYS LL-Delay or ADJ LL-Delay, 25 sessions). The rats completed one final peak-interval assessment (6 sessions) and then proceeded to a final impulsive choice procedure (8 sessions). In the final assessment, all rats completed an impulsive choice procedure that varied the SS delays across blocks of sessions (SYS SS-Delay).

Fig. 2.

Experimental design for Experiment 1. The first peak evaluation (Peak 1) was inserted within the intervention training sessions (during LL lever training). FI = fixed interval; ADJ = adjusting; SYS = systematic; LL = larger-later; SS = smaller-sooner

3.1.1. Intervention Procedures

The intervention procedure lasted for 45 sessions for both groups. The interventions were divided into two phases, where each (SS and LL) option was trained successively in isolation. The order of training on the options was counterbalanced. To equate the total number of SS and LL training trials, the SS intervention continued for 15 sessions and the LL intervention continued for 30 sessions. For the FI intervention condition, the SS intervention sessions required the rats to complete a response-initiated FI-5 s reinforcement schedule to earn 1 pellet per trial. The LL intervention sessions required the rats to complete a response-initiated FI-30 s schedule to earn 2 pellets per trial. A 60-s ITI was used during the FI intervention. For the ND control condition, the SS sessions required a FI-0 s (immediate pellet priming, functionally a fixed-ratio 2 contingency) to earn 1 pellet per trial and a 65-s ITI followed. The time not experienced in the delay was added to the ITI to match the inter-reinforcer interval in intervention and control groups. The LL sessions included an FI-0 s contingency to earn two pellets per trial and a 90-s ITI followed. Sessions lasted until 100 food pellets were delivered or 120 min elapsed.

3.1.2. LL-Delay Impulsive Choice Tasks

The first 5 sessions of the LL delay choice assessment, for both the SYS group and the ADJ group, involved a choice between an SS option that delivered 1 pellet after 5 s and a LL option that delivered 2 pellets after 5 s. This established baseline preference for the larger reward magnitude. Following those sessions, the rats worked on either the SYS or ADJ choice procedures. Experimental sessions for the choice procedures included 80 trials composed of 20 four-trial blocks. Sessions ended after 120 min regardless of the number of trials completed. The four-trial blocks started with two forced-choice trials that randomly offered one option (1 SS and 1 LL trial per block) and then two free-choice trials with both options available. The SS option delivered 1 pellet after a 5-s delay for both the ADJ and SYS procedures. The LL option delivered 2 pellets after a delay and the LL delay systematically increased across session-blocks irrespective of choices (SYS Procedure) or dynamically adjusted across trial-blocks based on recent choices (ADJ Procedure). The ITI duration was 60 s in both the SYS and ADJ procedures.

In the ADJ procedure, the LL delay started at 5 s. For the two free-choice trials in a block, if a rat chose both LL options, then in the subsequent block the LL delay was 1 s longer; if a rat chose both SS options, then the LL delay was 1 s shorter; and if the rat chose both the LL and the SS options, then the delay remained the same in the next trial-block. The minimum LL delay was 5 s, and no artificial constraints were placed on the maximum LL delay. The LL delay at the end of a session carried over as the starting LL delay for the next session. These contingencies remained constant for 20 sessions.

In the SYS LL-delay procedure, the LL started at 5 s and increased to 15, 30, and 60 s across 5-session blocks for 20 sessions.

3.1.3. SYS SS-Delay Impulsive Choice Task

At the end of the experiment, the rats were placed in a SS-delay impulsive choice task that was identical to the LL-delay task, except the SS delays increased across blocks of sessions (abbreviated SYS SS-delay procedure; Panfil et al., 2020). In Sessions 1-4, the rats chose between 2 pellets after 30 s (LL) and 1 pellet after 10 s (SS). In each subsequent session, the SS delay was systematically increased (15, 20, 20, 25, and 30 s) while the LL delay was held constant at 30 s. Due to experimenter error, the SS 20-s condition was delivered for 2 sessions.

3.1.4. Peak-Interval Task

Three peak-interval tests were conducted. These peak test trials were always on the LL lever and intermixed with typical LL trials. The peak-interval test sessions involved 70 trials comprised of 60 forced-choice LL training trials and 10 peak trials. Peak trials were distributed within each 7-trial block, with 1 peak trial randomly intermixed with 6 training trials. The FI training trials proceeded exactly like the forced-choice LL trials in the intervention (see Figure 1). During peak test trials, the lever was inserted, a lever press initiated a 90-s delay, and after the delay elapsed, the lever retracted (without the delivery of a pellet), and a 60-s ITI was initiated. Lever presses during peak trials were recorded but did not produce any programmed consequences. Sessions lasted until all trials were completed or 120 min passed.

The first peak-interval test was embedded in the final six sessions of the LL intervention (FI 30 s) and the ND LL control condition (FR 2 contingency leading to 2 pellets). This first peak-interval test evaluated timing abilities prior to experiencing the choice tasks. The second peak-interval test lasted six sessions and followed the initial choice phase. The training trials were identical to the contingencies in the LL intervention peak task in that rats in the ND group experienced an FR 2 and the FI group experienced an FI 30 s. The second peak-interval test measured timing performance of the intervention after experiencing the SYS or ADJ choice contingencies. The third peak-interval test lasted for six sessions and occurred after the second choice procedure. In this test, all rats experienced the FI 30 s (including the ND group rats) to evaluate their timing performance on the 30-s LL delay for all rats (not just the FI intervention group rats).

3.2. Data Analysis

For the impulsive choice models, there were some order effects due to the counterbalancing of the choice procedure. Thus, only the data from the first-choice assessment is included here. A full analysis of the order effects including all the data is provided in the Supplemental Materials (Figures S1 and S3; Table S5). Briefly, the same general patterns were observed in the second SYS choice task, but there was a reduction in the effect sizes that appeared to be due to carryover from exposure to the previous ADJ choice task. The second ADJ choice task showed more LL choices after previous exposure to the SYS choice task. The SYS SS-delay procedure occurred at the end of the experiment for all rats, so all choice data is included in the SYS SS-delay choice model.

For the SYS LL-delay model, only the last three sessions at each delay were included in the model to remove sessions where the rats were adapting to the change in delay parameters (i.e., to capture more stable preferences in the SYS procedure akin to our approach with the ADJ procedure; see supplemental materials for further details). For the SYS LL-delay model, LL choice was predicted as a function of the LL delay and intervention group. Group assignment was a categorical variable and effect coded with two levels (FI and ND). The LL delay was a continuous variable and re-scaled to vary between 0 (5-s LL delay) and 1 (60-s LL delay). We tested the intercept for the regression models at the 5-s delay, where the SS and LL only differed in magnitude, so performance reflected preference for the larger reward. Proportion of LL choices were predicted as a function of SS delay, intervention group assignment, and their interaction. The SS delay and group assignment were scaled/coded using the same approach as the SYS LL model.

For the ADJ model, the last 5 sessions were included in the analysis to capture preference. Adjusted LL delay was predicted as a function of the session, the intervention group assignment, and their interaction. Group assignment was a categorical variable and effect coded with two levels (FI and ND). Session was a continuous variable re-scaled between 0 and 1 (0 corresponds to session 16, 1 corresponds with session 20). If the slope of the session parameter did not differ significantly from 0, then that indicated that the preference stabilized around an adjusted LL delay. See Supplemental Materials (Figure S1) for a depiction of the adjusting delay choices across trial blocks.

For the modeling of the peak interval data, a separate model was fitted to each of the three peak-interval assessments. The first and second peak-interval assessments estimated the model parameter values as a function of choice assessment procedure (SYS vs. ADJ) for the rats in the FI intervention group. The model only fit data from the FI intervention group because data from the ND control group did not conform to a peak function. The third peak assessment, which used FI contingencies for all 36 rats, predicted the model parameter values as a function of the intervention group (FI vs. ND).

3.3. Results and Discussion

The top panel of Figure 3 shows the results from the ADJ task (see Table S1 for model output). There was no difference in the PSE between the FI group (15.5 s) and the ND group (15.9 s) and no change in the PSE across the five sessions, and no Group × Session interaction. Thus, the PSEs were stable in both groups across sessions, and there was no intervention effect on the PSEs.

Fig. 3.

The top panel shows the point of subjective equality (PSE) between the SS and LL delays as a function of session in the ADJ choice procedure. Note that the ND function is jittered for display purposes. The bottom panel shows the proportion of LL choices in the SYS LL-Delay choice procedure as a function of LL delay (s). The error bars are the standard errors of the estimated marginal means from the mixed effects model.

The bottom panel of Figure 3 shows the results from the SYS LL-delay task (see Table S1 for model output). At the 5-s LL delay intercept, there were fewer LL choices in the FI group than ND group, z = −3.86, p < .001, counter to expectations. Proportion of LL choices decreased as a function of LL delay, z = −41.82, p < .001, and the slope was steeper in the ND group than FI group (Group × LL Delay interaction), z = 3.43, p = .001. Because both groups hit the floor at the 60-s delay, the reduced sensitivity to delay in the FI group should not be unambiguously interpreted as a positive intervention effect (i.e., reduced delay discounting) as this was potentially an artifact of the floor effects. A post-hoc test compared proportion LL choices at the 30-s delay, to provide an assessment of group differences at the middle delay, and the FI group made fewer LL choices than the ND group, z = −2.42, p = .016.

Figure 4 shows the results from the SYS SS-Delay procedure (see Table S1 for model output). At the 10-s SS delay intercept, there were more LL choices for the ND group compared to the FI group, z = −2.55, p = .01. Proportion of LL choices increased as the SS delay increased, z = 32.40, p < .001. Replicating the unexpected effect in the SYS LL-delay procedure, the SYS SS-delay procedure showed that the ND control group produced more LL choices than the FI intervention group. A post-hoc test compared proportion LL choices at the 20-s (middle SS) delay and the FI group made fewer LL than the ND group, z = −2.33, p = .020.

Fig. 4.

The results from the SYS SS-Delay impulsive choice procedure showing the proportion LL choices as a function of SS delay. The error bars are the standard errors of the estimated marginal means from the mixed effects model.

The top panel of Figure 5 shows the fitted peak functions for the first peak evaluation (see Table S2 for model output). Fitted peaks only correspond to the FI group that produced data amenable to a peak assessment. The ND group data is shown for visual comparison to confirm that the rats behaved consistent with the schedule contingencies. Because the ND condition was an FI 0-s (i.e., FR 2) contingency, the responses rates dropped off sharply within the first 10 s. We included choice type (SYS or ADJ) in the model to evaluate baseline timing abilities in the two choice groups. The rats that were assigned to receive the ADJ condition first had greater baseline precision in their timing than rats in the SYS condition, t = −2.25, p = .025. No other model parameters differed between groups.

Fig. 5.

The results from three peak interval tasks (from top to bottom in order) for the fixed interval (FI) and no-delay (ND) groups. The FI and ND groups in the first two peak tasks are differentiated by groups that received Adjusting (ADJ) or Systematic (SYS) tasks in the first choice phase. Fitted curves from the non-linear model are provided along with the data points in each figure. There was no fit to the ND condition. The dashed line at 30 s indicates expected time of reinforcement on FI LL trials.

The middle panel of Figure 5 shows the fitted peak functions for the second peak evaluation (following the choice procedures in the first phase reported above). The rats that recently experienced the ADJ condition showed greater precision in their timing than rats in the SYS condition, t = −18.16, p < .001. This was likely due to their pre-existing baseline differences, reported in their first peak evaluation, and not an effect of the choice procedure. However, the SYS group developed greater timing accuracy compared to the ADJ group, t = 8.72, p < .001. The rats that experienced the ADJ condition also showed a greater decline in response rates after the peak, t = −8.98, p < .001. Note that the linear portion of the peak has not been formally mapped onto any specific underlying timing processes, so this difference is difficult to interpret. Finally, response rates at the start of the interval (y-intercept) were greater for rats in the ADJ group compared to the SYS group, t = 5.10, p < .001.

The bottom panel of Figure 5 shows the fitted peak functions for the third peak evaluation when all rats experienced FI 30-s training trials. The peak function for the rats in the FI intervention showed better timing accuracy than rats in the ND group, t = −2.27, p = .023, in that they peaked closer to 30 s. However, there were no group differences in the variance of their timing functions. Compared to rats in the ND group, the rats in FI intervention also had a higher peak response rate, t = 4.12, p < .001, a lower starting response rate, t = −5.08, p < .001, and a greater decline in responding following the peak, t = −2.77, p = .006.

Overall, the rats in the ADJ group did not show an effect of intervention. We predicted that intervention efficacy should be reduced in the ADJ task, so this was consistent with that prediction. However, we found that the rats in the FI group tested with the SYS LL-delay procedure were more impulsive than the ND control, counter to previous findings that the FI intervention promotes self-control. We also observed this effect in a follow-up SYS SS-delay procedure, replicating this effect across choice procedures and training durations. The ADJ group showed better timing precision, but this was also observed in the baseline phase prior to experience with the ADJ contingencies so likely does not represent an effect of the ADJ task experience. The SYS group, however, displayed greater timing accuracy than the ADJ group. This effect emerged after choice testing. It is possible that the reliable delays experienced in the SYS choice task trained the rats to accurately time the 30-s LL delay, whereas the unreliable delays in the ADJ task provided no such advantage (e.g., Cardinal et al., 2002 demonstrating that rats are insensitive to the individual ADJ delays across trials). Compared to the rats in the ND group, the rats in the FI group showed greater timing accuracy, with the ND group overestimating the delay. Given the hypothesized role in the FI intervention improving self-control by improving timing cognition, this was a predicted outcome. However, the improvement in timing is puzzling considering the choice results where the FI intervention group showed poorer self-control than the ND group. These results represent a potential dissociation between the expected effects of the FI intervention on timing and self-control. However, the timing differences should be interpreted with caution given that there were group differences prior to experience with the choice tasks. Experiment 2 implemented a pre-post design to better address this issue.

4. Experiment 2

Experiment 1 resulted in an unexpected outcome where the FI intervention decreased self-controlled choices relative to the ND control. The fact that the “reverse intervention” effect was still observed in the SS Delay procedure at the end of the experiment (~3 months following intervention) suggests that this effect was robust and persistent. A conspicuous difference between Experiment 1 and the prior intervention studies (e.g., Panfil et al., 2020; Smith et al., 2015; Stuebing et al., 2018) was the use of a 5-s rather than a 10-s SS delay. The 5-s SS delay was chosen to mirror the parameters used in the Peterson et al. (2015) study, which had conducted a thorough examination of temporal tracking and choice behavior in SYS and ADJ procedures.

In the context of an impulsive choice task, shorter SS delays generally lead to more impulsive choices (e.g., Bailey et al., 2018; Mazur & Biondi, 2009). The intervention was designed to train rats to accurately and precisely time the SS and LL delays and learn the SS and LL delay contingencies outside of the choice task. However, it is possible that the rats were unintentionally trained to favor the SS option because between the two options it was relatively short. In contrast, the ND control group may have learned that both options provided pellets immediately, but the LL option provided more pellets.

Experiment 2 tested the hypothesis that the short SS delay was responsible for the increase in impulsive choice due to the FI intervention. Rats were exposed to interventions using a 5- or 10-s SS delay (coupled with a 30-s LL intervention delay). An additional purpose for Experiment 2 was to evaluate generalization of intervention effects on impulsive choices by varying the SS delay parameters (5- and 10-s) in the choice procedure. Rats were assigned to either a 5- or 10-s SS delay in the choice procedure and to either a 5- or 10-s SS delay in the intervention procedure (a 2 × 2 factorial design). This manipulation was designed to separate the effects of the SS delay in the choice and intervention tasks as well as to assess generalization of specific delays across tasks. Overall, the design assessed whether the results of Experiment 1 were due to the choice procedure, the intervention procedure, or a combination. If there is generalization between choice and intervention SS delay experiences, then the incongruent groups may show a different pattern of results from the congruent groups. We evaluated the relationship between timing and impulsive choice using the peak procedure to determine whether intervention effectiveness in increasing self-control corresponded with superior timing precision (Smith et al., 2015).

Interpretations of the effects of the FI intervention and ND control were limited because without a pre-intervention assessment of impulsive choice there was no evaluation of improvements in baseline self-control, only differences between groups. Experiment 2 addresses this limitation by including a pre-intervention choice procedure that tests for changes in impulsive choice and permits the matching of intervention group assignment based upon pre-existing levels of impulsive choice. In addition, because the effect sizes of the “reverse intervention” effect were smaller than the typical intervention effect sizes, we increased the sample size to increase our ability to reliably detect effect sizes in the range observed in Experiment 1.

4.1. Design and Procedure

After initial training (see section 2.3.1), the experiment proceeded through four phases (see Figure 6) – pre-intervention choice (25 sessions), intervention (45 sessions; with peak-interval trials for 6 sessions in the LL intervention training), post-intervention choice (25 sessions), and a second peak-interval assessment (6 sessions). Different groups received different SS delays in the intervention (5- or 10-s SS delay) and choice (5- or 10-s SS delay) procedures resulting in a 2 × 2 factorial design. This resulted in the rats having congruent (5-s/5-s, n = 13; 10-s/10-s, n = 12) or incongruent (5-s/10-s, n = 11; 10-s/5-s, n = 12) choice/intervention SS delays, respectively.

Fig. 6.

Experimental design for Experiment 2. The first peak (Peak 1) assessment during the intervention phase.

The first (pre-intervention) impulsive choice test provided a baseline level to compare to the second (post-intervention) test and to ensure matched baseline levels of choice behavior for intervention group assignments. Rats were assigned to intervention groups by rank-order matching based on mean pre-intervention LL choices and slope of the choice function. The process of intervention assignment to ensure that the groups were matched in their pre-intervention choices resulted in a group size imbalance between the 5-s/5-s and 5-s/10-s groups.

4.1.1. Impulsive Choice Task

The first five sessions involved an initial choice evaluation with an LL delay offering two pellets with a duration that was equal to the SS delay value (either 5 or 10 s, depending on the SS choice group). The remaining impulsive choice task lasted for 25 sessions (5 sessions at each LL delay). The impulsive choice procedure followed the general methods (Figure 1), with SS delays leading to 1 pellet and LL delays leading to 2 pellets. The SS group assignment determined the SS delay, half of the rats (n = 24) experienced a 5-s SS delay, and the remaining rats (n = 24) experienced a 10-s SS delay. The SS delay assignment was consistent across both choice phases (pre-/post-intervention) for all rats. The LL delay was systematically manipulated across 5-session blocks. The LL delays (leading to 2 pellets) were scaled relative to the SS delay so that LL:SS = 1:1, 1.5:1, 3:1, 6:1, and 12:1. For the 5-s SS choice task, the LL delays were 5, 7.5, 15, 30, and 60 s. For the 10-s SS choice task, the LL delays were 10, 15, 30, 60, and 120 s. The ITI was 60 s for all groups.

4.1.2. Intervention Procedures

The intervention procedure in Experiment 2 was like Experiment 1 except instead of an FI intervention group and ND control group, there were two FI intervention groups that differed in the SS delay using in training (5 or 10 s). After the pre-intervention impulsive choice phase, the rats were assigned to the 5- or 10-s SS Intervention Delay. The LL Intervention Delay was 30 s in both groups. The FI intervention contingencies were the same as Experiment 1.

4.1.3. Peak-Interval Procedure

The peak-interval task is the same as the procedure used in Experiment 1. There were two peak-interval assessments. The first was embedded in the final 6 LL intervention training sessions and the second occurred after the post-intervention choice procedure and lasted 6 sessions.

4.2. Data Analysis

Impulsive choice was evaluated using two separate models that focused on the 5-s SS Delay Choice group (5-s SS delay; 5, 7.5, 15, 30, and 60-s LL delays) and the 10-s SS Delay Choice group (10-s SS delay; 10, 15, 30, 60, and 120-s LL delays). Both models predicted the proportion of LL choices as a function of the normalized LL Delay (delay ranges between 0 and 1), SS Intervention Delay (5- or 10-s SS delay), Impulsive Choice Phase (pre-intervention or post-intervention), and their interactions. Supplementary Materials shows a full factorial model that compares the SS Choice Delay conditions.

The first peak-interval assessment was missing the first 4 of the 6 sessions of data due to a computer error that corrupted the data files (but did not affect the conduct of the experiment), this assessment had noisy data that resulted in poor model fits; therefore, we only report the second peak-interval results that produced quality functions. The peak-interval assessment estimated the model parameter values as a function of the SS Choice Delay group, the SS Intervention Delay group, and their interaction.

4.3. Results and Discussion

Figure 7 shows the results from the two impulsive choice models (for the 5- and 10-s choice groups) reporting the effects of the Intervention group, Delay, and Pre/Post intervention choice conditions. The 5-s SS Delay Choice model (Figure 7, top panel) compared the congruent (5-s/5-s) and incongruent (5-s/10-s) Choice/Intervention SS-Delay groups (see Table S3 for the model output). The intercept at 5 s was an index of preference for the larger magnitude as both delays were the same. There was a main effect of group at the intercept, where the 5-s/5-s group made more LL choices overall compared to the 5-s/10-s group, z = 2.52, p = .012. There also was a Group × Pre/Post interaction, z = 6.22, p < .001. A post-hoc analysis of the interaction revealed that LL choices at the 5-s delay were increased post-intervention for the 5-s/5-s group, z = 4.61, p < .001, and were reduced for the 5-s/10-s group, z = −4.20, p < .001. Note, however, that the visualization of the data in the figure and the model prediction are mis-aligned in the 5-s/10-s group. This is because 5 rats in this group showed an increase in choices at 7.5 s compared to the 5-s delay. However, because the model treated LL delay as a continuous variable, it fit a continuously increasing function to the data from these animals. Thus, these results should be interpreted with caution. An analysis comparing pre-/post-intervention choices at the 30-s LL delay showed that LL choices decreased for rats in the 5-s/5-s group (z = −5.42, p < .001) and the 5-s/10-s group (z = −10.22, p < .001) post-intervention. Thus, both groups showed decreased self-control at the intermediate delay post-intervention.

Fig. 7.

The results from the impulsive choice task in Experiment 2. The top panel shows proportion of LL choices from the 5-s SS Choice Delay and the bottom panel shows results from the 10-s SS Choice Delay. The insets in each figure depict a magnified version of the data at the y-intercept. The error bars are the standard errors of the estimated marginal means from the mixed effects model. Note that the data points in each figure are jittered for display purposes. In addition, there are different x-axis scales for the top and bottom panels. Legend format is pre-/post intervention: SS Choice Delay/SS Intervention Delay.

LL choices decreased as a function of LL delay (slope), z = −79.60, p < .001. The choice function slope was steeper following the intervention (Pre/Post × LL Delay), z = −7.50, p < .001. In addition, the slope of the choice function was steeper in the 5-s/5-s group overall (Group × LL Delay), z = −6.61, p < .001. Finally, post-intervention, the 5-s/5-s group showed greater sensitivity to delay than the 5-s/10-s group (Group × Pre/Post × LL Delay), z = −3.31, p = .001.

The 10-s SS Delay Choice model (Figure 7, bottom panel) compared the congruent (10-s/10-s) and incongruent (10-s/5-s) Choice/Intervention SS Delay groups. The intercept (10-s LL delay) provided an index of reward preference. There was a Group × Pre/Post interaction, z = −3.85, p < .001. Post-hoc analyses testing the effects of pre-/post- intervention at revealed that the 10-s/10-s group did not significantly change LL choices at the intercept, z = 1.81, p = .070, and the 10-s/5-s group showed fewer LL choices at the intercept following the intervention, z = −3.65, p < .001. An analysis comparing pre-/post-intervention choices at the 60-s LL delay showed that LL choices increased for rats in the 10-s/10-s group (z = 2.20, p = .028) post-intervention, thus demonstrating a positive intervention effect. The 10-s/5-s group did not show any changes pre- versus post-intervention in LL choices at the 60-s LL delay.

LL choices decreased as a function of LL delay, z = −77.92, p < .001 in the 10-s choice task. The 10-s/5-s group displayed an overall steeper slope in their choice function (Group × LL Delay), z = −3.02, p = .003 compared to the 10-s/10-s group. There was no interaction of Pre/Post with LL Delay indicating that the slopes were similar in the 10-s pre- and post-intervention choice tasks.

For the peak-interval results, Figure 8 shows the fitted peak functions for the second peak evaluation (following the post-intervention choice phase). Rats from the 5-s SS Choice Delay (5-s/5-s and 5-s/10-s) had a lower peak response rate compared to rats in the 10-s SS Choice Delay (10-s/5-s and 10-s/10-s), t = −3.35, p = .001. The SS intervention conditions did not affect peak-interval timing and there were no other statistically significant effects on peak timing.

Fig. 8.

Results from the second peak session for Experiment 2. The top panel shows the data for the groups that received the 5-s SS delay choice task, and the bottom panel shows the 10-s SS delay choice task groups. The fitted functions are shown (lines) along with the data points.

Overall, the 5-s/5-s group showed stronger preference for the larger magnitude following the intervention. However, the intervention for the 5-s/5-s group showed a steeper decline in LL choices as a function of LL delay, thus demonstrating increased impulsive choices with increasing delays. This interesting mix of intervention effects may provide insights into the possible consequences of interventions. The congruent 10-s/10-s group did not show improvements at the intercept and did not show changes in sensitivity to increasing LL delays, but the middle LL delay (60 s) did show increased LL choices post-intervention. Thus, the 10-s/10-s group did benefit from the intervention. This modest effect mirrors a previous study where the SYS LL-delay procedure did not produce as robust an effect compared to the SYS SS-delay procedure (Bailey et al., 2018; Experiment 1), so this may be a product of the LL delay procedure. The incongruent groups (5-s/10-s and 10-s/5-s), without exception, showed increased impulsive choices following the intervention, and this effect was observed at both the intercept and as a steeper slope in LL choices as a function of delay.

Given that the 5-s SS Delay intervention groups showed more impulsive choices, we would have predicted that timing accuracy and precision would be worse in those groups. However, there were no differences in timing between any of the groups. The lack of a relationship between impulsive choice and timing might have occurred for two reasons: (1) The peak-interval procedure tested responses on the LL lever, but the intervention may have had stronger effects on SS preferences. Thus, any (unobserved) timing disruptions on the SS lever for the incongruent groups might not have generalized to responses on the LL lever. (2) This experiment did not include an ND control group that explicitly limits delay exposure. Rats in all four groups had 30 sessions of FI 30 intervention training, therefore differences in timing on the LL lever might have faded over extensive training.

5. General Discussion

The goal of these experiments was to test the generalization of the FI time-based intervention to different impulsive choice tasks to shed light on possible underlying mechanisms. It was anticipated that the FI intervention would improve self-control and timing performance with a stronger effect when the impulsive choice task involved systematic delays (SYS) compared to adjusting delays (ADJ). Consistent with our hypothesis, the ADJ procedure failed to show an intervention effect. A power analysis (see Supplementary Materials) indicated that this result was not likely due to an insufficient sample size, but rather may be a true null result. The failure to show an intervention effect on the current ADJ procedure suggests that intervention effects may require rats to track/time specific delays during the choice procedure for the intervention to be maximally effective. However, some caution needs to be applied to the interpretation of the ADJ results because the SYS procedure did not display typical intervention effects under these intervention parameters. Instead, the ND group produced greater LL choices than the FI group on the SYS LL-delay procedure, and this effect was replicated in the SYS SS-delay choice procedure in Experiment 1.

Experiment 2 sought to replicate and explain the unexpected reverse intervention effect in Experiment 1. The procedure tested the 10-s/10-s condition that successfully produced an intervention effect previously (Bailey et al., 2018) and the 5-s/5-s condition used in Experiment 1. The 5-s/5-s group slightly increased LL choices at the 5-s intercept, relative to the pre-test, but decreased LL choices at the longer delays. This suggests that the 5-s/5-s intervention improved preference for the larger reward, but worsened self-control by increasing delay sensitivity. In contrast, the 10-s/10-s intervention had no effect on LL choices at the intercept or on the slope. However, at the intermediate LL delay, the rats made more LL choices post-intervention. Thus, it appears that the 5-s/5-s intervention was more effective at promoting reward preference, whereas the 10-s/10-s intervention was more effective at promoting delay tolerance at longer LL delays.

The pattern seen in the 5-s/5-s condition in Experiment 2 was also apparent in the ND control in Experiment 1. The combined results of the ND control and the 5-s/5-s condition support the idea that exposure to shorter delays during the intervention training (0-5 s) produced a dual impact of increasing magnitude preference at shorter, equal delays and decreasing self-control at longer, differential delays. This may reflect interacting mechanisms of increased reward preference coupled with decreased delay tolerance. The increase in delay sensitivity in the ND and 5-s/5-s groups is consistent with research by Fox (2021), who demonstrated that rats that experienced an ND control become more impulsive compared to maturational controls (no experimental experience). Fox (2021) concluded that experience with immediacy may worsen self-control by increasing shorter delay preference, which may be another form of delay intolerance. Fox (2021) did not include an assessment of reward magnitude preference, so it is not clear whether their ND group may have been more self-controlled when the SS and LL delays were equal, and short, in duration. Nevertheless, their results suggest that preference for short delays could be operating in the current experiments.

Experiment 2 additionally examined the impact of incongruent SS delays between the intervention task and the impulsive choice task to better understand generalization processes. The incongruent interventions reduced LL choices at the intercept and increased delay sensitivity, which collectively translates into more impulsive choices. We had expected that the 5-s/10-s group would show improvements in self-control post-intervention due to the experience with the 10-s SS Intervention Delay. The 10-s SS delay (coupled with the 30-s LL delay) in the intervention should increase delay tolerance better than the 5-s SS delay intervention. Instead, both incongruent interventions impaired self-control. Thus, the results are not easily interpreted in terms of delay tolerance. The results also do not align with the predictions derived from contrast processes, where the subjective value of the SS option would improve between intervention and post-intervention choice in the 5-s/10-s group and worsen in the 10-s/5-s group. However, a general impairment in incongruent conditions could emerge because rats learned the specific delays in the intervention procedure and experienced a generalization decrement in transferring temporal information from the intervention task to the impulsive choice task (e.g., Rayburn-Reeves & Zentall, 2009).

Finally, another key focus motivating these experiments was to determine whether timing processes participated in impulsive choices. The SYS tasks that involve fixed delays were affected by the intervention, whereas the ADJ task was not (Exp 1), consistent with our original hypothesis. This indicates that choice tasks that involve dynamic delays are less sensitive to time-based intervention effects and lends support for generalization of specific temporal information between intervention and choice tasks. Consistent with this idea, the intervention worsened self-control in both incongruent groups (Exp 2). Generalization decrement of timing judgment on the SS lever might explain the absence of any intervention benefits because the rats’ temporal judgments of the delays were inaccurate (e.g., Rayburn-Reeves & Zentall, 2009). Overall, the findings are consistent with the hypothesis that specific temporal learning processes may be involved in the intervention effects on choice tasks.

However, the peak procedure results paint a more complex picture. In Experiment 1, the FI intervention group displayed better timing accuracy, but they did not show better self-control. In Experiment 2, the 10-s/10-s group showed improved LL choices at intermediate delays but did not clearly show corresponding improvements in the peak procedure. Several other studies have also shown an equivocal relationship between timing performance and self-control (Fox et al., 2019; Rung et al., 2018). These dissociations show that accurate and/or precise timing is most likely not a necessary condition for self-control promoting intervention effects. However, it is worth noting that the current results indicate that the interventions did not affect timing of the LL delay. It is possible that timing of the SS delays would better reflect the transference of temporal information between the intervention and choice tasks. This should be a focus for future research.

5.1. Conclusions

The present studies delivered a time-based intervention and demonstrated a role for absolute and relative delay effects on choice which may have operated through multiple mechanisms including delay tolerance, larger reward preference, and temporal generalization processes. Coupled with previous literature, the results suggest a constellation of factors that may affect the efficacy and nature of intervention effects on impulsive choices. The question may be not which mechanisms are responsible, but how much each factor contributes to the intervention’s self-control promoting effects, depending on the parameters and construction of the intervention and choice tasks. Future research should further isolate the involvement of specific mechanisms and evaluate their interactions.

Both the positive (10-s/10-s) and detrimental effects of the interventions in the current studies were associated with effect sizes in the small to medium range. This may, in part, be due to the use of the SYS LL-delay choice task, as noted previously. The relatively small effect sizes could suggest that intervention effects may not produce socially significant, or real-world, benefits (or detriments). However, everyday life involves many opportunities to make impulsive choices and a small shift in self-control could translate into socially significant outcomes over a longer span of time, at least in some domains. For example, daily food choices that result in a small increase or decrease in caloric intake can lead to significant changes in body weight over several weeks. Given the cumulative nature of many everyday choices, further explorations of intervention effects on impulsive choices, even in cases where effect sizes are in the small to medium range, would nevertheless seem to be a fruitful venture for future research.

Highlights.

• Time-based intervention effects were tested with different impulsive choice tasks.

• Interventions may not be as effective in the adjusting-delay choice task.

• We tested shorter and longer smaller-sooner delays in intervention and choice tasks.

• Shorter delays increased magnitude discrimination and delay sensitivity.

• Incongruent delays in intervention and choice tasks increased impulsive choices.