Mechanisms of impulsive choice: II. Time-based interventions to improve self-control

Abstract

Impulsive choice behavior has been proposed as a primary risk factor for other maladaptive behaviors (e.g., gambling, substance abuse). Recent research has suggested that timing processes may play a key role in impulsive choice behavior, and could provide an avenue for altering impulsive choice. Accordingly, the current experiments assessed a set of time-based behavioral interventions to increase self-control while simultaneously assessing effects on timing processes within the impulsive choice task. Three experiments assessed temporal interventions using a differential reinforcement of low rates task (Experiment 1) and exposure to either a variable or fixed interval schedule (Experiments 2–3). The efficacy of the interventions was assessed in Sprague-Dawley (Experiments 1–2) and Lewis (Experiment 3) rat strains. Impulsive choice behavior was assessed by measuring preferences of a smaller-sooner (SS) versus a larger-later (LL) reward, while timing of the SS and LL durations was measured during peak trials within the impulsive choice procedure. The rats showed an increased preference for the LL following all three time-based interventions and also displayed increased temporal precision. These results add to the increasing evidence that supports a possible role for temporal processing in impulsive choice behavior and supply novel behavioral interventions to decrease impulsive behavior.

1. Introduction

Impulsive choice behavior has been defined as choosing a smaller reward available sooner over a larger reward available later when the larger reward is objectively more optimal in terms of reward earning potential; impulsive choice has been identified as a stable trait variable in both humans (Jimura et al. 2011; Kirby 2009; Odum 2011b; Odum and Baumann 2010; Simpson and Vuchinich 2000) and rats (Galtress et al. 2012; Garcia and Kirkpatrick 2013; Marshall et al. 2014). Impulsive choice as a trait has been suggested to play an important role in attention deficit hyperactivity disorder (ADHD; Barkley et al. 2001; Castellanos et al. 2006; Marco et al. 2009; Solanto et al. 2001; Sonuga-Barke et al. 1992; Sonuga-Barke, Wiersema, et al. 2010; Tripp and Alsop 2001), pathological gambling (Alessi and Petry 2003; Dixon et al. 2006; Dixon, Marley, et al. 2003; MacKillop et al. 2011; Reynolds 2006), obesity (Davis et al. 2010; Stoeckel et al. 2013; Weller et al. 2008) and mental disabilities (Dixon et al. 1998). Furthermore, previous research has shown that impulsive choice is a risk factor (e.g., de Wit 2008; MacKillop et al. 2011; Verdejo-García et al. 2008) and a predictor of treatment outcomes (Krishnan-Sarin et al. 2007; Yoon et al. 2007) for drug abuse.

Given that impulsive choice appears to serve as a primary risk factor for other maladaptive behaviors, it is imperative to consider potential methods for improving self-control. However, impulsivity is a broad and multifaceted construct (see Evenden 1999), which makes it difficult to target specific mechanisms. Recurring themes within behavioral research have defined impulsive choice in terms of two broad categories: an inability to inhibit responding and an inability to delay gratification (e.g. Evenden 1999; Lane et al. 2003; Reynolds et al. 2004; Sonuga-Barke 2002).

There is growing evidence to suggest that an inability to delay gratification, possibly due to temporal processing deficits, plays a key role in impulsivity. Impulsive individuals, as measured through delay discounting paradigms, produce more errors on timing tasks than control subjects (Baumann and Odum 2012), and impulsivity measured through psychometric scales is associated with poorer discrimination of temporal intervals (Van den Broek et al. 1992). In addition, real time delay discounting measurements are related to increased errors in time production and estimation tasks (Corvi et al. 2012). Impulsive children with ADHD, measured through computerized delay of gratification tasks, show greater frustration towards unexpected delays (Bitsakou et al. 2009), and adolescents with ADHD make more errors in time reproduction tasks than do controls (Barkley et al. 2001; Smith et al. 2002). Finally, individuals with ADHD display steeper impulsive choice functions than controls (e.g. Barkley et al. 2001; Scheres et al. 2010; Wilson et al. 2011) and place greater value on the immediate access to rewards (Marco et al. 2009; Neef et al. 2001).

Accordingly, Sonuga-Barke et al. (1992) proposed the delay aversion hypothesis which posits that individuals with ADHD are motivated to avoid long, potentially aversive delays that lead to impulsive behaviors. In support of this idea, delay aversion has predictive validity in identifying ADHD cases (Bitsakou et al. 2009; Marco et al. 2009; Solanto et al. 2001). For example, if individuals with ADHD, and perhaps impulsive individuals in general, show an aversion to delays, it is also possible that this may predispose them to have less precise timing. Because delay-averse individuals opt out of delaying gratification, they experience longer delays less often and this could impair their ability to learn to time those delays (see also Galtress et al. 2012; Marshall et al. 2014; McClure et al. 2014). Furthermore, impulsive motor behaviors (i.e., an inability to inhibit responding) and timing deficits have both been proposed as key endophenotypes in ADHD (Castellanos et al. 2006). Therefore, it appears that the temporal processing of delays to reward is a key mechanism underlying impulsive choice and in determining its overall value.

A separate descriptive model regarding the integration of reward amount and delay is delay discounting (DD), which proposes that the subjective value of rewards decreases as a function of the delay until receipt of the reward. DD is often assessed within an impulsive choice task by presenting choices between a smaller-sooner (SS) reward and a larger-later (LL) reward. The SS is usually considered the impulsive choice, while the LL is usually considered the self-controlled choice (see Odum 2011a, for a review). Indeed, the prominent model of DD has incorporated a probable role for temporal processing in determining impulsive choice behavior (Mazur 1987, 2001). The following hyperbolic discounting equation (Equation 1) is designed to predict the subjective value (V) of a reward of a particular amount (A) and delay (D):

$$V=\frac{A}{1+kD}$$ (1)

in which k is the discounting rate and has been used to describe individual differences in impulsive choice behavior (e.g., Odum 2011a; Odum 2011b; Odum and Baumann 2010). Through various attempts to fit the model to data from individual subjects, systematic deviations in model fits have been observed, which has led to the addition of a second free parameter, S, reflecting an individual’s sensitivity to delay (Rachlin 1989):

$$V=\frac{A}{{(1+kD)}^S}$$ (2)

When S is equal to 1, Equations 1 and 2 yield identical results, but Equation 2 (with S values typically below 1) has been shown to fit individual subject data more accurately and reduces systematic variance through altering the relationship between delay and amount when computing subjective value (Green et al. 1994; Myerson and Green 1995; Rodriguez and Logue 1988; Rodzon et al. 2011). This altered discounting equation further supports the idea that delay processing plays an important role in determining overall subjective value, in conjunction with research indicating that models of DD that incorporate subjective time fit behavioral data more accurately than models that do not (Takahashi et al. 2008).

In fact, several studies have indicated a direct connection between impulsive choice and timing. Zauberman, Kim, Malkoc, and Bettman (2009) found that priming human subjects to delays increased attention to delay and decreased discounting rates. These studies corroborate the results of Marshall et al. (2014) and other related studies (Baumann and Odum 2012; Cheng 1992; Galtress et al. 2012; McClure et al. 2014; McGuire and Kable 2012, 2013; Wittmann and Paulus 2008, 2009; Zauberman et al. 2009), indicating an important role of timing processes in impulsive choice behavior. Therefore, future attempts to increase self-control should aim at improving timing processes.

Previous attempts to produce improvements in self-control through temporal processing have most commonly utilized the interval fading procedure, which is designed to produce delay tolerance through exposure to gradually increasing delays to the LL reward (or gradual decreases in delay to the SS reward). This method results in moderate increases in LL preference (Binder et al. 2000; Dixon et al. 1998; Dixon and Holcomb 2000; Dixon, Rehfeldt, et al. 2003; Mazur and Logue 1978; Neef et al. 2001; Schweitzer and Sulzer-Azaroff 1995; Stein et al. 2013). An alternative approach by Eisenberger and Adornetto (1986) examined the effect of exposure to the specific reward amounts and delays from an impulsive choice task by using the delay-amount combinations from the choice task as reinforcers for a variety of other tasks. Children were given additional exposure to the LL delay/reward and this increased self-control, indicating that mere exposure to the LL amount and delay outside of the discounting task promoted increased preference for the LL choice. In addition, an earlier study by Eisenberger et al. (1982) found that exposure to a variable long delay (average = 78 s) maintained self-control levels whereas exposure to a variable short delay (average = 5 s) increased levels of impulsivity. Furthermore, Madden et al. (2011) assessed rats’ baseline levels of impulsivity followed by training on either a mixed delay schedule of short (either .01-s or 20-s) and long (either 20 or 60-s) intervals or a fixed delay that was associated with 50% LL choices from the impulsive choice task. The rats were then returned to an impulsive choice task in which they displayed a significant increase in LL choices. While no further assessment of the rats’ timing was conducted, this study indicates that temporal processes are likely involved in impulsive choice behavior. Importantly, these studies indicate that behavioral interventions can increase self-control. However, none of these prior studies specifically assessed timing processes in conjunction with alterations in impulsive choice behavior.

Accordingly, the goal of the present experiments was to assess the effects of a set of time-based behavioral interventions in rats by testing impulsive choice and timing before and after the implementation of the intervention. Experiment 1 employed a differential reinforcement of low rates (DRL) task with Sprague-Dawley rats, an outbred strain that has been shown to exhibit stable individual differences in impulsive choice (Galtress et al. 2012; Marshall et al. 2014). The DRL procedure requires inhibition of responding for a target duration and is effective in suppressing early responses (Pizzo et al. 2009) which have been proposed to reflect impulsive behaviors within interval timing procedures (Matell and Portugal 2007). The DRL task is reliant on precise timing and leads to sharp peaks in inter-response times that are roughly centered on the criterion time (Pizzo et al. 2009). Experiment 2 delivered fixed interval (FI) and variable interval (VI) schedules of reinforcement to Sprague-Dawley rats. These tasks require less inhibition of responding, as responding early only results in wasted effort rather than resulting in increased delays to reward, but the tasks are heavily linked with interval timing (Church et al. 1998). The VI schedule was included as a comparison condition because previous interval exposure studies have used both fixed and variable delays (Eisenberger et al. 1982; Madden et al. 2011), but have not determined the relative efficacy of the two schedules in affecting choice and/or timing behaviors. The direct comparison of FI and VI schedules assessed whether any intervention effect was reliant on exposure to the specific intervals from the impulsive choice task (the FI intervention), or if exposure to the range of intervals around (and including) the SS and LL delays would also produce an intervention effect (the VI intervention). Finally, Experiment 3 utilized the timing interventions from Experiment 2 in Lewis rats, which have been previously shown to exhibit deficits in self-control (Anderson and Diller 2010; Anderson and Woolverton 2005; García-Lecumberri et al. 2011; Garcia and Kirkpatrick 2013; Huskinson et al. 2012; Madden et al. 2008; Stein et al. 2012), to determine whether the behavioral interventions could mitigate more severe impairments in self-control.

2. Experiment 1

2.1. Material and Methods

2.1.1. Animals

Twenty-four male experimentally naïve Sprague-Dawley rats were used in this experiment. The rats were approximately 11 weeks old at the onset of behavioral testing. At 10 weeks of age the rats had a mean ad libitum weight of 298 g (range = 267–331 g) and were placed on a restricted diet of standard lab chow to approximately 85% of their free feeding weight for their target age; their feeding was adjusted to maintain this weight for the remainder of the study. The rats were pair-housed in clear plastic shoebox cages with free access to water on a reverse 12 hr light-dark cycle. All behavioral testing was carried out during the dark portion of the cycle.

2.1.2. Apparatus

All phases of the experiment were conducted using 24 operant chambers (Med Associates, St. Albans, VT). The chambers measured approximately 25 × 30 × 30 cm, and were also housed within a ventilated noise reducing isolation box measuring approximately 74 × 38 × 60 cm. The chambers contained two retractable levers, two nose poke keys with cue lights, a houselight, a food cup, and a water bottle. The houselight was positioned in the top-center of the back wall. The two levers (ENV-122CM) were situated on either side of the food cup at approximately one third of the total height of the chamber. The nose poke keys with cue lights (ENV-119M-1) were located directly above each lever. A magazine pellet dispenser (ENV-203) delivered 45-mg food pellets (Bio-Serv, Frenchtown, NJ) into the food cup. Head entries into the food cup were recorded by an LED-photocell. The water bottle was mounted outside of the chamber directly opposite the food cup, and water was available through a metal tube that ran through a hole in the lower-center of the back wall. Med-PC IV controlled the experiment and recorded the time of events with a 2-ms resolution (Tatham and Zurn 1989).

2.1.3. Procedure

2.1.3.1. Pre-training

All rats received one session of pre-training prior to the start of the experiment, consisting of four blocks. There was a 1-min interval prior to the start of the first block and a 1-min inter-block interval. The first block (magazine training) consisted of a random time 60-s schedule that delivered a total of 30 food pellets. The second block involved initial lever press training and was composed of four sub-blocks within which the rats received a one-pellet food delivery for each lever press. Both left and right levers were trained independently in alternating blocks for a total of 10 food deliveries per sub-block. In the remaining two blocks, food deliveries occurred on a random ratio schedule; both levers were inserted simultaneously, with a mean of 3 and 5 lever presses required for each food delivery, respectively, and with 10 food deliveries (5 per lever) per sub-block. There was a 30-s black out period in which the levers retracted between sub-blocks in all cases.

2.1.3.2. Impulsive choice task, pre-intervention

Following pre-training, all rats experienced an impulsive choice task (Garcia and Kirkpatrick 2013; Marshall et al. 2014). Each session consisted of free choice, forced choice, and peak trials. On free choice trials, both levers were inserted into the chamber. The rat made a choice by pressing one of the levers, after which the cue light above the chosen lever was lit and the alternative lever retracted. After the criterion delay elapsed, the next lever press on the chosen lever resulted in the offset of the light and lever retraction. The SS lever delivered 1 pellet of food after 10 s and the LL lever delivered 2 pellets of food after 30 s, with lever assignments counterbalanced across rats. Forced choice trials followed the same structure as free choice trials except that only one lever was inserted into the chamber. Peak trials were identical to forced choice trials, but the lever was inserted for 90 s. Lever presses were recorded but had no consequence, and the trial ended without reinforcement. After each trial, there was a 120-s intertrial interval (ITI), during which the levers were retracted and the chamber was dark.

Trials were delivered in blocks containing 15 free choice, 8 forced choice (4 SS and 4 LL), and 2 peak (1 SS and 1 LL) trials that were randomly intermixed and blocks were delivered continuously within the session (without any inter-block interval). Each session ended after 180 trials were completed, 240 reinforcers were earned, or 6 hr had elapsed. The impulsive choice task continued until choice behavior was stable for 10 consecutive sessions, lasting for a total of 24 sessions.

2.1.3.3. DRL intervention

Following the impulsive choice task, the rats were divided into three groups: Group 10, Group 30 and Group 10/30, with each group matched for percentage LL choice during the baseline impulsive choice task. Each group was trained on the DRL schedule, but with different DRL criteria to match the SS and/or LL delays from the impulsive choice task. Group 10 was trained on the 10-s criterion, Group 30 was trained on the 30-s criterion, and Group 10/30 was trained on both criteria with the 10- and 30-s criteria delivered in alternating sessions. Each session began with the insertion of one lever associated with either the 10- or 30-s criterion. The first response resulted in the onset of the DRL duration. If a response was made before the criterion had elapsed, the interval reset. The first response made after the criterion had elapsed resulted in the delivery of 1 pellet for the short criterion (10 s) and 2 pellets for the long criterion (30 s) and a reset of the DRL criterion duration. The lever assignments for the DRL task were the same as in the choice task such that the 10-s DRL was delivered on the SS lever and the 30-s DRL on the LL lever.

Sessions lasted until 240 food pellets were delivered or 6 hr elapsed. Group 10 and Group 30 received training for 10 sessions, and Group 10/30 received 20 sessions of training, with 10 sessions each for the 10-s and 30-s criteria.

2.1.3.4. Impulsive choice task, post-intervention

Following DRL training, each group returned to the impulsive choice task, which was identical to that of the pre-intervention task. This phase lasted for a total of 12 total sessions.

2.1.4. Data analysis

Choice behavior during free choice trials was analyzed over the last five sessions of pre-intervention training (PRE), sessions 2–6 of post-intervention testing (POST_E), and the last five sessions of post-intervention testing (POST_L) using an empirical log odds ratio (Equation 3). The first session of post-intervention testing was not included in the POST_E analysis because the rats were re-adjusting to the choice task after a long period of single lever exposure during the intervention phase.

$$\mathit{LogOdds}=log\left(\frac{N_{LL}+.5}{N_{SS}+.5}\right)$$ (3)

in which log is the natural logarithm, N_LL is the number of LL choices, and N_SS is the number of SS choices. The log odds ratio provides an index of the strength of LL choices that is less susceptible to ceiling or floor effects, more sensitive to smaller changes in choice preference, and more amenable to ANOVA than percentage choice measures. A value of .5 was added to the numerator and denominator to avoid computational problems encountered with exclusive preference of SS or LL options (Garcia and Kirkpatrick 2013; Haldane 1956; Marshall et al. 2014).

To assess the stability of choice behavior between the pre- and post-intervention tests, a test-retest reliability analysis was conducted on the log odds of LL choices for individual rats in the pre- versus post-intervention (both early and late) tests using a Pearson’s correlation coefficient.

In addition, responding during peak trials was analyzed to examine timing behavior during the impulsive choice task. For each rat, responses per minute in 1-s bins during peak trials was determined for the pre- and post-intervention phases. To characterize these peak functions, a modified Gaussian distribution was fit to each SS and LL peak response gradient pre- and post-intervention (see Guilhardi et al. 2007, for a similar technique). The fitted function took the following form:

$$r+A\phi (\mu ,\sigma )$$ (4)

in which φ(μ, σ) was a Gaussian probability density function with a mean of μ and a standard deviation of σ, r was the baseline (or operant) level of responding, and A was a scaling parameter for the Gaussian function to adjust for differences in response rates. A goodness-of-fit measure (omega-squared, ω²) was computed to evaluate the adequacy of each fit. The fitting of Equation 4 was conducted using nonlinear fitting tools in MATLAB (The MathWorks; Natick, MA). Four dependent measures were derived from these fits: (1) the time at which responding reached its maximum rate (peak time, μ), (2) the spread of the peak (σ), (3) the rate of responding at the peak time (peak rate, R, which was the value of Equation 4 at the peak time, μ), and (4) the coefficient of variation (CV; σ/μ). Response rate analyses were averaged over the last 5 sessions of the pre- and post- intervention choice data.

Statistical analyses were conducted in SPSS. Test statistics are only provided for significant comparisons, with the criterion α= .05.

2.2. Results

2.2.1. Impulsive choice task

2.2.1.1. Choice behavior

Figure 1A displays the results from the pre- and post-intervention (early, POST_E and late, POST_L) impulsive choice task, collapsed across groups. LL choice behavior increased following the intervention and was similar in both the early and late stages of post-intervention testing, indicative of an increase in self-control. Separate ANOVAs were conducted to compare the intervention effects for the early and late post-intervention tests with the variables of Pre/Post and Group. For the POST_E tests, there was a Pre/Post main effect, F(1, 21) = 6.00, p = .023, η_p² = .222, and no main effect of Group or Group x Pre/Post interaction, Fs(1, 21) < 1. For the POST_L tests, there was a Pre/Post main effect, F(1, 21) = 5.53, p = .028, η_p² = .209, and no main effect of Group or Group × Pre/Post interaction, Fs(1, 21) < 1.

Figure 1.

A: Experiment 1 mean (± standard error of the mean, SEM) log odds of LL choices collapsed across groups during the impulsive choice task. PRE = pre-intervention; POST_E = post-intervention early sessions; POST_L = post-intervention late sessions. B: Log odds of LL choices in the pre- versus post-intervention (early and late) tests for individual rats. The dashed lines are the best fitting regression lines through the data and the solid line represents the unity relationship as a comparison.

In addition to examining group differences in choice behavior, we also conducted a test-retest reliability analysis on the choice behavior of individual rats to determine whether the rats were generally stable in their choice behavior between the pre- and post-intervention early and late assessments (Figure 1B). In general, the rats with lower LL choices pre-intervention showed the largest improvements in LL choices, but these rats still made fewer LL choices compared to rats with high pre-intervention LL choices. Accordingly, there was a strong positive correlation in individual differences in log odds of LL choices pre-versus post-intervention both early, r = .94, p < .001 and late, r = .88, p < .001, in the post-intervention testing.

2.2.1.2. Timing behavior

The peak trials delivered during the impulsive choice task in the pre- and post-intervention phases were analyzed to assess any effects of the DRL intervention on timing (Figure 2). The rats timed the durations accurately, with peaks near 10 s on the SS trials and near 30 s on the LL trials. The post-intervention response functions showed a higher peak rate and a narrower function, indicating increased temporal precision.

Figure 2.

Experiment 1 responses per minute as a function of time during smaller-sooner (SS) and larger-later (LL) peak trials in the pre- and post-intervention impulsive choice task, collapsed across groups.

A modified Gaussian distribution was fit to the individual pre- and post-intervention SS and LL peak data (Equation 4). This function accounted for a mean of 92% of the variance in the peak functions. Figure 3 shows the mean peak time (A), peak rate (B), peak standard deviation (STD; C), and peak coefficient of variation (CV; D) for the pre- and post-intervention responding during the SS and LL peak trials. Relative to the pre-intervention testing, the post-intervention peak rates increased, and the standard deviations and CVs decreased; there were no apparent changes in peak time. Separate 2 × 2 ANOVAs were conducted on each measure with the variables of Lever and Pre/Post. For peak times, there was a main effect of Lever, F(1, 23) = 1094.98, p < .001, η_p² = .979, due to earlier peak times on the SS lever. An analysis of peak rates indicated a main effect of Pre/Post, F(1, 23) = 22.26, p < .001, η_p² = .492, that was due to an increase in post-intervention peak rate. Analysis of the peak standard deviation revealed main effects of Lever, F(1, 23) = 127.26, p < .001, η_p² = .847, and Pre/Post, F(1, 23) = 6.22, p = .020, η_p² = .213. There were lower standard deviations associated with the SS lever, and with the post-intervention testing on both levers. Lastly, the analysis of peak CV revealed main effects of Lever, F(1, 23) = 40.38, p < .001, η_p² = .637, and Pre/Post, F(1, 23) = 4.68, p = .041, η_p² = .169. There were lower CVs associated with the LL lever and with the post-intervention testing on both levers.

Figure 3.

Experiment 1 mean (± SEM) peak time (A), peak rate (B), and peak standard deviation (STD, C), and peak coefficient of variation (CV, D) obtained from the Gaussian fitted functions on SS and LL peak trials pre- and post-intervention.

2.2.2. DRL intervention task

We conducted systematic analyses of DRL efficiency and inter-response times that disclosed the typical patterns of responding for DRL 10 and 30 tasks (Pizzo et al. 2009), providing confidence that the rats had learned the DRL task to a high level of proficiency. These results are not shown for the sake of brevity.

2.3. Discussion

The DRL intervention produced stable increases in LL choices that were apparent early and were maintained during post-intervention testing (Figure 1), while simultaneously increasing temporal precision on peak trials. The increased precision was shown through reduced variability and increased rates of responding around the expected time of food delivery in the impulsive choice task (Figures 2–3). To our knowledge, this is the first study to demonstrate increases in both self-control and timing precision as a result of a time-based intervention. The effect of the DRL intervention on timing within the choice task is interesting because the DRL contingency operated on timing of IRTs rather than on timing of external cues. The co-occurrence of effects on timing and choice measures as a function of the intervention suggests that shared processes may be influencing timing precision and self-control, consistent with other recent research (Marshall et al. 2014; McClure et al. 2014). While impulsive choice has been previously mitigated with delay exposure techniques (Eisenberger et al. 1982; Madden et al. 2011; Stein et al. 2013), increasing the salience of delays (Zauberman et al. 2009), and exposure to both reward amounts and delays (Eisenberger and Adornetto 1986), the previous studies did not examine changes in timing behavior in conjunction with monitoring choice behavior.

The results are consistent with the delay aversion hypothesis, which proposes that impulsive choices are driven by an aversion to delays (Sonuga-Barke et al. 1992). Delay aversion could possibly emerge through deficits in timing processes, as variability in timing correlates with delay aversion and impulsive choice behavior (Marshall et al. 2014; McClure et al. 2014). The present findings suggest a relationship between timing precision and impulsive choice behavior because the DRL intervention reduced variability in timing while also increasing self-control. The reduction in variability in timing processes could potentially provide a means for reducing delay aversion. However, the DRL induces both inhibitory and timing processes (Monterosso and Ainslie 1999; Pizzo et al. 2009), and it is possible that improvements in inhibitory processes may have contributed to the changes in choice and timing behaviors. Thus, Experiment 2 involved the delivery of FI and VI schedules, which do not explicitly require response inhibition.

3. Experiment 2

3.1. Material and Methods

3.1.1. Animals

Twenty-four male experimentally naïve Sprague-Dawley rats were used in this experiment. At approximately 10 weeks of age, the rats were placed on a restricted diet of standard lab chow to obtain approximately 85% of their target weight adjusted for age; their feeding was adjusted throughout the study to maintain weights. The rats’ average ad lib weight was 309 g (281 – 346) at the beginning of experimental testing at 11 weeks of age. The housing and husbandry conditions were identical to those described in Experiment 1.

3.1.2. Apparatus

The apparatuses were identical to Experiment 1.

3.1.3. Procedure

3.1.3.1. Pretraining

The rats underwent three sessions of pretraining prior to the beginning of the experiment using the same procedures as in Experiment 1, except that lever and magazine training took place separately. There was one session of magazine training and two sessions of lever training. Training sessions lasted approximately 2 hr and were limited to 120 reinforcers.

3.1.3.2. Impulsive choice task, pre-intervention

Following pre-training, all rats experienced an impulsive choice task similar to Experiment 1. Each session lasted approximately 2 hr and consisted of a maximum of 78 trials comprised of 48 free choice, 24 forced choice (12 SS and 12 LL), and 6 peak (3 SS and 3 LL) trials delivered in three 26-trial blocks. The LL outcome was a 30-s delay for 2 pellets of food, while the SS was 1 pellet with delays of 5, 10 and 20 s that varied across three phases, each lasting ten sessions. The ITI was 60 s.

3.1.3.3. FI and VI schedules

After the impulsive choice task, the rats were split into two groups, FI and VI, matched for baseline LL choice behavior across the three initial phases. Each group performed a free operant FI or VI schedule with FI delays of 10 s, corresponding to the SS lever, and 30 s, corresponding to the LL lever. Delays on the VI schedule were uniformly distributed and averaged 10 s (range = 0–20 s) and 30 s (range = 0–60 s). The first lever press made following the delay criterion resulted in reinforcement and early responses had no consequence.

The rats received one FI or VI schedule on one lever each session, with the 10- and 30-s intervals alternating across sessions. Each session involved 150 SS or 75 LL trials delivered in 3 blocks (50 SS or 25 LL trials) with a 15-min interblock interval, lasting approximately 1 hr. The levers alternated between sessions for 30 sessions (15 per lever). However, due to potential generalization across delays, the design subsequently changed from alternating levers between sessions to a blocked session design in which each rat performed on either the SS or the LL for 20 total sessions, followed by the alternate lever presented for an additional 20 sessions of training. Here, sessions lasted for 210 total reinforcers (210 SS trials or 105 LL trials) or 1.5 hr. Training lasted for a total of 70 sessions.

3.1.3.4. Impulsive choice task, post-intervention

Following the intervention, the rats returned to the impulsive choice task, which was delivered in the same manner as the pre-intervention task. Each of three phases lasted for 10 sessions and delivered SS delays of 5, 10, and 20 s.

3.1.4. Data analysis

Choice behavior during free choice trials was analyzed over the last five sessions of each condition during the pre- and post-intervention tests using a log odds ratio (Equation 3). To assess the stability of choice behavior both within and between tasks, reliability and correlational analyses were conducted. Within-task (internal) reliability was assessed using a Cronbach’s alpha (α) coefficient to determine the stability of individual differences in choice and behavior across phases (conducted separately within the pre- and post-intervention tasks). Test-retest reliability was assessed using a Pearson’s correlation coefficient on the mean pre- and post-intervention log odds of LL choices.

In addition, peak trials during the last five sessions of each condition before and after the intervention were analyzed to examine timing behavior during the impulsive choice task. The individual rat’s SS and LL peak trials for each phase were fit with the modified Gaussian function (Equation 4). Three rats, all from Group FI, were removed from peak-trial analysis due to a failure to resolve Gaussian fits with parameters in a meaningful range.

3.2. Results

3.2.1. Impulsive choice task

3.2.1.1. Choice behavior

Separate 2 (Pre/Post) × 3 (SS Delay) × 2 (Group) mixed ANOVAs were conducted to assess changes in choice behavior between pre- and post-intervention tests for early and late sessions in the post-intervention period. The effects of the intervention in the early sessions (sessions 2–6) of each phase of post-intervention testing are shown in Figure 4A. The ANOVA revealed significant effects of Pre/Post, F(1, 22) = 14.20, p < .001, η_p² = .392, SS Delay, F(2, 44) = 144.17, p < .001, η_p² = .868, and a Pre/Post × SS Delay interaction, F(2, 44) = 12.66, p < .001, η_p² = 365. Post-hoc analyses of the interaction revealed a significant Pre/Post effect at SS delays of 5 s, t(23) = 5.25, p < .001, and 10 s, t(23) = 4.18, p < .001, but not at the SS delay of 20 s. As illustrated in Figure 4C for the late post-intervention testing, there was a similar pattern of effects to the early testing. Specifically, there were significant effects of Pre/Post, F(1, 22) = 25.16, p < .001, η_p² = .534, SS Delay, F(2, 44) = 171.61, p < .001, η_p² = .886, and a Pre/Post × SS Delay interaction, F(2, 44) = 5.74, p = .006, η_p² = .207. Post-hoc analyses of the interaction revealed a significant Pre/Post effect at SS delays of 5 s, t(23) = 4.68, p < .001, and 10 s, t(23) = 4.98, p < .001, but only a marginal effect at the SS delay of 20 s, t(23) = 2.06, p = .051.

Figure 4.

A: Experiment 2 mean (± SEM) log odds of LL choices for Groups FI and VI during the pre- and early post-intervention impulsive choice tasks. B: Log odds of LL choices in the pre- versus early post-intervention tests for individual rats. The dashed line is the best fitting regression line through the data and the solid line represents the unity relationship as a comparison. C: Mean (± SEM) log odds of LL choices the pre- and late post-intervention impulsive choice tasks. D: Log odds of LL choices in the pre- versus late post-intervention tests.

Given that impulsive choice was assessed at multiple choice points, we conducted assessments of internal reliability (see Section 3.1.4) to determine the stability in individual differences in the pre- versus post-intervention impulsive choice tasks. The reliability assessment of choice behavior revealed α = .85, α= .83, and α =.84 for the PRE, POST_E and POST_L testing, respectively. The test-retest reliability between the pre- and the early and late post-intervention assessments are shown in Figure 4B and 4D, respectively. In general, the rats with lower pre-intervention LL choices showed the largest improvements in LL choices, and there was a positive correlation in individual differences in log odds LL choices pre-versus early post-intervention, r = .62, p = .001 and late post-intervention, r = .55, p = .005.

3.2.1.2. Timing behavior

Figure 5 displays the peak response functions for the rats in Groups FI (left column) and VI (right column) for each SS delay condition during pre- and post-intervention testing. Generally, the functions peaked near the expected time of SS and LL food delivery in each of the three phases. Furthermore, there was a tendency for maximum response rates at the expected time of food to increase following the intervention, and for the standard deviation of the peak functions to decrease, similar to Experiment 1.

Figure 5.

Experiment 2 responses per minute as a function of time during smaller-sooner (SS) and larger-later (LL) peak trials for each SS delay (in the pre- and post-intervention impulsive choice task in the FI (A–C) and VI (D–F) groups.

Equation 4 was fit to the individual SS and LL peak interval functions for each phase pre-and post-intervention, and these fits accounted for a mean of 82% of the variance in the functions. Figure 6 shows the mean peak times (A and E), rates (B and F), standard deviations (C and G), and CVs (D and H) for Groups FI and VI in the pre- and post-intervention tests. Similar to Experiment 1, there were decreases in peak STD and peak CV and increases in peak rate between the pre- and post-intervention tests. A 2 (Lever) × 2 (Pre/Post) × 3 (SS Delay) × 2 (Group) mixed ANOVA was conducted on the peak time, rate, standard deviation, and CV measures.

Figure 6.

Experiment 2 mean (± SEM) peak time (A and E), peak rate (B and F), peak standard deviation (STD, C and G), and peak coefficient of variation (CV, D and H) obtained from the Gaussian fitted functions on SS (A–D) and LL (E–H) peak trials delivered to Groups FI and VI pre- and post-intervention. Note the different scaling of the ordinate axis in the peak time and standard deviation graphs for the SS and LL levers.

The analysis of peak times (Figure 6A and 6E) revealed significant effects of Lever, F(1, 19) = 393.06, p < .001, η_p² = .954, SS Delay, F(2, 38) = 71.57, p < .001, η_p² = .790, and a Lever × SS Delay interaction, F(2, 38) = 115.72, p < .001, η_p² = .859. The interaction was due to the rats changing their peak times to adjust to the changes in SS delay, whereas LL peak times remained stable. There were no effects of Pre/Post or Group on peak times.

The analysis of peak rates (Figure 6B and 6F) revealed significant main effects of Pre/Post, F(1, 19) = 25.58, p < .001, η_p² = .574, SS Delay, F(2, 38) = 8.45, p = .001, η_p² = .308, and a Pre/Post × SS Delay interaction, F(2, 38) = 4.21, p = .022, η_p² = .181. The interaction was due to the rats increasing their pre-intervention peak rates across levers as a function of SS delay, while post-intervention peak rates across levers were relatively stable as a function of SS delay.

The analysis of the peak standard deviations (Figure 6C and 6G) revealed significant main effects of Lever, F(1, 19) = 136.70, p < .001, η_p² = .878, Pre/Post, F(1, 19) = 34.09, p < .001, η_p² = .642, and SS Delay, F(2, 38) = 8.50, p = .001, η_p² = .309. There also were Lever × Pre/Post, F(1, 19) = 8.68, p = .008, η_p² = .313, and Lever × SS Delay, F(2, 38) = 59.68, p < .001, η_p² = .759 interactions. Post-hoc analyses of the Lever × Pre/Post interaction indicated a significant intervention effect on both levers, but this effect was larger for the SS lever. The Lever × SS Delay interaction was driven by increasing standard deviations on the SS lever, due to the increases in its delay, accompanied by decreasing standard deviations on the LL lever across SS delays, indicating greater temporal precision across phases.

Lastly, the analysis of peak CVs (Figure 6D and 6H) revealed main effects of Lever, F(1, 19) = 13.08, p = .002, η_p² = .408, Pre/Post, F(1, 19) = 12.97, p = .002, η_p² = .406, and SS Delay, F(2, 38) = 5.42, p = .008, η_p² = .222. The effect of Lever was due to lower CVs on the LL lever. The effect of Pre/Post was due to lower CVs during post-intervention testing. The effect of SS delay was due to a significantly lower CV during the 20-s SS-Delay phase relative to the 5-s SS-Delay phase, p < .05. There were no other main effects or interactions.

3.2.2. FI and VI schedules

A systematic analysis was conducted on the response rate functions during the FI and VI schedule exposure phase. This revealed that the rats developed both FI scallops that were steeper for the FI 10 s than for the FI 30 s and broader ramping functions for the VI schedules than the FI schedules. These behavioral patterns are typical of standard FI and VI performance, as reported in previously published studies (Church et al. 1998). These data are not shown here for the sake of brevity.

3.3. Discussion

Both FI and VI schedules produced an increase in self-control, similar to Madden et al. (2011) and these effects were apparent early in the post-intervention testing phase and were maintained through to the end of post-intervention testing. The increases in self-control were accompanied by increases in peak rate and decreases in peak spread and peak CV in timing both the SS and LL delays within the impulsive choice task. In addition, as the rats switched over to choosing the LL more often, their peak spread decreased, which most likely reflects the contribution of increased exposure/learning of the LL delay. The FI and VI interventions differed from the DRL task in that there was no externally imposed requirement for inhibition of responding, although these tasks may still involve some inhibitory processes, as early responses have been argued to reflect impulsivity (Matell and Portugal 2007). The successful intervention effects suggest that the intervention may be operating on time-based processes to improve self-control, but it is possible that inhibitory processes may also play a secondary role. This experiment advances our understanding of the potential involvement of timing processes in impulsive choice and provides further support for the delay aversion hypothesis (Sonuga-Barke et al. 1992), as improvements in self-control were found in the absence of externally-imposed inhibitory or reward salience factors.

Interestingly, the VI exposure produced similar changes in LL choices and timing of SS and LL delays as shown in the FI group. One might have expected superior intervention effects from the FI exposure, which provided experience of the specific SS and LL delays, but this did not appear to be the case. One factor that may be important is that the rats received a uniformly-distributed VI, which is associated with an increasing hazard function (Evans et al. 2000). The hazard function expresses the conditional probability of receiving food given that food has not been received yet. The hazard function for a fixed interval is a spike at the time of food availability, but the hazard function for a uniform distribution is a gradually ramping function. As a result, the conditional probability of reward on the VI increases gradually as a function of time, and this may have effectively reinforced waiting behavior to a similar (or even slightly better) degree than did the FI. Rats do appear to time ramping hazard functions, demonstrating broader peaks in responding on uniform VI schedules in comparison to FI schedules (Church et al. 1998), and broad ramping functions were observed in the present study during the intervention phase. Thus, it is possible that an effective intervention needs simply to provide a hazard function that reinforces waiting behavior to promote self-control. Further research should examine this possibility by using an exponentially-distributed random interval schedule as a comparison. Random intervals have a constant hazard function (Evans et al. 2000) and thus should not produce any intervention effects on either timing or self-control behavior.

The overall results support the idea that increased temporal precision and/or increased experience with delays may have resulted in increased self-control following the intervention, which has important implications for the reduction of maladaptive behaviors and replicates the findings of Experiment 1. In conjunction with Experiment 1, the present experiment demonstrated that timing processes could provide a potential route to increased self-control in Sprague-Dawley rats, which may be regarded as a ‘normal’ population sample. Accordingly, a relevant extension of the previous studies would be to test the intervention in a targeted sample population with known deficits in self-control. Therefore, Experiment 3 tested the FI and VI interventions with the Lewis strain of rats.

4. Experiment 3

4.1. Material and Methods

4.1.1. Animals

Twenty-four male experimentally-naïve Lewis rats were used this experiment. At approximately 8 weeks of age, the rats were placed on a restricted diet of standard lab chow and maintained at approximately 85% of their target weight, corrected for age. The rats average ad lib weight was 231 g (210 – 252) at 9 weeks of age. The housing and husbandry conditions were identical to those of Experiments 1 and 2.

4.1.2. Apparatus

The apparatuses were the same as in Experiments 1 and 2.

4.1.3. Procedure

All aspects of the procedure were the same as Experiment 2 except that the intervention was delivered in 20-session blocks of exposure to the SS and LL durations, for a total of 40 sessions of intervention training.

4.1.4. Data analysis

The current experiment used the same data analysis methods as Experiment 2. Three rats (two from Group FI, one from Group VI) were removed from the peak-trial analysis due to a failure to resolve Gaussian fits with parameters within a meaningful range.

4.2. Results

4.2.1. Impulsive choice task

4.2.1.1. Choice behavior

Figure 7A and 7C displays the pre- and post-intervention (early and late) choice behavior as a function of SS delay. The Lewis rats displayed generally low LL choice behavior, even in the 20-s SS delay condition, in which the LL was associated with a higher rate of reinforcement. In contrast to Experiment 2, the intervention effect was smaller and isolated to the first phase of testing with the 5-s SS both early and late in the post-intervention testing. A 2 (pre/post) × 3 (SS delay) × 2 (group) mixed ANOVA was conducted on the early intervention testing and yielded significant effects of SS delay, F(2, 44) = 63.73, p < .001, η_p² = .743, Pre/Post × Group, F(1, 22) = 4.93, p = .037, η_p² = .183, and Pre/Post × SS Delay, F(2, 44) = 32.34, p < .001, η_p² = 595. Post-hoc tests on the Pre/Post × Group interaction indicated that the Group VI displayed a stronger intervention effect than the FI group, but neither the FI nor VI intervention effect was significant overall. Post-hoc tests on the Pre/Post × SS Delay interaction revealed a significant increase in LL choices at the 5-s SS delay (p < .001), but no effect at the 10-s delay and a significant decrease in LL choices at 20-s SS delay (p < .001). A similar analysis conducted on the late post-intervention testing revealed significant effects of SS delay, F(2, 44) = 95.92, p < .001, η_p² = .813, Pre/Post × Group, F(1, 22) = 5.65, p = .027, η_p² = .204, and Pre/Post × SS Delay, F(2, 44) = 5.02, p = .011, η_p² = .186. Post-hoc tests on the Pre/Post × Group interaction indicated that the Group VI displayed a stronger intervention effect than the FI group, but neither the FI nor VI intervention effect was significant overall. Post-hoc tests on the Pre/Post × SS Delay interaction revealed a significant increase in LL choices at the 5-s SS delay (p = .038), but no effect at the 10- or 20-s SS delays.

Figure 7.

A: Experiment 3 mean (± SEM) log odds of LL choices for Groups FI and VI during the pre- and early post-intervention impulsive choice tasks. B: Log odds of LL choices in the pre- versus early post-intervention tests for individual rats. The dashed line is the best fitting regression line through the data and the solid line represents the unity relationship as a comparison. C: Mean (± SEM) log odds of LL choices in the pre- and late post-intervention impulsive choice tasks. D: Log odds of LL choices in the pre- versus late post-intervention tests.

As in Experiment 2, we conducted an assessment of internal reliability to determine the stability in individual differences in the PRE, POST_E and POST_L impulsive choice testing, and this indicated α= .89, α= .94, and α=.93, respectively. The test-retest reliability correlation for choice behavior is displayed in Figure 7B and 7D. In general, the changes in choice behavior were negligible and were generally better for rats with higher pre-intervention log odds LL choices. There was a positive correlation in individual differences in log odds of LL choices pre- versus early post-intervention test, r = .84, p < .001, and late post-intervention test, r = .86, p < .001.

4.2.1.2. Timing behavior

Figure 8 shows the mean response gradients for the rats in Group FI (left column) and Group VI (right column) for each of the SS and LL delays. Generally, the peak functions were centered near the expected time of food delivery, with good tracking of the changes in the SS delay and intact timing of the LL delay throughout. Post-intervention functions were generally sharper and were associated with a higher peak rate of responding, but standard deviations during peak trials were higher compared to both Experiments 1 and 2.

Figure 8.

Experiment 3 responses per minute as a function of time during smaller-sooner (SS) and larger-later (LL) peak trials for each SS delay in the pre- and post-intervention impulsive choice task in the FI (A–C) and VI (D–F) groups.

Equation 4 was fit to the individual SS and LL peak interval data for each phase pre- and post-intervention, accounting for a mean of 84% of the variance in the peak functions on average. Figure 9 displays the mean peak time (A and E), peak rate (B and F), peak standard deviation (C and G), and peak CV (D and H) for Groups FI and VI in the pre- and post-intervention phases on the SS (A–D) and LL (E–H) levers. A 2 (Lever) × 2 (Pre/Post) × 3 (SS Delay) × 2 (Group) mixed ANOVA was conducted on the peak time, rate, standard deviation, and CV measures.

Figure 9.

Experiment 3 mean (± SEM) peak time (A and E), peak rate (B and F), peak standard deviation (STD, C and G), and peak coefficient of variation (CV, D and H) obtained from the Gaussian fitted functions on SS (A–D) and LL (E–H) peak trials delivered to Groups FI and VI pre- and post-intervention. Note the different scaling of the ordinate axis in the peak time and standard deviation graphs for the SS and LL levers.

For the peak time measure (Figure 9A and 9E), the LL lever was associated with later peak times overall, reflected by the main effect of Lever, F(1, 19) = 197.07, p < .001, η_p² = .912. There also was a main effect of SS Delay, F(2, 38) = 89.30, p < .001, η_p² = .825, due to increasing peak times when the SS delay was increased. There also was a Lever × SS Delay interaction, F(2, 38) = 63.21, p < .001, η_p² = .769, due to the selective effect of SS delay manipulations on SS peak times. There were no intervention effects or group effects on peak times.

For the peak rate measure (Figure 9B and 9F), there were significant main effects of Lever, F(1, 19) = 9.00, p = .007, η_p² = .321, Pre/Post, F(1, 19) = 64.04, p < .001, η_p² = .771, and SS Delay, F(2, 38) = 12.18, p < .001, η_p² = .391. There also was a Lever × Pre/Post × SS Delay interaction, F(2, 38) = 3.65, p = .035, η_p² = .161. Post-hoc analyses on the three-way interaction indicated that there were significant Pre/Post effects on both levers for all SS delays except on the SS lever at the 10-s SS delay. Further, the magnitude of changes was generally greater on the LL lever. Thus, there was a general intervention effect, but the strength of the effect was moderated by lever and SS delay.

For the peak standard deviation measure (Figure 9C and 9G), there was a main effect of Lever, F(1, 19) = 98.85, p < .001, η_p² = .839. There was also a Lever × SS Delay interaction, F(2, 38) = 56.13, p < .001, η_p² = .747, a Pre/Post × SS Delay interaction, F(2, 38) = 7.78, p = .001, η_p² = .290, and a Lever × Pre/Post × SS Delay interaction, F(2, 38) = 3.42, p = .043, η_p² = .153. Follow-up analyses on the three-way interaction indicated that the standard deviation on the SS lever significantly decreased post-intervention for all SS delays (p ≤ .026). However, on the LL lever, there were significant Pre/Post differences during the 5-s (p = .048) and 10-s (p = .037) delays but not during the 20-s delays. The general pattern of change on the SS lever was a decrease in standard deviation post-intervention, whereas the standard deviation on the LL lever decreased primarily as a function of SS delay in the pre-intervention phase.

Lastly, for the peak CV measure (Figure 9D and 9H), there was a main effect of SS Delay, F(2, 38) = 6.16, p = .005, η_p² = .245, and a Lever × SS Delay interaction, F(2, 38) = 3.30, p = .048, η_p² = .148. Follow-up analyses on the Lever × SS Delay interaction revealed a main effect of SS delay on SS peak CV, F(2, 46) = 7.75, p = .001, η_p² = .252, and no main effect of SS delay on LL peak CV.

4.2.2. FI and VI schedules

As in Experiment 2, the rats exhibited typical FI and VI behavior patterns indicating that both groups learned the interval durations experienced during the intervention phase (data not shown).

4.3. Discussion

The FI and VI schedule exposure to the Lewis strain of rats produced an intervention effect that was apparent for the 5-s SS delay both early and late in testing. Early in testing, there also was a significant increase in impulsive choices at the 20-s delay, but this effect was no longer present later in testing. In addition, the VI produced an overall stronger increase in LL choices than the FI, confirming the general pattern seen in Experiment 2. This suggests that the increasing hazard function associated with the VI schedule, which reinforces waiting behavior, may be slightly more effective in training self-control than the single-interval exposure associated with the FI schedule.

With regard to timing effects, the FI and VI schedules produced increases in response rates across all delays and both levers, but the effects on the standard deviation and CV of the peak were less robust in comparison to Experiment 2. Specifically, the intervention effects appeared to operate on SS peak timing only, as LL peak timing was much more variable. On the LL lever, there were substantial changes in peak standard deviation during the pre-intervention phase, followed by relatively little change post-intervention. The changes during the pre-intervention phase may have occurred due to changes in choice patterns. When the SS delay was 5 s, the Lewis rats rarely chose the LL, limiting the ability to learn the LL delay. As the SS delay increased, choices of the LL increased and this presumably produced decreases in standard deviations due to greater exposure to the LL delays. This pattern was more pronounced than in Experiment 2, presumably because the Lewis rats displayed fewer LL choices throughout, consistent with previous research with this strain (Anderson and Diller 2010; Anderson and Woolverton 2005; García-Lecumberri et al. 2011; Huskinson and Anderson 2012; Huskinson et al. 2012; Madden et al. 2008; Stein et al. 2012).

The lack of a robust intervention effect in the Lewis rats could be due to several reasons, but three of which are considered here. First, the timing and choice deficits in the Lewis rats may be too severe to mitigate with a simple behavioral intervention. Thus, it is possible that rats with more serious timing and choice deficiencies may not benefit from time-based interventions as readily as rats with more intact timing and choice abilities. This is consistent with the observation of generally weaker effects of the intervention of both timing (particularly of the LL) and choice behavior. In addition, this may explain the enhancement of impulsive choices seen early in the post-intervention phase at the 20-s delay, suggesting that the intervention may have produced a transient increase in delay aversion that then weakened with further exposure to the choice task. Second, it is possible that exposure to the longer delays during the intervention may have been less beneficial in the Lewis rats due to their poorer timing in the pre-intervention phase. It is possible that the timing deficits in the Lewis rats may be too severe to mitigate with exposure to delays alone. For example, the Lewis rats showed higher standard deviations overall, and their performance on the LL lever suggested that they were still learning the LL delay over the course of pre-intervention choice training (Figure 9G). Third, the Lewis rats may have required more training to produce an intervention effect. They did receive less training than the rats in Experiment 2 although they received more training than the rats in Experiment 1. In addition, the amount of training delivered in all of the present experiments is within the usual range for inducing accurate and precise timing in typical studies with DRL, VI, and FI schedules (e.g., Church et al. 1998; Church et al. 1994; Pizzo et al. 2009), so this explanation seems less likely than the first two.

Ultimately, the weaker intervention effect in the Lewis rats suggests that additional factors may be operating on their choice and/or timing behavior. Garcia and Kirkpatrick (2013) tested Lewis rats on an impulsive choice task with increasing SS delays and found that as the delay to the SS increased, the choice behavior of the Lewis rats became increasingly suboptimal, similar to what was found here. In addition, the Lewis rats displayed decreases in LL choices between the first and second half of the experimental session, in contrast to the increase in LL choices observed in the Wistar, Wistar-Kyoto, and Spontaneously-Hypertensive Rat strains, suggesting that the Lewis rats developed delay aversion over the session. Furthermore, in contrast to the deficits observed with delay manipulations, the Lewis rats displayed relatively intact choice behavior when the LL magnitude was manipulated and increased their LL choices over the session in the magnitude condition. These results suggest that the Lewis strain may have an inherent, genetically-predisposed aversion to delayed outcomes (see García-Lecumberri et al. 2011; Garcia and Kirkpatrick 2013; Madden et al. 2008), which could decrease selection of longer delays and presumably impact the timing of those delays. One possible mechanism proposed by Garcia and Kirkpatrick (2013) that could produce delay aversion, deficits in adjusting to changes in delays, and noisier timing functions is poor attention to time. Decreased attention to time has been shown to result in increased variance in peak timing functions (Brown 1997; Galtress and Kirkpatrick 2009) and has been argued to occur through fluctuations in the switch in scalar timing theory throughout a trial (Fortin 2003; Fortin and Massé 2000). Thus, attentional variations could provide one possible route for explaining the effects of timing and/or delay aversion on choice behavior (discussed further below).

An additional factor that may have impaired the intervention effect in the Lewis rats is reward valuation deficits in integrating delay and magnitude information. One key neurobiological difference in the Lewis strain (compared to control strains) is lower dopamine (and serotonin) functionality in the mesolimbic pathway, particularly the nucleus accumbens (Flores et al. 1998; Harris and Nestler 1996; Higuera-Matas et al. 2011; Martin et al. 1999). The nucleus accumbens has been proposed as a site for integrating delay and magnitude information to form an overall valuation signal (Gregorios-Pippas et al. 2009; Kable and Glimcher 2007). Furthermore, lesions of the nucleus accumbens increase impulsive choice (Cardinal et al. 2001; Galtress and Kirkpatrick 2010), and dopamine and/or serotonin levels within the accumbens and related reward-system structures predict impulsive choice (Evenden 1999; Ho et al. 1998; Kirkpatrick et al. 2014; Loos et al. 2010; Pardey et al. 2012). Accordingly, poor neurotransmitter function within the reward system may impair the integration of delay and magnitude information, which may thereby produce suboptimal (impulsive) choice behavior in the Lewis rats. This suggests that successful interventions may need to employ pharmacological treatments to increase dopamine and/or serotonin. These treatments could be coupled with exposure to delays and/or magnitudes so that delay and magnitude are properly synthesized into a more accurate subjective valuation signal, as previous pharmaceutical interventions alone using Lewis rats have shown mixed results (Anderson and Diller 2010; Anderson and Woolverton 2005; Huskinson and Anderson 2012; Huskinson et al. 2012).

5. General Discussion

The current series of experiments assessed different time-based behavioral interventions that aimed to decrease impulsive choice behavior; all three interventions also concurrently improved timing precision within the choice task. A successful intervention effect was observed in normal rats using a DRL task in Experiment 1, which involved both inhibitory and timing processes, and FI and VI schedules in Experiment 2, which relied more heavily on timing processes. The intervention effects were stronger in Experiment 2 with the FI and VI schedules. There also was a transient intervention effect with a target population, the Lewis strain, using the FI and VI schedules in Experiment 3. In general, all three temporal paradigms induced an increase in self-control coupled with improvements in temporal precision.

Given that the time-based interventions concurrently improved both self-control and timing precision indices of behavior, the results support the potential importance of timing processes in impulsive choice/self-control. The current results are consistent with recent reports of a relationship between timing precision, delay tolerance, and self-control (Marshall et al. 2014; McClure et al. 2014). In fact, Marshall et al. (2014) found that timing precision accounted for more than 50% of the variance in impulsive choice behavior, that delay tolerance accounted for 40% of the variance, and that these two measures were significantly inter-correlated. This suggests that timing precision, delay tolerance, and impulsive choice may all emerge from a common underlying factor. The current findings are also consistent with previous research demonstrating that exposure to reward and/or delay (Eisenberger and Adornetto 1986; Madden et al. 2011; Stein et al. 2013), and increased salience of delay (Zauberman et al. 2009), can exert powerful effects on impulsive choice. However, these studies did not measure any concomitant effects on specific temporal processes. While interventions aimed at improving timing have been suggested previously (Baumann and Odum 2012; Cheng 1992; Galtress et al. 2012; McGuire and Kable 2012, 2013; Wittmann and Paulus 2008, 2009), the current experiments expand on the previous literature by demonstrating the potential importance of improvements in timing processes to reduce impulsive choice.

The present results were unlikely driven by changes in choice behavior due to the effect of repeated testing, as evidenced by the high internal and test-retest reliability that was observed and the fact that the intervention effects were apparent early in the post-intervention testing. Internal reliability assessments for Experiments 2 and 3 indicated strong consistency in individual differences within the pre- and post-intervention assessments, consistent with other recent research (Galtress et al. 2012; Garcia and Kirkpatrick 2013; Marshall et al. 2014). In addition, in all three studies, there was significant test-retest reliability, indicating that the rats were stable in their choice behavior across the pre- and post-intervention assessments. The combined results are consistent with impulsive choice serving as a trait variable in rats, similar to what has been observed in humans (Jimura et al. 2011; Kirby 2009; Odum 2011b; Odum and Baumann 2010; Simpson and Vuchinich 2000). The results are also consistent with a recent study examining stability of impulsive choice in rats over multiple assessments (Broos et al. 2012) and with recent research in our laboratory examining test-retest stability over periods of 2–3 weeks (Smith et al. in preparation) and up to 5 months (Peterson et al. under review). In all of these cases, there were no significant migrations in choice behavior over repeated testing, suggesting that the current results were not due to these factors.

While the present study indicated that the timing intervention operated through a mechanism that concurrently produced reductions in variability of peak timing functions in Experiments 1 and 2, the underlying source of these effects remains to be elucidated. Within the context of the scalar expectancy theory (SET; Gibbon and Church 1984; Gibbon et al. 1984), there are two main candidates for reducing the standard deviation of the peak functions: (1) decreases in the threshold for responding, and (2) increases in attention. According to SET, timing is accomplished by a pacemaker-accumulator clock process coupled with reference memory and decision components. A pacemaker emits pulses, with the mean and variability of pulse rates determined by parameters in the model, which are sent to an accumulator that sums the pulses during a timed interval. A switch operates to gate pulses into the accumulator, and this switch has been proposed to potentially fluctuate over time (Fortin 2003; Fortin and Massé 2000). At the time of reinforcement, the pulse count is transferred into reference memory as an individual item. The decision process compares a sample from reference memory to the current pulse count in the accumulator, generating a similarity metric that unfolds in time. When the current count is sufficiently similar to the sample in relation to the threshold, responding occurs.

A lower decision threshold would produce sharper peak functions, so an alteration of decision thresholds is a possible candidate for explaining the intervention effects on timing. Decreasing decision thresholds may also bear some relationship with increased self-control, as both would presumably require inhibitory processes. A decreased decision threshold would result in more selective responding around the time of reinforcement, producing inhibition of responses outside of the selective window. Changes in the decision threshold would also only affect timing performance, and would have no impact on the representation of time. However, if the decision threshold were playing a major role in the intervention effects, then one might expect to have seen the strongest effects in the DRL intervention, followed by the FI, and then the VI. The DRL intervention most effectively capitalizes on inhibitory processes, whereas the FI would result in sharper peak functions compared to the VI, thereby reflecting lower decision thresholds. In fact, the opposite pattern was observed.

An alternative mechanism is a modulation of attention to time, which has been proposed to affect switch activation. Variations in attention to time would result in variability in the engagement of the switch across intervals, resulting in greater variance in timing functions (see Crystal et al. 2003 for a relevant example; Fortin 2003; Fortin and Massé 2000). The intervention could induce more consistent and greater attention to time, thereby decreasing variance in peak functions through altering the experience of time. Decreases in the variance of interval timing could lead to better representations of the delays, which could then lead to increased self-control.

Finally, it is possible that the intervention effects were due to mere exposure to delays in the absence of effects on core timing processes. Delay exposure could increase delay tolerance as the primary effect and improve the precision of timing as a secondary effect. Stein et al. (2013) assessed the effect of exposure to either no delay, a fixed delay, or a progressively increasing delay on impulsive choice behavior. They found that exposure to either fixed or progressive delays increased LL choices compared to the no-delay condition, consistent with the present findings. Their findings are interesting in light of the fact that, similar to their study, we did not observe any differences in exposure to different schedules of delays, suggesting that a range of different delay exposure techniques may be effective in reducing impulsive choice behaviors. It is possible that exposure to a ramping hazard function is sufficient to improve both timing precision and self-control, a factor that should be examined in future research. The results of the present studies suggest that exposure to a wider range of delays (as in the VI intervention) may be more advantageous for promoting self-control than exposure to the particular SS or LL delays (Experiments 2 and 3), and also suggest that exposure to SS alone, LL alone or both SS and LL delays are similarly effective (Experiment 1). Altogether, this suggests the mere exposure to delays may be sufficient to induce an intervention effect, even if those delays are not a part of the choice task. However, further research is needed to determine whether intervention effects on both choice and timing behaviors are seen if exposure only occurs to delays that outside of the range (shorter or longer) used in the choice task.

Whether delay tolerance or temporal precision is the primary factor, the present findings are consistent with multiple observations of associations between timing and impulsive choice in individuals with attentional deficits, such as ADHD patients who display increased impulsive choice (e.g. Barkley et al. 2001; Scheres et al. 2010; Wilson et al. 2011), increased delay aversion (Bitsakou et al. 2009; Marco et al. 2009; Solanto et al. 2001), and timing deficits (Baldwin et al. 2004; Barkley et al. 1997; Cappella et al. 1977; Kerns et al. 2001; Noreika et al. 2013; Smith et al. 2002; Sonuga-Barke, Bitsakou, et al. 2010). Sonuga-Barke (2003) proposed that delay aversion would lead to two primary outcomes: (1) avoidance of long delays, which would lead to greater choices of SS rewards; and (2) increased attention to non-temporal stimuli during the experience of long delays. When coupled with the scalar timing model, this second component of the delay aversion hypothesis predicts increased variability in peak functions due to poor temporal attention. This idea is generally consistent with the recent findings indicating that rats that were the most impulsive exhibited high variance in their timing functions (Marshall et al. 2014; McClure et al. 2014), and also a high degree of delay aversion (Marshall et al. 2014).

Presumably, delay aversion and timing precision might interact, with delay aversion resulting in reduced exposure to intervals that would negatively impact the timing of longer intervals (Baumann and Odum 2012; McGuire and Kable 2012, 2013). Thus, if an individual has a poor understanding of delays, then this could lead to aversion to and reduced choice of longer delays to reward, consistent with previous literature (Baumann and Odum 2012; McGuire and Kable 2012, 2013; Wittmann and Paulus 2008). Thus, further research should focus on understanding the causal nature of the relationship among timing, delay aversion, and impulsive choice.

An additional important factor in the present studies was the use of a fixed ITI (120 s in Experiment 1 and 60 s in Experiments 2 and 3), which resulted in a higher overall rate of reinforcement for LL choices (from a molar maximizing standpoint) for all SS delays. For example, with delays of 10 versus 30 s for 1 versus 2 pellets and a 120-s ITI, the within-trial reinforcement rate was higher for the SS (1/10 versus 2/30), but the overall reward rate (including the ITI time) was higher for the LL (1/130 versus 2/150). The LL became increasingly better as the SS delay increased, but LL choices led to greater overall reward earning at all SS delays. If local reinforcement rates were the basis of choice behavior, then the SS would have been preferred when the delay was 5 or 10 s, but the LL would have been preferred at the 20-s SS delay. Individuals with delay aversion have been argued to pay more attention to local rates of reinforcement (Bitsakou et al. 2009), and this is consistent with the choice patterns in the Lewis strain, so this could explain the behavior of the Lewis rats and the behavior of the most impulsive rats across the three studies. In all three experiments, increasing self-control led to greater reward earning. It would be interesting to determine whether the intervention would induce greater LL choice behavior in situations where the LL did not result in greater reward earning overall. This would gauge whether the intervention operates purely on within-trial factors and thus decreases delay aversion (e.g., increased preference for longer delays) or facilitates the integration of delay and magnitude information, thus encouraging the use of longer time horizons that incorporate the ITI for molar maximizing (e.g., increase overall reward earning rates; Balc et al. 2011; Bateson 2003).

The development of successful behavioral interventions could present the opportunity to mitigate other problem behaviors, and thus is a potentially important enterprise. Impulsive choice behavior is associated with numerous maladaptive behaviors, and impulsivity may also be a causal risk factor for these behaviors (Kreek et al. 2005; Verdejo-García et al. 2008), particularly during adolescence and young adulthood, when impulsivity is more robust (Green et al. 1994) and relatively stable (White et al. 1994). Furthermore, increasing self-control may also increase the individual’s propensity towards more positive outcomes (Mischel et al. 1989). Thus, interventions to increase self-control may improve the ability for individuals to make more adaptive decisions that could imbue a wide range of benefits.

In conclusion, the present series of experiments have shown that time-based interventions, which decreased variance in timing, can be an effective mechanism to increase self-control. This was shown with both a DRL intervention, which drew on both temporal and inhibitory processes, and an FI and VI exposure intervention, which relied primarily on temporal processes. These results suggest that increases in inhibitory mechanisms may not be necessary to increase self-control. Instead, the interventions may improve self-control through decreasing variation in temporal processing or potentially reducing delay aversion, or a combination of both. Furthermore, the interventions were successful at increasing self-control in a normal population of Sprague-Dawley rats and partially successful with the Lewis rats. Ultimately, the present research has provided a potential avenue for translational research to develop behavioral intervention techniques to reduce maladaptive impulsive behaviors in various sub-populations. In conjunction with the results from other recent studies (Marshall et al. 2014; McClure et al. 2014), the present results indicate that timing processes may serve as an important mechanism in understanding individual differences in self-control and a key target for interventions for increasing adaptive decision making.

Highlights.

• Impulsive choice behavior is a primary risk factor for other maladaptive behaviors

• Three time-based behavioral interventions were implemented to increase self-control

• The interventions increased preference for delayed rewards

• The interventions also increased temporal precision within the choice task

• The results indicate a role for temporal processing in impulsive choice behavior