Hazard function effects on promoting self-control in variable interval time-based interventions in rats

Abstract

The present experiments investigated properties of time-based interventions used to increase self-control. Rats received impulsive-choice assessments before and after interventions that consisted of different distributions of delays to reinforcement. In Experiment 1, rats received an intervention with an increasing hazard function where delays were more evenly distributed, a decreasing hazard function where delays were mostly short, or a constant hazard function where delays were exponentially distributed. Surprisingly, rats that received the decreasing hazard function made the most self-controlled choices. Response rates during intervention trials showed that rats anticipated reinforcement based on the shape of the distributions they received. In Experiment 2, rats received an intervention with a decreasing hazard function with a steep slope or a shallow slope. Both time-based interventions increased self-control and produced similar response-rate patterns, indicating that the slope of the decreasing hazard function may not play a strong role in intervention efficacy. While this research aligns with previous literature showing that time-based interventions improved self-control, exposure to short delays produced the biggest improvements. Ultimately, exposure to short delays may increase the subjective value of the larger–later choice while occasional long delays may promote the ability to wait, which may have important implications for translational applications.

Impulsive choice is the construct of choosing smaller–sooner rewards over larger–later rewards when doing so is disadvantageous (Mazur, 2000; Odum, 2011). Time-based interventions using delays to reinforcement to reduce impulsive choices have been used in humans (Eisenberger & Adornetto, 1986) and nonhumans (Bailey et al., 2018; Fox et al., 2019; Panfil et al., 2020; Renda & Madden, 2016; Rung et al., 2018) with both showing promising results. Despite the success of these interventions, little is known about their specific properties or mechanisms producing such effects. To delve into the specifics of these interventions, it is important to understand the elements of impulsive choices that may be affected by the interventions.

Delay discounting, a key process in impulsive choices, is the decay of reward value as a function of time (Baumann & Odum, 2012). Discounting may be a product of delay intolerance and/or poor time perception (Litrownik et al., 1977; Takahashi, 2005; Wittmann & Paulus, 2008). Individuals who do not perceive time accurately and/or precisely may have lower delay tolerance (or vice versa) leading to avoidance of long delays, consequently decreasing the opportunity to acquire or improve timing abilities (Marshall et al., 2014; Smith et al., 2015; Wittmann & Paulus, 2008). Ultimately, this may create a cycle of poor time perception, delay intolerance, steep discounting, and impulsive choices. To break this cycle, time-based interventions provide unavoidable exposure to delays, which may lead to reductions in impulsive choices by influencing timing and/or delay tolerance.

Decreases in impulsive choices may be indicative of increased delay tolerance such that self-controlled individuals are more tolerant of the aversive aspects associated with waiting. A common time-based intervention to target delay tolerance is interval fading, which consists of repeatedly exposing subjects to intervals that increase or decrease incrementally. Fading likely targets delay tolerance as it typically involves slow exposure to delays to build individuals’ ability to wait. For example, pigeons repeatedly chose between a smaller reinforcer available immediately (control condition) or after a 6-s delay (experimental condition) and a larger reinforcer available after a 6-s delay. Then, in the experimental group, the delay to the smaller reinforcer gradually decreased from 6 to 0 s across sessions and delay to the larger reinforcer remained constant at 6 s. Pigeons in the experimental group made more delayed, larger reinforcer choices than control pigeons even when cues were switched, suggesting fading interventions decrease impulsive choices by increasing delay tolerance (Mazur & Logue, 1978). Fading may not be the only way to improve delay tolerance. Stein et al. (2013) compared two interventions to a no-delay (ND) control task. The fixed-delay (FD) intervention consisted of repeated exposure to a single delay while the progressive-delay (PD) intervention exposed rats to delays that increased across trials. In a postintervention impulsive choice test, both FD and PD groups made fewer impulsive choices than the ND group, suggesting that improving delay tolerance with an interval-fading intervention may be similarly effective to a fixed-delay intervention when trying to decrease impulsive choices. Similar results have been demonstrated in humans (Binder et al., 2000; Neef et al., 2001; Schweitzer & Sulzer-Azaroff, 1988). Taken together, delay exposure, regardless of the form it is delivered, may improve delay tolerance and lead to decreased impulsive choices.

Timing processes are likely necessary to learn impulsive choice task contingencies. If timing abilities are poor, then the delays to each choice may be perceived inaccurately and/or imprecisely, potentially affecting control by choice outcomes. Smith et al. (2015) proposed that time-based interventions reduce impulsive choices by improving timing of delays. Across three different experiments, male rats completed a baseline impulsive choice task and then received training under differential-reinforcement-of-low-rates (DRL), fixed-interval (FI), or variable-interval (VI) schedules. The DRL intervention required rats to wait the elapsed interval to make the first response; otherwise, the interval reset. The FI intervention involved forced exposure to two FI schedules (10 s and 30 s). The FI 10-s schedule was delivered on the smaller–sooner (SS) lever and the FI 30-s schedule was administered on the larger–later (LL) lever. The VI intervention was like the FI intervention in that rats received forced exposure to intervals delivered on different levers in separate blocks of sessions, but intervals were varied with mean intervals that matched the FI intervention. During the VI intervention, each interval was randomly sampled from a uniform distribution that averaged 10 s and 30 s on SS and LL levers, respectively. Rats could respond during the interval on FI and VI schedules but had to respond after the target interval to receive food. In the postintervention impulsive choice task, all interventions were effective at decreasing impulsive choices and were associated with increased temporal precision as measured on peak trials delivered during the choice task. Temporal precision was reflected in sharper timing functions (i.e., smaller peak standard deviation), indicating lower timing error. In sum, this suggests that improved timing may reduce impulsive choices in male rats.

VI schedules are of potential interest because they provide exposure to a range of intervals, which may be an advantage in promoting transfer to choice delays. VI schedules can be constructed from distributions with different properties. Smith et al. (2015) noted that the uniform VI schedule they delivered had a linear increasing hazard function with probability of food delivery increasing with time. They proposed that the increasing hazard function may be an important factor in intervention efficacy. The hazard function is defined as the rate of an event occurrence at a given time, given that event has not yet occurred (Evans et al., 2000). The hazard function can be calculated from the distribution of delays in a reinforcement schedule. A VI schedule may have an increasing, constant, or decreasing hazard function, depending on the distribution of delays. A VI schedule with a constant hazard function would arise from an exponential distribution of delays, for example. When comparing hazard functions, it is possible that increasing hazard functions may promote delay tolerance as rats may be more willing to wait for larger rewards when the probability of food is increasing over time. The linear increasing hazard function results in a built-in delay of reinforcement gradient. For example, an individual may be more likely to continue to wait at a bus station if the bus is increasingly likely to arrive in the coming minutes but will be more likely to give up if the likelihood of the bus arriving is decreasing in the coming minutes. Thus, the hazard property of interval schedules may provide at least one explanation for how time-based interventions produce their effects.

The current experiments aimed to determine if different hazard functions influenced efficacy of time-based interventions on impulsive choice. Experiment 1 investigated three VI interventions with differing delay distributions associated with different hazard functions. We predicted that the intervention using the increasing hazard function would decrease impulsive choices, the decreasing hazard function would increase impulsive choices, and the constant hazard function would have no effect on impulsive choices. Instead, we found that the decreasing hazard function was most effective, possibly due to the preponderance of very short delays. Experiment 2 assessed whether altering the slope of the decreasing hazard function, by increasing the probability of very short delays, affected intervention efficacy. Previous research indicated that timing responses show sensitivity to the slope of increasing hazard functions (Matell et al., 2014). However, we did not find any effect of slope of the hazard function on choice behavior, and only subtle effects on timing behavior.

Method

Subjects

Sixty experimentally naïve male Sprague Dawley rats (Charles River, Stone Ridge, NY) were used in Experiments 1 (N = 36) and 2 (N = 24). All rats arrived at the facility (Kansas State University, Manhattan, KS) at 21 days of age and began behavioral testing at 35 days of age. Rats were pair-housed and remained on a 12-hr reverse light:dark schedule (lights off at 7 am). They were tested during the dark phase of the cycle. Rats were food restricted to roughly 85% of their free-feeding body weight. Rats had unrestricted access to water in home cages and in the operant chambers.

Apparatus

The experiment used 24 operant chambers (Med-Associates, St. Albans, VT), containing a food cup (ENV-200R7), two nose poke lights (ENV-119M-1), and two retractable levers (ENV-112CM). The chambers were 25 × 30 × 30 cm and sat inside ventilated, noise attenuating boxes that were 74 × 38 × 60 cm. The retractable levers were located approximately 6.5 cm from the bottom of the chamber and 4 cm on either side of the food cup. The levers protruded 1.9 cm into the chamber when inserted and required approximately 0.25 Newtons of force to press. A nose-poke light was located 9 cm above each lever, and the color was yellow. Nose-pokes were automatically recorded, but had no programmed consequences. On the opposite wall there was an opening for the water sipper tube. On the outside of the box were two pellet dispensers (ENV-203) that dispensed 45-mg food pellets (Bio-Serv, Flemington, NJ) into the food cup. Experimental events were controlled and recorded with 2-ms resolution by the software program MED-PC IV (Tatham & Zurn, 1989) in Experiment 1. Experimental events were controlled and recorded with 1-ms resolution by the software program MED-PC V in Experiment 2.

Procedure

Initial Training

Initial training was conducted over course of three sessions (see Bailey et al., 2018 for details). The first session of initial training consisted of magazine training and lever training. Magazine training consisted of food delivery to the food cup on a variable-time 60-s schedule. Lever training consisted of a fixed-ratio (FR) 1 schedule of reinforcement on each lever sequentially. Only one lever was extended at a time, and levers alternated after 20 food pellets were earned (10 on each lever). Following the first session of lever training, two consecutive sessions were delivered. Both sessions contained lever training on an FR 1, a random-ratio (RR) 3, and an RR 5 schedule of reinforcement in that order. The FR 1 schedule of reinforcement during these sessions were the same lever training where only one lever was extended at a time and each press resulted in one food pellet until 20 food pellets were earned across levers. During the RR 3 schedule, both levers were extended and three responses were required on average to earn one food pellet. Rats worked on both levers and independent schedules until 20 food pellets were earned (10 on each lever). The RR 5 schedule and delivery was the same except an average of five responses was required to earn one food pellet.

Preintervention Impulsive-Choice Task

The impulsive-choice task involved rats choosing between a smaller–sooner (SS) reinforcer (one pellet) and a larger–later (LL) reinforcer (two pellets). The abbreviated task used in the current study was previously compared to an extended impulsive-choice task within an FI intervention paradigm and performance was positively correlated across the two tasks (Panfil et al., 2020). All delays delivered in the impulsive choice task were a fixed duration. The task lasted for a total of eight sessions consisting of five phases. The first phase lasted four sessions in which the SS delay was 10 s. After that, the SS delay increased for every consecutive session: 15, 20, 25, and 30 s. The delay to the LL was always 30 s. Each session contained a mixture of free-choice and SS and LL forced-choice trials with a block of six SS forced-choice trials at the beginning of every session. After the first six SS forced-choice trials at the beginning of the session, rats received 48 free-choice, 12 SS forced-choice, and 12 LL forced-choice trials presented in a random order. Free-choice trials involved the insertion of both levers. Once a lever was pressed, the nose-poke light above the pressed lever turned on, the alternative lever was retracted, and the scheduled delay began. The next press after the scheduled delay resulted in food delivery, the nose-poke light turned off, the lever retracted, and the 60-s intertrial interval (ITI) started. Forced choice trials were the same as free choice trials except only one lever was inserted. Sessions lasted for 2 hr, until 132 food pellet reinforcers were earned (one or two per trial), or 78 trials were completed. Lever assignments were counterbalanced across rats and held constant throughout each experiment.

Time-Based Interventions

Rats were assigned to intervention groups using rank-order matching based on preintervention mean LL choices and the slope of the choice function to ensure there were no initial group differences. Analyses confirmed no initial differences were present (see data analysis and results sections). In Experiment 1, rats were assigned to one of three groups (n = 12): hazard function increasing (HFI), hazard function decreasing (HFD), or hazard function constant (HF0). In Experiment 2, rats were assigned to two groups (n = 12) and received decreasing hazard functions with a steep slope (ST) or shallow slope (SH).

In both experiments, the intervention groups received exposure to a distribution of smaller–sooner delays on SS intervention trials, and a distribution of larger–later delays on LL intervention trials. The two distributions of delays were delivered independently with a block of 20 sessions for the SS intervention trials and a block of 40 sessions for the LL intervention trials, with order counterbalanced across rats. The distribution of delays for the SS intervention trials had a mean of 10 s, and the distribution of delays for the LL intervention trials had a mean of 30 s. Sessions consisted of forced-choice trials delivered exactly like the forced-choice trials in the choice task, with 100 trials for the SS intervention sessions or 50 trials for the LL intervention sessions. One lever was inserted at the beginning of the trial. After the first press, the nose-poke light above the lever turned on, and the scheduled delay began. The next press after the scheduled delay resulted in food delivery, the nose-poke light turned off, the lever retracted, and the 60-s ITI started. The SS intervention sessions lasted for approximately 2 hr, and the LL intervention sessions lasted for approximately 1.25 hr. Responses were recorded to determine if lever pressing tracked the distribution of delays.

Delay Distributions (Experiment 1).

The delays for each group were sampled from a Weibull distribution; the parameters for each group for the SS and LL delays are given in Table 1. The Weibull was selected because it contains a family of hazard functions with increasing, decreasing, and natural exponential functions (HF0) encompassed in the distribution (Evans et al., 2000). The hazard (h) function for each group, shown in the top row of Figure 1, is determined by Equation 1, where β is the shape of the function, η is the scale parameter.

Table 1.

Delay Distribution Parameters of SS and LL Delays for Experiment 1

	HFI SS	HFD SS	HF0 SS	HFI LL	HFD LL	HF0 LL
β	2	0.5	1	2	0.5	1
η	11	5	10	34	15	30
Mean	10	10	10	30	30	30

Note. The shape (β), scale (η), and mean of the distribution of SS and LL delays delivered to the three groups in Experiment 1. HFI = hazard function increasing, HFD = hazard function decreasing, HF0 = hazard function constant, SS = smaller–sooner, LL = larger–later.

Figure 1. Hazard and Probability Density Functions for Experiment 1.

Note. Top Row: The hazard functions, generated from a Weibull function with different shape and location parameters, for the three groups in Experiment 1 for the SS (left) and LL (right) delays with means of 10 s and 30 s, respectively. Note that the HFI hazard function is plotted on a secondary axis for the SS delay function for easy of viewing. Bottom Row: The probability density functions for the three groups for the SS (left) and LL (right) delays. Note that the probability density for the HFD group is plotted on the secondary y-axis for ease of viewing. Delays were randomly sampled in each intervention session from the relevant distribution but with the restriction that delays could not exceed 100 s. HFI = hazard function increasing, HFD = hazard function decreasing, HF0 = hazard function constant, SS = smaller–sooner, LL = larger–later.

$$h=\frac{\beta x^{\beta -1}}{{\eta }^{\beta }}$$ (1)

The distribution of delays (probability density, PD) associated with each hazard function, shown for each group in the bottom row of Figure 1, is determined by Equation 2.

$$PD=h*{\text{e}}^{-{\left(\frac{x}{\eta }\right)}^{\beta }}$$ (2)

Delays (D) were randomly sampled from the distributions for delivery in the experimental sessions using Equation 3, where r was a uniformly distributed random number between 0 and 1, with the restriction that we did not deliver any delays > 100 s. These delays were resampled and replaced with delays less than 100 s. This is because some distributions included possible delays that were longer than was feasible to deliver (several minutes to hours).

$$D=\eta *-\text{ln}{\left(r\right)}^{1/\beta }$$ (3)

Delay Distributions (Experiment 2).

The intervention was the same as in Experiment 1 except both groups (shallow and steep) consisted of a decreasing hazard function with differences in the slope of the function (see Fig. 2 for delay distributions and Table 2 for parameters). Note that the steep (ST) function was generated with the same parameters as the HFD function in Experiment 1.

Figure 2. Hazard and Probability Density Functions for Experiment 2.

Note. Top Row: The hazard functions, generated from a Weibull function with different shape and location parameters, for the two groups in Experiment 2 for the SS (left) and LL (right) delays with means of 10 s and 30 s, respectively. The ST group was generated with the same parameters as the HFD group in Experiment 1. The SH hazard function is plotted on a secondary axis for ease of viewing. Bottom Row: The probability density functions for the two groups for the SS (left) and LL (right) delays. The probability density for the SH group is plotted on the secondary y-axis for ease of viewing. Delays were randomly sampled in each intervention session from the relevant distribution but with the restriction that delays could not exceed 100 s. ST = steep hazard function, SH = shallow hazard function, SS = smaller–sooner, LL = larger–later.

Table 2.

Delay Distribution Parameters of SS and LL Delays for Experiment 2

	ST SS	SH SS	ST LL	SH LL
β	0.50	0.75	0.50	0.75
η	5	8	15	24
Mean	10	10	30	30

Note. The shape (β), scale (η), and mean of the distribution of SS and LL delays delivered to the two groups in Experiment 2. ST = steep hazard function, SH = shallow hazard function, SS = smaller–sooner, LL = larger–later.

Postintervention Impulsive Choice Task

The postintervention assessment was identical to the preintervention assessment.

Data Analyses

Impulsive Choice Task

Choice data were analyzed using repeated-measures multilevel generalized linear modeling of the impulsive choice functions with MATLAB R2020a. This approach calculates a slope function (delay sensitivity) that indicates the delay-discounting rate (Young, 2017, 2018). We assessed group differences at two intercepts: 0 s and 30 s. The 0-s intercept estimated choices at a 0-s SS delay, providing a measure of preference for immediacy. The 30-s intercept assessed choices when the SS delay was 30 s (the same as the LL delay), providing a measure of preference for larger magnitudes. For all experiments, the fixed effects were tested in a full factorial model with Group (intervention group), Pre/Postintervention, and SS Delay. Rat (intercept) was included as a random effect. For Experiment 1, additional models were conducted on preintervention data to assess the matched-groups and postintervention data to assess the intervention effects. For these models, Group and SS Delay were fixed effects, and rat (intercept) was included as a random effect.

Choices were entered into the model as a binary variable (SS = 0, LL = 1) and treated as correlated repeated measures. All categorical variables were effect-coded such that the sum was zero. The reference group was the HF0 group for Experiment 1 and the ST group for Experiment 2. For both experiments, the last session of shortest SS delay and all consecutive sessions were used for data analysis. Thus, in Figures 3 and 5, the choices at the 10-s SS are from the last session of training on that delay. Appendix Figures A1 and A5 display graphs of all sessions of the impulsive choice task in Experiments 1 and 2, respectively. In Experiment 1, there were 13,562 total observations, 7,788 for prechoice and 5,774 for postchoice. In Experiment 2, there were 8,379 total observations, 4,967 for prechoice and 3,412 for postchoice. In Experiment 2, one rat did not complete any trials on one session of the preintervention choice assessment, and another rat did not complete any trials in two sessions of the postintervention choice assessment. We were not able to identify any equipment or other issues to explain the loss of data. Graphs depicting impulsive choice behavior pre- and postintervention for each subject are included in the Appendix (Experiment 1: Figs. A2–4; Experiment 2: Figs. A6–7).

Figure 3. Mean Proportion of LL Choices Pre- and Postintervention for Experiment 1.

Note. Error bars (+/− SEM) were computed with respect to the estimated marginal means of the fitted repeated measures multilevel logistic regression. The model SEM is reported because the mixed effects regression does not give equal weight to all individuals, so this better reflects the treatment of variance in the data by the model. HFI = hazard function increasing, HFD = hazard function decreasing, HF0 = hazard function constant, SS = smaller–sooner, LL = larger–later.

Figure 5. Mean Proportion of LL Choices Pre- and Postintervention for Experiment 2.

Note. Error bars (+/− SEM) were computed with respect to the estimated marginal means of the fitted repeated measures multilevel logistic regression. The model SEM is reported because the mixed effects regression does not give equal weight to all individuals, so this better reflects the treatment of variance in the data by the model. ST = steep hazard function, SH = shallow hazard function, SS = smaller–sooner, LL = larger–later.

Impulsive Choice Task, Internal Consistency

To assess the internal consistency of the rats’ choices in the impulsive choice task, Cronbach’s alpha was calculated for the pre- and postintervention choice assessments with R. Given that choices are a binomial outcome variable, alpha was calculated with the Kuder-Richardson 20 formula. Cronbach’s alpha examines the consistency in the rats’ choices across delay. Each delay tested in the impulsive choice task was treated as an item in the analysis, and the rats in each experiment were included as the individual participants. In these analyses, Cronbach’s alpha values of .7–.95 are viewed as an acceptable level of internal consistency (Tavakol & Dennick, 2011). Values in this range indicate that rats there were most impulsive at any one of the five SS delays were also consistently the most impulsive at the other delays.

Response Rates during Intervention

Responses per minute were analyzed using repeated-measures multilevel nonlinear modeling to assess whether the rats tracked the distribution of delays while learning the intervention. The response rates during the intervention were not the focus of the experiments, so details of the analyses are included in the Supporting Information and only summary details are provided in the main text. Graphs of the response rates with model fits are included in the Supporting Information.

Results

Experiment 1

Impulsive Choice Task

Figure 3 depicts the pre- and postintervention choices for all three groups (see Appendix for graphs of individual differences). At the 0-s intercept, there was a main effect of Pre/Post, t = 7.32, p < .001, b = 0.43 [0.32, 0.55], such that rats increased LL choices following the intervention. When comparing the two panels in Figure 3, the HFI and HFD groups showed greater LL choices after the intervention at the shorter SS delays. There was a Pre/Post × Group interaction, t = 5.31, p < .001, b = 0.44 [0.28, 0.60]. Post-hoc tests of the interaction were conducted to test the original hypotheses. At the 0-s intercept, the HFI group made more LL choices postintervention compared to their preintervention choices (b_Pre = −2.08; b_Post = −1.44), t = 10.11, p = .002. The HFD group also made more LL choices postintervention compared to preintervention at the 0-s intercept (b_Pre = −2.19; b_Post = −0.45), t = 74.68, p < .001. However, the HF0 group’s postintervention choices did not significantly differ from their preintervention choices at the 0-s intercept (b_Pre = −2.24; b_Post = −2.03), t = 0.96, p = .372. There were no significant group differences at the 30-s intercept, indicating that the interventions did not alter preferences for larger magnitudes at the longer delays. Overall, the HFD and HFI groups increased LL choices at the 0-s intercept in the postintervention choice task while the HF0 group did not (Fig. 3).

When examining the slope of the choice functions, there was a Pre/Post × SS Delay interaction, t = −5.54, p < .001, b = −0.50 [−0.68, −0.32], with shallower slopes of the choice functions in the postintervention choice task (Fig. 3). There also were significant Group × SS Delay, t = −3.17, p = .002, b = −0.41 [−0.66, −0.16], and Pre/Post × Group × SS Delay, t = −4.12, p < .001, b = −0.52 [−0.77, −0.27] interactions. Post-hoc tests of the Pre/Post × Group × SS Delay interaction were conducted to test the original hypotheses. The HFI group had a shallower slope postintervention compared to preintervention (b_Pre = 3.93; b_Post = 3.17), t = 5.92, p = .015. Also, the HFD group had a shallower slope postintervention compared to preintervention (b_Pre = 4.12; b_Post = 2.08), t = 43.75, p < .001. However, the HF0 group did not differ in their preintervention versus postintervention slope (b_Pre = 3.98; b_Post = 3.78), t = 0.37, p = .541. Thus, the HFI and HFD groups had flatter slopes after the intervention, but the HF0 slope did not change following intervention (Fig. 3).

To provide further understanding of these interactions, separate models were fitted to data from pre- and postintervention choice assessments to compare the three groups. The preintervention choice functions were used to match group assignments, and the regression confirmed that there were no significant effects of Group or any Group × SS Delay interaction (Fig. 3, top panel).

For the postintervention choices, at the 0-s intercept there was a main effect of Group, t = 3.32, p = .001, b = 0.92 [0.38, 1.46] in that the HFD group (b = −0.47) made significantly more LL choices at the 0-s intercept than the HFI (b = −1.59) and HF0 (b = −2.09) groups, but HFI and HF0 were not significantly different. There were no significant group differences in proportion LL choices at the 30-s intercept, indicating that the interventions did not differentially alter preferences for larger magnitudes.

There also was a significant Group × SS Delay interaction, t = −4.37, p < .001, b = −0.91 [−1.32, −0.50] in that the HFD group (b = 2.35) displayed a significantly shallower slope of their choice function compared to HFI (b = 3.59) and HF0 (b = 3.85), but HFI and HF0 slopes were not significantly different. This suggests that the HFD intervention reduced sensitivity to delay in comparison to the other interventions.

Impulsive Choice Task, Internal Consistency

In the preintervention impulsive choice assessment, Cronbach’s alpha was 0.86 [0.80, 0.93] across the three groups. For preintervention HFD group, Cronbach’s alpha was 0.89 [0.79, 0.99]. For preintervention HFI group, Cronbach’s alpha was 0.76 [0.55, 0.96]. Lastly, for preintervention HF0 group, Cronbach’s alpha was 0.88 [0.79, 0.98].

In the postintervention assessment, Cronbach’s alpha was 0.91 [0.86, 0.95] across all groups. For postintervention HFD group, Cronbach’s alpha was 0.93 [0.88, 0.98]. For postintervention HFI group, Cronbach’s alpha was 0.93 [0.87, 0.99]. Lastly, for postintervention HF0 group, Cronbach’s alpha was 0.84 [0.70, 0.98]. Values of 0.7–0.95 indicate acceptable levels of internal reliability (Tavakol & Dennick, 2011) in performance across the delays within the choice task.

Examination of the individual-subject data within each group (see the Appendix), shows considerable variation in the rats’ choice functions and their responses to intervention. There were individual differences in: (1) LL choices at the 10-s and 30-s intercepts; (2) response to intervention; and (3) degree of stability during the initial choice training at the 10-s delay. The differences in LL choices at the intercept(s) were captured in the regression models by including this variable as a random effect. While there were some individuals whose response to the interventions did not map onto the group effects, it also was the case that the majority of rats in the HFD (Fig. A2) and HFI (Fig. A3) groups showed increased LL choices at some or most SS delays postintervention, whereas a minority of rats in the HF0 group showed increased LL choices postintervention (Fig. A4). Thus, the individual rats generally supported the group-level results. Individual differences in responding over the four sessions of exposure to the 10-s delay may have also affected the intervention response. Rats that were LL preferring in the first four postintervention sessions tended to show the greatest intervention effect, whereas rats that were more SS preferring in those sessions showed either no intervention effect or had few LL choices postintervention.

Response Rates during Intervention

Overall, the response rates resemble the probability density functions more closely than the hazard functions (Fig. 1). The HFI group received relatively fewer short delays and responding was lower at the beginning of the SS and LL intervention trials and increased over time (Fig. 4). The HFD group received the highest probability of short delays, and their responding was highest early in the SS and LL intervention trials and dropped steeply over time. The HF0 group showed their highest response rates early in the SS and LL intervention trials and tapered off gradually over time. See the Supporting Information for detailed analyses of these data.

Figure 4. Responses per Minute During Intervention for Experiment 1.

Note. Responses per minute for each intervention group as a function of time for SS (top) and LL (bottom) intervention trials, collapsed across sessions in Experiment 1. HFI = hazard function increasing, HFD = hazard function decreasing, HF0 = hazard function constant, SS = smaller–sooner, LL = larger–later.

Experiment 2

Impulsive Choice Task

Experiment 2 had a simpler design than Experiment 1, so all variables were included in a single analysis (Fig. 5). There was a main effect of Pre/Post at the 0-s intercept, t = −6.82, p < .001, b = −0.54 [−0.69, −0.38], and 30-s intercept, t = −2.48, p = .013, b = −0.14 [−0.24, −0.03], such that rats increased LL choices at both intercepts following the interventions (Fig. 5). At both the 0- and 30-s intercepts, there were no effects of Group. This indicated that the groups made similar proportions of LL choices (Fig. 5; see Appendix for graphs of individual differences).

When comparing slopes, there was a Pre/Post × SS Delay interaction, t = 3.32, p = .001, b = 0.40 [0.16, 0.64], with rats decreasing delay sensitivity in the postchoice assessment (Fig. 5). There were no significant Group × SS Delay or Group × Pre/Post × SS Delay interactions. Thus, both intervention groups showed increased LL choices and reduced sensitivity to delay postintervention, but there were no group differences.

Impulsive Choice Task, Internal Consistency

In the preintervention choice assessment, the Cronbach’s alpha was 0.92 [0.87, 0.96] across both groups. For preintervention ST group, Cronbach’s alpha was 0.89 [0.81, 0.98]. For preintervention SH group, Cronbach’s alpha was 0.93 [0.88, 0.98]. In the postintervention choice task, the Cronbach’s alpha was 0.87 [0.79, 0.95] across both groups. For postintervention ST group, Cronbach’s alpha was 0.86 [0.76, 0.96]. For postintervention SH group, Cronbach’s alpha was 0.87 [0.76, 0.99]. Thus, there was an acceptable level of consistency in choices within the impulsive-choice task.

Individual differences mirrored what was observed in Experiment 1 (see Figs. A6–7 in the Appendix), with differences in the intercept, in response to intervention, and in responding over the first four sessions of the choice task. Most rats in both groups showed a positive intervention effect with increased LL choices postintervention, but a few rats in each group did not show this result. In addition, the distribution of individual differences was similar in the two groups, consistent with the group-level results.

Response Rates during Intervention

ST and SH groups responded at their highest rates early in the SS and LL intervention trials (Fig. 6). The response rates did not significantly differ on SS intervention trials. On LL intervention trials, the ST group had a significantly steeper slope during the first 30 s of intervention trials but a shallower slope during the latter half of the intervention trials compared to the SH group. See the Supporting Information for more details on model fits and graphs of model fits.

Figure 6. Responses per Minute During Intervention for Experiment 2.

Note. Responses per minute for each intervention group as a function of time for SS (top) and LL (bottom) intervention trials, collapsed across sessions in Experiment 2. ST = steep hazard function, SH = shallow hazard function, SS = smaller–sooner, LL = larger–later.

Discussion

These experiments expanded on established literature of time-based interventions involving variable intervals by studying properties that contribute to their effectiveness. Response rates during the interventions corresponded to the probability distributions of intervals received in both experiments. However, responding better reflected probability density functions than hazard functions. Specifically, the HFI group responded less at the beginning of the trial but increased responding up until average time of food delivery and then decreased (SS) or leveled off (LL) responding later in the trial. The HFD group responded most at the beginning of the trial, then decreased until the mean delay, and then decreased more gradually (SS) or levelled off (LL). Finally, the HF0 group showed a gradual decrease in responding across the trial.

Rats can track interval distributions in variable-interval and varying probability of reinforcement schedules (Church & Lacourse, 2001; Church et al., 1998; Marshall et al., 2014). However, evidence is mixed regarding whether rats track the probability density function or other distributional properties. This distinction is important for understanding what properties of the distribution they may be learning, and for interpreting potential mechanisms of intervention effects on impulsive choice. Harris et al. (2011) reported that rats’ goal-tracking responses approximated probability density functions in uniform and exponential distributions. In contrast, rats appeared to track the probability of reinforcement over time, which is more closely related to the hazard function (Kirkpatrick, 2002; Kirkpatrick & Church, 2003). In the current experiments, rats tracked the intervals during the interventions (see Supporting Information for further details), and this may have affected their choices. In Experiment 2, however, changing the relative probability of very short delays (by changing the slope of the hazard function) did not affect temporal tracking. Both groups tracked the distribution of delays similarly with only small differences in responding on LL intervention trials. Perhaps the change in probability was too subtle for detection.

The HFD intervention led to the greatest improvements in self-control in Experiment 1. This was contrary to our hypothesis, in which we expected HFD to increase impulsive choices. Instead, HFD and HFI increased LL choices and decreased delay sensitivity, but HFD produced a stronger intervention effect. In Experiment 2, there were nondifferential improvements in the two HFD groups, indicating that increasing the probability of very short delays had no measurable impact. It is possible that the HFD intervention reached a ceiling of intervention efficacy. Across experiments, there were considerable individual differences in choice functions and receptivity to interventions (see Appendix). However, most rats in HFD and HFI showed positive intervention effects in increasing their LL choices postintervention. Thus, the individual differences were broadly consistent with group-level results.

The HF0 was an ineffective intervention when comparing pre- and postintervention choices, which was consistent with our predictions. This was apparent in the individual differences, where few rats showed a positive intervention effect. The HF0 condition was created using Weibull parameters that produced an exponential distribution of delays, which results in a constant conditional probability of reinforcement over time. Lack of an intervention effect in this group indicates that mere exposure to variable delays was not sufficient to increase self-control. In terms of probability density, the exponential distribution is more similar to HFD than to HFI, yet these groups were the most disparate in their choice behavior. This suggests choice behavior may have been influenced by hazard functions, although temporal tracking was more aligned with probability density functions. Thus, distributional properties that influence choices may differ somewhat from properties that influence timing of responding.

The HFD intervention may be especially beneficial due to the high probability of very short delays. As previously noted, delay discounting refers to the decrease in subjective reward value as delay to reward increases. The hyperbolic delay discounting model from Mazur (1989) adapted for variable delays is as follows:

$$V={\sum }_{i=1}^n{P}_{i\frac{A}{1+k{D}_i}}$$ (4)

where V represents subjective value of reward, A is the amount of reward, k is the discounting rate, and P_i is the probability of a delay, D_i. Reward values associated with each intervention group for SS and LL delays assuming a k value of 0.1 are displayed in Figure 7. Comparison of value for SS and LL delays shows that subjective value of the LL was much higher in the HFD compared to the HFI and HF0. This is because the HFD intervention had higher probability of short delays, which drives value upward when those short delays are associated with the two-pellet LL reward. The 10-s HFD had approximately 40% of delays lasting 1 s or less, and the 30-s HFD had approximately 20% of delays lasting 1 s or less. Overall, increased subjective value may explain why the HFD intervention was most effective in Experiment 1. Training with HFD may have significantly increased LL value during the intervention, which translated to increased preference for the LL during the choice task. Although not tested here, this model predicts that the HFD intervention should be more effective than an FI intervention (with 10-s SS and 30-s LL). This would be an interesting test of the model for future work.

Figure 7. Calculated Subjective Reward Values for SS and LL Delays for Experiments 1 and 2.

Note. Subjective reward value determined from the hyperbolic delay discounting model adapted for variable delays and assuming a k value of 0.1. The top panel depicts the subjective value for the HFD, HFI, and HF0 interventions in Experiment 1. The bottom panel depicts the subjective value for the ST and SH interventions in Experiment 2. Note that the ST distribution in Experiment 2 is the same as the HFD in Experiment 1. V = Subjective Value, HFI = hazard function increasing, HFD = hazard function decreasing, HF0 = hazard function constant, ST = steep hazard function, SH = shallow hazard function, SS = smaller–sooner, LL = larger–later.

It is worth noting that the hyperbolic model failed to predict two major findings. First, the model does not explain the lack of HF0 intervention effect, where the LL is relatively more valued than the SS compared to HFI. The hyperbolic model predicts that the groups should have been ordered as HFD > HF0 > HFI, which was not the case. Second, when examining subjective reward values associated with ST and SH interventions (Fig. 7, bottom panel), the hyperbolic model predicts a stronger ST intervention effect compared to SH. Instead, the two HFD functions produced similar improvements in self-control. Altogether, another factor may contribute to intervention efficacy.

Intervention efficacy may be related to delay aversion (Sonuga-Barke et al., 1992). Exposure to short delays may have enhanced the LL choice, as is seen with fading interventions (Logue & Mazur, 1981; Mazur & Logue, 1978; Vessells et al., 2018), while occasional long delays may have promoted delay tolerance. Because long delays were infrequent, rats had to occasionally wait for long delays in the context where they mostly waited for short delays. The combination of both exposure to short delays and occasional long delays may have resulted in the most effective increases in self-control. However, we assessed this possibility in Experiment 2 and did not find any differences.

It is possible the slope manipulation was not robust enough to produce the intended outcomes. Parameters in Experiment 2 produced decreasing hazard functions that were not overly steep or shallow. Very steep functions would have resulted in near-exclusive delivery of immediate reinforcers, which do not consistently affect self-control (Fox et al., 2019; Peterson & Kirkpatrick, 2016; Renda et al., 2018; Stuebing et al., 2018). A shallower HFD function would have closely approximated HF0, which did not improve self-control in Experiment 1. Thus, more extreme parameter values would likely produce results outside of therapeutic range. Alternatively, perhaps any distribution with a significant percentage of very short delays coupled with relatively infrequent long delays may be sufficient to produce these effects. Future research should investigate this proposed synergy between value driven by short delays and tolerance driven by long delays.

Finally, the use of an abbreviated choice task to assess intervention effects may have affected choice results. We previously compared the abbreviated task against an extended task where training at each delay occurred over 10 sessions (Panfil et al., 2020; see also Foscue et al., 2012). Specifically, rats received either an FI intervention or no-delay control and were tested on both extended and abbreviated tasks. The intervention produced similar effects on both choice tasks. Also, there were significant positive correlations in individual performance on the tasks, suggesting that the measurements were comparable (Panfil et al., 2020; see also Foscue et al., 2012). In the current studies, Cronbach’s alpha values indicated an acceptable level of internal consistency, suggesting that the abbreviated task showed good internal reliability across choice delays.

However, choice behavior during the first four sessions of the impulsive choice task may not have reached stability (see Appendix). Generally, performance in these sessions tended to predict performance for the whole choice function for many rats. Additional examination of within-session changes in choice was conducted (data not shown). In both experiments, rats adapted to changes in delay more quickly with each session over the course of training, indicating that within- and between-session effects were consistent across conditions. Altogether, this suggests rats learned to make more self-controlled choices within each session as their experience with the task increased. Session and SS delay conveyed similar information because the SS delay increased across sessions, so additional analyses of session effects were not included here. In sum, the abbreviated task used in the current experiments showed noteworthy variability in choice behavior, which added complexity to understanding the intervention effects. Thus, results should be considered with a degree of caution as the choice task may have affected the outcomes. However, it is unlikely that the choice task itself produced the significant differences in Experiment 1, but it may have reduced sensitivity to detect more subtle differences between conditions in Experiment 2.

Although there may be some drawbacks to abbreviated choice tasks, there are also some potential advantages. Abbreviated tasks likely avoid overtraining which could induce choice habits. Abbreviated tasks are also more favorable for neuroscience research limited by amount of testing that can be delivered. Finally, abbreviated tasks may be more comparable to impulsive-choice tasks in humans which tend to have limited observations, thus increasing translational application. Overall, benefits and potential drawbacks should be considering when selecting choice tasks.

In conclusion, this research aligned with previous literature showing successful intervention effects with variable intervals (Bailey et al., 2018; Binder et al., 2000; Fox et al., 2019; Panfil et al., 2020; Renda & Madden, 2016; Renda et al., 2018; Smith et al., 2015; Stein et al., 2013) and expanded upon potential properties that may contribute to intervention efficacy. The hazard function property may not be the primary factor in time-based interventions using a VI schedule as originally proposed, apart from HF0. The probability density, or distribution of delays, may be an important factor, but clearly not the sole mechanism affecting timing and choice behavior. The HFD intervention may increase self-control by increasing subjective value of the LL choice and/or by engendering delay tolerance by exposure to occasional long delays. The HFD intervention may be an alternative intervention for specific populations such as individuals with extreme levels of impulsive choice that are less receptive to other time-based interventions such as the FI intervention. With more research, this line of work on properties of time-based interventions could result in novel translational applications.

Appendix

Group and Individual-Subject Figures for all Sessions of the Impulsive Choice Task

Figure A1 shows proportion of LL choices per group for each session of the choice task both pre- and postintervention in Experiment 1. Figures A2–4 depict proportion of LL choices for each subject across the impulsive choice task before and after the interventions in Experiment 1. Figure A5 shows proportion of LL choices per group for each session of the choice task both pre- and postintervention in Experiment 2. Figures A6–7 depict proportion of LL choices for each subject across the impulsive choice task before and after the interventions in Experiment 2.

Figure A1. All Sessions of LL Choices Pre- and Postintervention for Experiment 1.

Note. Mean proportion of LL choices pre- and postintervention for each group from Experiment 1 as a function of each session at each SS Delay for each group.

Figure A2. HFD Individual Graphs.

Note. Proportion of LL choices pre- and postintervention as a function of each session at each SS Delay for each individual rat in the hazard function decreasing (HFD) group from Experiment 1.

Figure A3. HFI Individual Graphs.

Note. Proportion of LL choices pre- and postintervention as a function of each session at each SS Delay for each individual rat in the hazard function increasing (HFI) group from Experiment 1.

Figure A4. HF0 Individual Graphs.

Note. Proportion of LL choices pre- and postintervention as a function of each session at each SS Delay for each individual rat in the hazard function zero (HF0) group from Experiment 1.

Figure A5. All Sessions of LL Choices Pre- and Postintervention for Experiment 2.

Note. Mean proportion of LL choices pre- and postintervention for each group from Experiment 2 as a function of each session at each SS Delay for each group.

Figure A6. SH Individual Graphs.

Note. Proportion of LL choices pre- and postintervention as a function of each session at each SS Delay for each individual rat in the shallow (SH) group from Experiment 2.

Figure A7. ST Individual Graphs.

Note. Proportion of LL choices pre- and postintervention as a function of each session at each SS Delay for each individual rat in the steep (ST) group from Experiment 2. Note that rat 11 did not complete any trials on one session of the preintervention choice assessment, and rat 5 did not complete any trials in two sessions of the postintervention choice assessment.