Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2025 Nov 1.
Published in final edited form as: J Exp Anal Behav. 2024 Oct 15;122(3):270–281. doi: 10.1002/jeab.4211

Using sign tracking to experimentally increase self-control in rats

Saba Mahmoudi 1, Gregory J Madden 1
PMCID: PMC11570336  NIHMSID: NIHMS2022069  PMID: 39406694

Abstract

Impulsive choice describes a preference for a smaller-sooner reward (SSR) over a larger-later reward (LLR). A large body of research has examined different procedures for decreasing impulsive choice in nonhuman subjects. One limitation of these procedures is the extensive training duration required to achieve the desired results. To address this limitation, the current experiment examined the effects of a brief course of Pavlovian training, designed to establish a conditioned stimulus (CS) that could be strategically used to encourage LLR choices. Forty male Long-Evans rats were randomly assigned to appetitive Pavlovian or unpaired training. A lever insertion signaled an upcoming unconditioned stimulus (i.e., food presentation) for Pavlovian rats and it acquired CS properties. The lever was uncorrelated with the US in the unpaired group, and it did not acquire CS properties. In the subsequent impulsive-choice assessment, the lever from the training phase served as the lever rats pressed to choose the LLR. After an LLR choice, the lever remained in the chamber during the delay to the LLR, just as the SSR lever remained in the chamber until that reward was delivered. Pavlovian-trained rats sign tracked toward the lever CS and made significantly fewer impulsive choices than did rats in the unpaired group.

Keywords: impulsivity, Pavlovian conditioning, sign-tracking, translational behavior analysis

An impulsive choice occurs when a smaller-sooner reward (SSR; e.g., $50 now) is preferred over a larger-later reward (LLR; $100 in a month; Ainslie, 1975). One behavioral mechanism that can underlie impulsive choice is discounting the value of the LLR (i.e., delay discounting; Odum, 2011). In humans, steep delay discounting is positively correlated with a variety of maladaptive behaviors such as overeating, substance use disorders, gambling, and risky sexual behaviors (Alessi & Petry, 2003; Amlung et al., 2016; Chesson et al., 2006; Odum et al., 2000). In addition, steep delay discounting is predictive of early drug use (Audrain-McGovern et al., 2009; Brody et al., 2014; Fernie et al., 2013; Khurana et al., 2013; Kluwe-Schiavon et al., 2020) and relapse to problem drug use (Coughlin et al., 2018; Harvanko et al., 2019; MacKillop & Kahler, 2009; Sheffer et al., 2014). Interventions that reduce delay discounting have positive effects on human cigarette smoking, alcohol consumption, and consumption of obesogenic foods (Athamneh et al., 2020; Daniel et al., 2013; Snider et al., 2016; Stein et al., 2016; Sze et al., 2017). The prevalence of these maladaptive behaviors and the potential of interventions that reduce delay discounting to reduce these behaviors justifies further investment and experimental assessments.

An extensive body of research has explored methods for reducing impulsive choice and delay discounting in nonhuman subjects (Rung & Madden, 2018; T. Smith et al., 2019). Successful experimental manipulations include reward bundling (Ainslie & Monterosso, 2003), temporal training (Bailey et al., 2018; Panfil et al., 2020; Peterson & Kirkpatrick, 2016), and exposure to delayed reinforcement contingencies (e.g., Renda et al., 2018, 2021). In the case of reward bundling, impulsive choice is reduced when a single choice commits the subject to SSRs on the next several trials; conversely, a single LLR choice locks in LLRs for several trials. Extended exposure to bundling contingencies can produce lasting reductions in impulsive choice (Stein et al., 2013). Likewise, extended exposure to temporal training (e.g., a history of responding on fixed-interval schedules) and delayed reinforcement contingencies (waiting for food after lever pressing with no ongoing operant contingency) can produce lasting reductions in impulsive choice (e.g., Bailey et al., 2018; Renda & Madden, 2016). If these procedures have a shortcoming, it is the extended duration of the training, typically lasting 60–120 sessions, each usually composed of > 70 training trials; e.g., Rung et al., 2018; A. Smith et al., 2015). This may make these procedures impractical if translated into interventions designed to reduce steady-state impulsive choice in humans. Novel approaches may be needed to reduce these training durations.

One category of learning that has not been explicitly evaluated in service of reducing impulsive choice is Pavlovian learning. Pavlovian learning occurs when a neutral stimulus (NS) acquires a conditioned stimulus (CS) function, which is to say that the CS alone evokes one or more conditioned responses (CRs). The CS acquires this response-evoking function because it signals that a phylogenetically important unconditioned stimulus (US) is imminent (Pavlov, 1927). For example, if free food (the US) occurs once, on average, every 90 s, and a lever is inserted into the chamber 8 s before each of these food events, the lever CS will evoke Pavlovian CRs such as positive-affect vocalizations, entering the feeder aperture before food is delivered, and approaching and interacting with the lever CS (Anderson & Spear, 2011; Meyer et al., 2014; Robinson & Flagel, 2009; Sangarapillai et al., 2021). The last of these CRs (approaching and interacting with the lever CS), commonly referred to as “sign-tracking,”1 might prove useful in decreasing impulsive choice. Specifically, if Pavlovian-trained rats are attracted to and interact with a lever CS, that lever might be arranged as the lever that, when pressed, produces the LLR in a test of impulsive choice. This may increase self-control choice.

One reason to consider Pavlovian conditioning as a strategy for reducing impulsive choice is that it can be completed quickly. When training procedures are optimized (see below), sign tracking (and other CRs) can be established in under 250 training trials (Anselme et al., 2013; Lee et al., 2018; Mahmoudi et al., 2023), which is far more efficient than the 2,000–12,000 training trials typical of extant methods for reducing impulsive choice in nonhumans (e.g., Peck et al., 2020; A. Smith et al., 2015). Thus, Pavlovian procedures may prove useful in increasing the practicality and acceptability of impulsivity interventions that might one day be used with human participants. Although Pavlovian sign tracking has been demonstrated in human participants (e.g., Colaizzi et al., 2023; Garofalo & di Pellegrino, 2015), this area is underresearched (Felix & Flagel, 2024).

To rapidly establish Pavlovian conditioned responding in general, and sign tracking in particular, the would-be CS should signal a large reduction in the average delay to the US (for a review, see Gibbon et al., 1977). In the earlier example, the average delay to the US was 90 s, and the 8-s insertion of the lever CS signaled a 90/8 or an 11.25-fold reduction in the delay to food. If the delay-reduction signaled by the CS was less substantial (e.g., from 16 s to 8 s, a 2-fold reduction in the signaled delay-reduction) Pavlovian conditioning would proceed more slowly and sign tracking would be rare (e.g., Bersh, 1951; Lee et al., 2018; Thomas & Papini, 2020).

A second procedure for producing robust sign tracking is to introduce uncertainty about the US by making its post-CS occurrence probabilistic and varying the magnitude of the US (i.e., the number of food pellets obtained; Anselme et al., 2013; Robinson et al., 2014, 2019). These are designed to make the US more surprising when it occurs after the CS. According to “incentive hope” theory (Anselme, 2013, these surprises produce greater midbrain dopamine activation, which enhances the incentive salience of the CS.

Krank et al. (2008) evaluated whether Pavlovian CSs could influence choice in a free-operant concurrent-schedules arrangement with variable-interval schedules of ethanol reinforcement. When the cue-light CS was presented above the lever that the rat was responding on, it increased the response rate, a Pavlovian-to-instrumental transfer effect. However, if the cue-light CS was presented over the lever the rat was not actively responding on, it did not attract the rat to that lever. Perhaps the CS would be more effective at influencing choice in a discrete-trials procedure (like those arranged in impulsive-choice tasks), as the CS could be presented before a choice is made and operant responding is underway.

The purpose of the present experiment was to use these Pavlovian procedures to produce robust sign tracking to a lever CS in Pavlovian-trained rats and then to evaluate whether this attraction to the lever would increase self-control choice when, in the impulsive-choice task, pressing the lever CS produced the LLR. A second group of rats first completed comparable training sessions in which the inserted lever was unpaired with the food US. Steady-state choice was assessed in both groups of rats. Although several studies have evaluated the operant choices made by rats that are biologically predisposed toward sign tracking (e.g., Tunstall & Kearns, 2015), to our knowledge this is the first attempt to experimentally establish sign tracking and then evaluate whether it can increase self-control choice.

METHODS

Subjects

Male Long-Evans rats (N = 40; 70-days old at the beginning of the experiment) were bred and reared by our research group from animals purchased from Envigo on June 10, 2021. The animals were weaned at 21 days of age and pair-housed by sex until they were 60 days old, with possible litter effects controlled for by randomizing litter assignment (20 litters) across groups. To avoid inbreeding we kept a record of the male and female rats used to breed each litter and ensured that paired rats were not related to each other. Subjects were randomly assigned to either the Pavlovian training (PAV; n = 20) or the unpaired control group (UN; n = 20). Only male subjects were used to reduce animal use in the initial phase of this research line. The sample size was informed by a power analysis (G*Power2) of anticipated group differences in sign tracking, smaller expected effects on impulsive-choice outcomes, and unexpected mortalities.

Ten days prior to the experiment, rats were single housed in cages (46 × 25 × 21 cm) with free access to bedding and enrichment items. The rats were kept under a 12 h light : 12 h dark cycle and maintained at approximately 85% of their free-feeding growth weights, with unlimited access to water in the home cage. Sessions were conducted across four daytime shifts, the first beginning 2 hr into the light cycle and the last ending approximately 3 hr before the dark cycle. The study was approved by the Institutional Animal Care and Use Committee (IACUC) at Utah State University (protocol # 11747).

Materials and apparatus

Sessions were conducted in six Med-Associates operant chambers (St. Albans, VT), each equipped with two retractable levers that were installed on the left and right panels of the front wall (6.5 cm from the grid floor) with a pellet receptacle located between them; the receptacle was equipped with a head-entry detector, and a Med Associates pellet dispenser was used to deliver 45-mg grain-based pellets (#F0165, Bio-Serv, Flemington, NJ). The chambers were also equipped with a houselight and a white noise speaker (80 dB) located on the rear wall. The operant chambers were enclosed by sound-attenuating enclosures (Med-Associates).

Procedures

Throughout the experiment, the onset and offset of the white-noise speaker signaled the beginning and end of the session.

Magazine training

The rats completed one 40-trial session in which they learned to eat pellets from the food receptacle. During these sessions, two pellets were delivered at a time, according to a variable-time (VT) 120-s schedule, with a rectangular distribution; the minimum interfood interval was 20 s. The head-entry detector recorded when and how often the rat entered the receptacle.

Group assignment

To ensure that there were no known differences between the two groups prior to training, the rats were block-randomly assigned to either the PAV or UN group; the criterion for blocking was the number of head entries during magazine training.

Training

Pavlovian group (PAV)

Rats in the PAV group completed eight daily Pavlovian training sessions. Sessions (approximately 90 min each) consisted of 24 trials, each beginning with the 8-s presentation of a lever located on the front wall of the chamber. For each rat, only one lever was ever inserted during the training phase, with its left or right location counterbalanced between subjects, within groups. These CS presentations were controlled by a VT 120-s schedule seeded with intervals from a rectangular distribution ranging from 9 to 231 s. After the 8-s CS, there was a 0.5 probability of food being delivered (either 1, 2, or 4 pellets; 0.33 probability of each) and a 0.50 probability that the CS terminated without the food US (total pellets per session = 28).

Unpaired-control group (UN)

Rats in the UN group completed eight daily sessions of unpaired training (90 min each). One VT 120-s timer (same distribution as above) controlled the 8-s lever presentations (24 per session). A second VT 240-s timer independently controlled food delivery (1, 2, or 4 pellets; 0.33 probability each), thereby holding constant across groups the frequency of the US and lever presentations and the number of pellets (28) per session. If a US event was scheduled to occur within 5 s of the time that the lever was in the chamber, the US presentation was postponed until 5 s after the lever was retracted.

Measuring sign tracking

To quantify sign tracking to the lever CS, we used the Pavlovian conditioned approach (PCA) index, a widely used and validated measure of sign tracking (Ahrens et al., 2016; Haight et al., 2017; Keefer et al., 2020; Lee et al., 2018; López et al., 2018; Meyer et al., 2012). PCA index values range from −1 to 1, with values exceeding 0.50 indicative of sign-tracking. These values are obtained by taking the average of the three measures shown in Table 1. The first is response bias, which quantifies the number of presses made on the lever CS relative to the number of times the feeder aperture is entered during the CS (prior to the US). The second component is the difference in the probability of a trial containing a lever press versus a feeder entry. The final component, latency, returns positive values if the lever is pressed before the feeder is entered; values are negative when the opposite response sequence occurs.

TABLE 1.

Formulas for calculating PCA index and component scores.

Response bias (lever presses − feeder entries) / (lever presses + feeder entries)
Probability (trials with a lever press / total trials) − (trials with a feeder entry / total trials)
Latency (average latency to enter the feeder − average latency to press the lever) / CS duration
PCA Index (response bias + probability + latency) / 3
Open in a new tab
Test of conditioned reinforcement

Following the training phase, to determine whether the lever had acquired conditioned reinforcing properties, all rats completed a single 30-min test session that arranged procedures that are commonly used in the Pavlovian literature (e.g., Beckmann & Chow, 2015; Robinson et al., 2019; Robinson & Flagel, 2009). Specifically, the food cup in the center of the intelligence panel was removed and replaced with a retractable lever, with two nose-poke ports, installed on either side of the lever. Sessions began with the illumination of both nose-poke ports. Contingent on a single response in the active nose-poke port (location randomly assigned between rats), both nose-poke lights were extinguished and the lever was inserted for 8 s. After 8 s, the lever was retracted, and both nose-poke ports were illuminated again. Responses in the inactive nose-poke port and on the lever were recorded but had no programmed consequence.

Retraining

Following the test of conditioned reinforcement, the lever was returned to its position in the training phase and the food cup was reinstalled. The rats then completed a minimum of two retraining sessions. The procedures were identical to those in the training phase. Retraining ended when (a) the subject’s two-session average PCA index score was within 0.2 of their score in the final two sessions of the training phase or (b) 15 sessions were completed, whichever came first.

Impulsive-choice task

Next, the rats made choices between an SSR and an LLR in an impulsive-choice task that has frequently been used to evaluate interventions designed to increase self-control in rats (e.g., Panfil et al., 2020; T. Smith et al., 2022, 2024). For this phase, a second retractable lever was installed on the other side of the food cup. No additional lever-press training was provided or required (all rats completed all forced-choice trials in the first intertemporal choice session). Sessions consisted of six forced-choice (trials 1, 2, 7, 8, 13, and 14) and 14 free-choice trials. All trials began with the illumination of the houselight and the insertion of one or two of the side levers. On forced-LLR trials, the lever from the training phase was inserted into the chamber. When the rat pressed that lever once, a 20-s delay was initiated, and the lever remained extended with the houselight on for the duration of the delay. After the delay, the lever was retracted, two pellets were delivered, and the houselight was turned off. On forced-SSR trials, the other lever was inserted; pressing it once initiated a different delay in which the lever remained in the chamber and the houselight remained on. At the end of the delay, the lever was retracted, a single food pellet was delivered, and the houselight was turned off. Free-choice trials were identical to forced-choice trials with the exception that (a) both levers were inserted at the beginning of the trial and (b) the unselected lever was retracted during the delay to the chosen reward. A postfood blackout (both levers retracted, and houselight off) ensured that trials began every 30 s regardless of the reward chosen. With 20 trials lasting 30 s each, sessions were completed in approximately 10 min.

The delay to the SSR was manipulated in an ascending sequence (5, 10, 15, and 20 s) across phases within the impulsive-choice assessment (e.g., Bailey et al., 2018; Panfil et al., 2020). Phases lasted a minimum of eight sessions and until choice stabilized in accord with the following criteria: (a) no monotonic trend over the last four sessions and (b) the mean LLR choice percentage in the last two sessions did not deviate from the mean of the preceding two sessions by more than 10%. Phases ended after 24 sessions if these criteria were not met.

Data analysis

Statistical analyses were completed using GraphPad Prism version 8.3.0 for Windows (GraphPad Software, San Diego, California USA) and R version 4.2.2 (R Core Team, 2022). Multilevel model (MLM) analyses were conducted in R using the lme4 package (Bates et al., 2015). The full data set and analysis files are available on Open Science Framework at https://osf.io/cxjaz/?view_only=90eef855c9344ae38356c4428a152d28

One subject from the UN group preferred the smaller reward amount in the final phase of the impulsive choice assessment (i.e., 20-s delay to both rewards). This persistent side bias could not be ameliorated with procedures used in prior experiments conducted in our lab (e.g., Renda et al., 2021). Therefore, this rat’s data were excluded from all analyses.

The PCA index scores in the final two training sessions were nonnormally distributed (Shapiro–Wilk tests p < .0001). Therefore, group differences were assessed with a Mann–Whitney nonparametric test. Differences between groups in lever presses across training were measured using multilevel model analysis (MLM; see Appendix A for the model’s equation).

In the test of conditioned reinforcement, distributions of responses in the inactive and active ports were similarly nonnormal (Shapiro–Wilk tests p < .0001), so group differences were measured using MLM analysis. In the impulsive-choice task, the values for average percentage of LLR choice (free-choice trials) in the stable sessions at each delay were analyzed using the MLM analysis because it is a better method when there are groups with an unequal number of subjects (Hox et al., 2017). In addition, the choice data are multilevel by nature: repeated measure of LLR across the four phases (different SSR delays; Level 1) nested within subjects (Level 2). Group differences in the rate of lever pressing during the delays to the SSR and LLR in the test of impulsive choice were also assessed using MLM (equations provided in Appendix A).

Four hypothesis-driven one-tailed correlation analyses were conducted. The data used in these analyses were not normally distributed (Shapiro–Wilk tests p < .01), so Spearman’s r was used. Two analyses evaluated whether the rats’ PCA index values in the training and retraining phases were predictive of their stable choices in the impulsive-choice assessment; the latter was quantified as area under the choice curve (AUC; range 0 to 1.0), with larger values indicative of more LLR choices. The third analysis evaluated whether conditioned-reinforcer efficacy (quantified as the difference in the number of active and inactive nose pokes made in the test of conditioned reinforcement) was predictive of AUC in the impulsive-choice assessment. The final analysis assessed the correlation between PCA index values and conditioned-reinforcer efficacy.

RESULTS

As depicted in Figure 1A, PCA index scores were significantly higher (p < .001; i.e., more sign tracking) in the PAV group (M = 0.5, SEM = 0.1) than in the UN group (M = −0.6, SEM = 0.05). Figure 1B shows the median lever presses per session during the training phase for both groups. The MLM analysis indicated a main effect of session (p < .001) and a significant session by group interaction (p < .001; see Appendix B for all MLM standardized coefficients and test statistics). Pairwise comparison showed that the PAV group pressed the lever significantly more than the UN group during Sessions 3–8 (ps < .019); PAV rats reached an asymptotic rate by the fourth session. In the test of conditioned reinforcement that followed (Figure 1C), PAV rats made significantly more responses in the active port than UN rats (M difference = 40.6, SEM = 7.02; p < .001). There was no significant difference between groups in responses made in the inactive port (M difference = 12.9, SEM = 7.02, p = .071). Thus, there was a significant Group × Port interaction (p = .002). Pairwise comparisons revealed that rats in the PAV group responded significantly more in the active than in the inactive port (M difference = 17.3, SEM = 5.7, p = .004). In contrast, UN rats’ responding was not significantly different across ports (M difference = −10.5, SEM = 5.85, p = .082). At the conclusion of the retraining phase, PCA index scores were, once again, significantly higher in the PAV (M = 0.6, SEM = 0.09) than the UN group (M = −0.6, SEM = 0.08, p < .001; Figure 1D). Finally, Figure 1E reveals that PCA index values prior to the test of conditioned reinforcement were significantly correlated with the conditioned-reinforcing efficacy of the CS (i.e., the difference between responses made in the active and inactive nose-poke ports; Spearman’s r = .57, p < .001).

FIGURE 1.

FIGURE 1

Open in a new tab

Panel A: Mean (± SD) PCA index in the last two sessions of the training phase; data separated by group. Panel B: Median and IQR of lever presses per session during the training phase. Panel C: Mean (± SD) and number of responses in the active and inactive ports in the test of conditioned reinforcement. Panel D: PCA index in final two session of the retraining phase. Panel E: Correlation between conditioned reinforcement and PCA index. *p < 0.05; ****p < 0.0001.

As shown in Figure 2A, in the impulsive-choice assessment there was a significant main effect of delay (p < .001), with LLR choices increasing across the delays to the SSR regardless of group. Likewise, there was a significant main effect of group (p = .016), with rats in the PAV group making more self-control choices than UN rats across the range of delays. The Group × Delay interaction was significant (p = .005), with pairwise comparisons revealing significant group differences in choice at the 5-s (M difference 29.03, SEM = 8.6, p = .001), 10-s (M difference 19.10, SEM = 8.6, p = .03), and 15-s delays (M difference 22.72, SEM = 8.6, p = .010). When both rewards were delayed by 20 s, there was no significant difference in choices made between groups (M difference 2.68, SEM = 8.6, p = .756); they both preferred the larger (2-pellet) reward.

FIGURE 2.

FIGURE 2

Open in a new tab

Panel A: Mean and individual subjects; percent choice of the larger-later reward, plotted as a function of delay to the smaller-sooner reward; data separated by Pavlovian-trained (PAV) and unpaired (UN) groups. Panel B: Rate of lever pressing during the delay to the SSR and LLR; data separated by group. *p < 0.05; **p < 0.01.

Figure 2B shows the median rate of lever pressing during the delay to the SSR and LLR in the test of impulsive choice. Data are separated by group, lever (SSR or LLR), and the delay to the SSR (x-axis). In addition, rates are presented from the initial four sessions and the final four sessions of each phase. There was a significant Group × Lever × Delay interaction (p = .035), which is visually evident with UN rats pressing the LLR lever less than PAV rats when the delay to the SSR was 5 s. Within-group analyses revealed that UN rats responded significantly slower on the LLR lever than the SSR lever during the initial four sessions of this first phase (M difference 0.54, SEM = 0.1, p = < .001). Thereafter, there were no significant within- or between-groups differences in rates of lever pressing (ps > .091).

Figure 3A shows the AUC values from the impulsive-choice assessment plotted as a function of the PCA index values obtained in the final two sessions of the training phase. The positive correlation is significant (Spearman’s r = .29, p = .035). Figure 3B illustrates that the conditioned-reinforcing efficacy of the CS was not predictive of AUC in the test of impulsive choice (Spearman’s r = .08, p = .305). Similar to the correlation from the initial training phase, Figure 3C shows that PCA index values in the retraining phase were significantly positively correlated with AUC (Spearman’s r = .33, p = .021). One PAV-trained rat was an outlier, with low PCA index values (no sign tracking) at the end of training and retraining. This rat had the second-lowest AUC value among all rats in the experiment.

FIGURE 3.

FIGURE 3

Open in a new tab

Panel A: Correlation between area under the curve (AUC) in the impulsive-choice test and PCA index obtained in the final two sessions of the training phase. Data separated by Pavlovian-trained (PAV) and unpaired (UN) groups in all panels. Panel B: Correlation between AUC and the measure of conditioned reinforcement (the number of nose pokes made in the active port [produced the CS] minus the pokes made in the inactive port). Panel C: Correlation between AUC and PCA index in the retraining phase. Higher AUC values indicate a higher percentage of LLR choices.

DISCUSSION

The results of the present experiment demonstrate, for the first time, that a Pavlovian CS that evokes sign tracking (PCA index > 0.50) can be used to significantly increase steady-state preference for the LLR (i.e., self-control choice) relative to a control group. The amount of sign tracking during the Pavlovian conditioning phase was a significant predictor of subsequent self-control choice. This finding illustrates one way in which Pavlovian conditioning can be used to quickly decrease impulsive choice in rats. Given the efficiency of Pavlovian conditioning, future experiments should explore other Pavlovian techniques that may produce comparable outcomes (e.g., presenting the CS as a contingent consequence of making a self-control choice).

We will make four points about these findings. First, the difference in self-control choices made across groups is not due to a difference in sensitivity to differences in reward amounts (i.e., 1 vs. 2 pellets). This was evaluated in the final phase of the impulsive-choice assessment when the delay to the 1-pellet reward was the same as the delay to the 2-pellet reward (20 s). In this phase, the strength of preference for the larger reward is due to the difference in reward amount, as this is the only difference between the choice alternatives (Bailey et al., 2018; Marshall et al., 2014; Panfil et al., 2020). The difference in choice between groups in this final phase did not approach significance; therefore, there is no evidence to suggest that greater self-control choice in the PAV group was due to a difference in sensitivity to reinforcer amount.

The second discussion point concerns the possible functions of the levers during the impulsive-choice assessment. For PAV rats, the LLR lever functioned as a Pavlovian antecedent attractor (it evoked sign tracking), which drew these rats to the LLR lever and may have played a role in the LLR choice response. After these rats made an LLR choice, the lever remained in the chamber throughout the delay to food. Because access to this lever previously functioned as a conditioned reinforcer for these PAV rats (Figure 1C), it is possible that the significant increase in self-control choice was partially due to a conditioned reinforcement effect—that is, continued access to the LLR-lever was contingent on making a self-control choice—and this contingency may play a role in the greater prevalence of that choice in PAV rats. This hypothesis could be tested empirically by repeating the experiment but retracting the LLR-lever when that lever is pressed in the impulsive-choice test. If this produces the same increase in self-control choice, then the conditioned-reinforcement hypothesis would not be supported. Short of that empirical test, the data in Figure 3B might be viewed as unsupportive of this hypothesis. That is, the conditioned reinforcing efficacy of the CS (or lack thereof) was not a significant predictor of self-control choice (AUC). Thus, rats that responded the most to access the lever during the test of conditioned reinforcement were no more likely to choose the LLR during the impulsive-choice assessment. Instead, the amount of sign-tracking significantly predicted the prevalence of self-control choice (Figures 3A and 3C). Nonetheless, empirical tests of the conditioned reinforcement hypothesis should be conducted.

It is also important to consider the possible functions of the levers for rats in the UN group. During the training phase, what would eventually be the LLR lever was never presented within 5 s of the food US. Thus, it is possible that for UN rats the LLR lever acquired Pavlovian inhibitory properties (Cunningham et al., 1977; Rescorla, 1969). That is, UN rats may have avoided the LLR lever in the impulsive-choice assessment because it was uncorrelated with food in the prior phase. If this hypothesis is correct, then UN rats would be expected to avoid the LLR lever in the test of impulsive choice. Figure 2B shows that UN rats responded at a lower rate on the LLR in the first four sessions of the test of impulsive choice (5 s – Initial), but by the end of that phase, and throughout the remaining phases, UN rats responded at comparable rates on both levers during delays to the SSR and LLR. Although not definitive, this analysis offers no evidence to support the hypothesis that conditioned inhibition in UN rats is responsible for the steady state difference in impulsive choice between PAV and UN rats. That said, the present data cannot rule out the possibility that the significant difference in impulsive choice between PAV and UN groups was at least partially driven by UN rat’s aversion to the LLR lever. Future experiments should arrange for control rats to complete truly random control procedures in which lever and food presentations are controlled by separate VT timers with no restrictions on the co-occurrence of these events (Rescorla, 1969).

The third point for discussion concerns the external validity of these findings. Can the results be replicated in other tests of impulsive choice (e.g., Evenden & Ryan, 1996), with a different sequence of delays across sessions, in other species, or, importantly, in female rats? If male rats are more likely to sign track to a CS than female rats, then we might expect a diminished effect on impulsive choice. Studies assessing sex effects on sign tracking are mixed, with some studies reporting higher CS-directed behavior in females (Fattore et al., 2014; Hammerslag & Gulley, 2013; van Haaren et al., 1987), one reporting more robust sign tracking in males (Hellberg et al., 2018), and another reporting no difference between males and females (Pitchers et al., 2015). If females are somewhat more likely to develop sign tracking, then we would expect a greater reduction in impulsive choice in females, as they would be even more attracted to the CS leading to the LLR. That said, one might be concerned that Pavlovian training would be less able to reduce impulsivity if female rats were more impulsive than males. The modal finding in the literature, however, is that there are no sex differences in rodent impulsive choice (Burton & Fletcher, 2012; Doremus-Fitzwater et al., 2012; Eubig et al., 2014; Lukkes et al., 2015; Orsini & Setlow, 2017; Perry et al., 2007, 2008; Weafer & de Wit, 2014). Nonetheless, future studies should (a) assess whether a CS that evokes sign tracking can be used as an antecedent attractor to promote self-control in female rats and (b) evaluate the replicability of this effect with other tests of impulsive choice.

Finally, it is worth underscoring that the period of Pavlovian training needed to produce these significant improvements in self-control choice—192 trials in eight sessions—is brief relative to other laboratory procedures that produce comparable reductions in impulsive choice. For example, the delay-exposure procedures used in our lab also produce large, steady-state reductions in impulsive choice (Renda & Madden, 2016). However, to achieve a reduction as large as the one obtained in the present study (Cohen’s ds = 0.69), at least 30 sessions of delay-exposure training (2,400 trials) are required (Renda et al., 2021). The 12.5-fold reduction in laboratory training duration achieved in the present experiment is ideal for many reasons (conservation of time, labor, and resources). Likewise, an ideal intervention for reducing impulsive choice in humans would be one that is fast-acting, easy, and foolproof. If the Pavlovian approach proves to work quickly outside the lab with humans, it might also be easy because the CS→US sequence is not contingent on behavior, so behavior need not be monitored during training. The Pavlovian approach is not, however, foolproof. There are many ways to arrange NS→US sequences that will not lead to Pavlovian learning (needed for the NS to acquire CS properties). For example, if the individual presenting the NS precedes it with an attention-directing phase such as “1-2-3 eyes on me” this auditory stimulus may overshadow the NS, preventing it from acquiring the desired CS function. Likewise, if the NS signals a very small reduction in the delay to the US, it may take a very long time for the NS to acquire a CS function (see Madden et al., 2023, for further discussion and review of principles of effective Pavlovian conditioning).

If Pavlovian training were carried out effectively in an applied setting, then the caregiver/teacher might use the effects of training when asking a child if they would prefer to briefly play with a toy now (SSR) or instead hold the CS while waiting patiently for a longer duration play interval (LLR). Would seeing the CS increase the probability of a self-control choice? Would holding the CS while waiting increase the child’s ability to patiently wait? Would the CS provide the child with a sanctioned way of reminding the teacher that they are patiently waiting for their turn with the toy? A first step in assessing the efficacy of this intervention would be to establish effective human Pavlovian training procedures in applied settings.

To conclude, this is the first experiment to examine whether Pavlovian learning can be used to increase self-control choice. The results are promising because of the brief Pavlovian training duration and the large and significant reduction in impulsive choice. Future experiments should evaluate the independent contributions of the CS as a prechoice antecedent and/or as a postchoice stimulus in producing this reduction in impulsive choice. Such information may prove useful when translating these findings to interventions.

ACKNOWLEDGEMENTS

The authors would like to thank Sara Peck, Katherine C. Garland, Gabrielle M. Sutton, Joshua I. Jones, Sophia S. Sperber, Christine T. Layne, and Kelsey B. Smith for their assistance in conducting the experiment.

APPENDIX A

Equations use in the MLM Models

Lever presses across training Leverpresses < –lmerTest :: lmer(LeverPress ~ Group * SSN +(1|ID)
Test of conditioned reinforcement CR < –lmerTest :: lmer(Numberresponses ~ Group * port +(1|ID)
Impulsive choice task IC < –lmerTest :: lmer(Mper_ll ~ group * Delay + (1|id)
Lever presses during the impulsive choice task ICleverpress < –lmerTest :: lmer(Rate ~ Group*Lever*Block*Delay + (1|ID)
Open in a new tab

APPENDIX B

Standardized coefficient and test statistics for the MLM analyses

Table B1.

Standardized coefficient and test statistics for the number of lever presses across training sessions. Data were standardized using a z score.

Variable Standardized coefficient β t
Intercept −0.092 −0.477
  Group −0.258 −0.941
  Session 0.107 6.075
Interaction
  Group × Session −0.121 −4.775
Open in a new tab

TABLE B2.

Standardized coefficient and test statistics for the test of conditioned reinforcement. Data were standardized using a z score.

Variable Standardized coefficient β t
Intercept 0.823 4.428
  Port −0.657 −3.033
  Group −1.542 −5.792
Interaction
  Port × Group 1.054 3.398
Open in a new tab

Data were standardized using a z score.

TABLE B3.

Standardized coefficient and test statistics for the impulsive choice task. Data were standardized using a z score.

Variable Standardized coefficient β t
Intercept −0.345 −1.915
  Group −0.872 −3.377
  Delay 10 0.538 3.438
  Delay 15 0.829 5.307
  Delay 20 1.089 6.964
Interaction
  Group × Delay 10 0.298 1.331
  Group × Delay 15 0.189 0.845
  Group × Delay 20 0.791 3.532
Open in a new tab

TABLE B4.

Standardized coefficient and test statistics for the rate of lever presses during the impulsive choice task. Data were standardized using a z score.

Variable Standardized coefficient β t
Intercept −0.287 −0.728
  Group −2.364 −4.183
  Lever 0.013 0.025
  Session 0.262 1.128
  Delay 10 0.666 1.280
  Delay 15 0.512 0.985
  Delay 20 0.518 0.997
Interaction
  Group × Lever 2.251 3.022
  Group × Session 0.981 2.946
  Lever × Session 0.072 0.219
  Group × Delay 10 2.069 2.778
  Group × Delay 15 2.296 3.082
  Group × Delay 20 2.014 2.704
  Lever × Delay 10 −0.284 −0.387
  Lever × Delay 15 −0.357 −0.486
  Lever × Delay 20 −0.392 −0.533
  Session × Delay 10 −0.429 −1.306
  Session × Delay 15 −0.205 −0.622
  Session × Delay 20 −0.267 −0.812
  Group × Lever × Session −1.002 −2.127
  Group × Lever × Delay 10 −1.852 −1.758
  Group × Lever × Delay 15 −2.186 −2.075
  Group × Lever × Delay 20 −2.221 −2.109
  Group × Session × Delay 10 −0.819 −1.738
  Group × Session × Delay 15 −1.069 −2.269
  Group × Session × Delay 20 −1.045 −2.217
  Lever × Session × Delay 10 0.162 0.348
  Lever × Session × Delay 15 −0.025 −0.054
  Lever × Session × Delay 20 0.001 0.003
  Group × Lever × Block × Delay 10 0.651 0.977
  Group × Lever × Block × Delay 15 0.911 1.368
  Group × Lever × Block × Delay 20 1.093 1.641
Open in a new tab

Footnotes

1

Extensive evidence has revealed that rats’ sign tracking to a lever is not maintained with adventitious reinforcement; it is evoked by the CS (e.g., Davey et al. 1981).

CONFLICT OF INTEREST STATEMENT:

None of the authors have any real or potential conflict(s) of interest, including financial, personal, or other relationships with organizations or pharmaceutical companies that may inappropriately influence the research and interpretation of the findings.

ETHICS APPROVAL

The study was approved by the Institutional Animal Care and Use Committee (IACUC) at Utah State University (protocol # 11747).

DATA AVAILABILITY STATEMENT

The full data set and analysis files are available on the Open Science Framework at https://osf.io/cxjaz/?view_only=90eef855c9344ae38356c4428a152d28.

REFERENCES

  1. Ahrens AM, Singer BF, Fitzpatrick CJ, Morrow JD, & Robinson TE (2016). Rats that sign-track are resistant to Pavlovian but not instrumental extinction. Behavioural Brain Research, 296, 418–430. 10.1016/j.bbr.2015.07.055 [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Ainslie G (1975). Specious reward: A behavioral theory of impulsiveness and impulse control. Psychological Bulletin, 82(4), 463–496. [DOI] [PubMed] [Google Scholar]
  3. Ainslie G, & Monterosso JR (2003). Building blocks of self-control: Increased tolerance for delay with bundled rewards. Journal of the Experimental Analysis of Behavior, 79(1), 37–48. 10.1901/jeab.2003.79-37 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Alessi SM, & Petry NM (2003). Pathological gambling severity is associated with impulsivity in a delay discounting procedure. Behavioural Processes, 64(3), 345–354. 10.1016/S0376-6357(03)00150-5 [DOI] [PubMed] [Google Scholar]
  5. Amlung M, Vedelago L, Acker J, Balodis I, MacKillop J, Joseph S, & Hamilton H (2016). Steep delay discounting and addictive behavior: A meta-analysis of continuous associations. Addiction, 112(1), 51–62. 10.1111/add.13535 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Anderson RI, & Spear LP (2011). Autoshaping in adolescence enhances sign-tracking behavior in adulthood: Impact on ethanol consumption. Pharmacology Biochemistry and Behavior, 98(2), 250–260. 10.1016/J.PBB.2011.01.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Anselme P, Robinson MJF, & Berridge KC (2013). Reward uncertainty enhances incentive salience attribution as sign-tracking. Behavioural Brain Research, 238(1), 53–61. 10.1016/j.bbr.2012.10.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Athamneh LN, Stein JS, & Bickel WK (2020). Narrative theory III: Evolutionary narratives addressing mating motives change discounting and tobacco valuation. Experimental and Clinical Psychopharmacology, 28(3), 276–290. 10.1037/pha0000315 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Audrain-McGovern J, Rodriguez D, Epstein LH, Cuevas J, Rodgers K, & Wileyto EP (2009). Does delay discounting play an etiological role in smoking or is it a consequence of smoking? Drug and Alcohol Dependence, 103(3), 99–106. 10.1016/J.DRUGALCDEP.2008.12.019 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Bailey C, Peterson JR, Schnegelsiepen A, Stuebing SL, & Kirkpatrick K (2018). Durability and generalizability of time-based intervention effects on impulsive choice in rats. Behavioural Processes, 152, 54–62. 10.1016/J.BEPROC.2018.03.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Bates D, Mächler M, Bolker B, & Walker S (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. doi: 10.18637/jss.v067.i01. [DOI] [Google Scholar]
  12. Beckmann JS, & Chow JJ (2015). Isolating the incentive salience of reward-associated stimuli: Value, choice, and persistence. Learning and Memory, 22(2), 116–127. 10.1101/lm.037382.114 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Bersh PJ (1951). The influence of two variables upon the establishment of a secondary reinforcer for operant responses. Journal of Experimental Psychology, 41(1), 62–73. 10.1037/h0059386 [DOI] [PubMed] [Google Scholar]
  14. Brody GH, Yu T, Mackillop J, Miller GE, Chen E, Obasi EM, & Beach SRH (2014). Catecholamine levels and delay discounting forecast drug use among African American youths. Addiction, 109(7), 1112–1118. 10.1111/add.12516 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Burton CL, & Fletcher PJ (2012). Age and sex differences in impulsive action in rats: The role of dopamine and glutamate. Behavioural Brain Research, 230(1), 21–33. 10.1016/J.BBR.2012.01.046 [DOI] [PubMed] [Google Scholar]
  16. Chesson HW, Leichliter JS, Zimet GD, Rosenthal SL, Bernstein DI, & Fife KH (2006). Discount rates and risky sexual behaviors among teenagers and young adults. Journal of Risk Uncertainty, 32, 217–230. 10.1007/s11166-006-9520-1 [DOI] [Google Scholar]
  17. Colaizzi JM, Flagel SB, Gearhardt AN, Borowitz MA, Kuplicki R, Zotev V, Clark G, Coronado J, Abbott T, & Paulus MP (2023). The propensity to sign-track is associated with externalizing behavior and distinct patterns of reward-related brain activation in youth. Scientific Reports, 13(1), Article 4402. 10.1038/s41598-023-30906-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Coughlin LN, Tegge AN, Sheffer CE, & Bickel WK (2018). A machine-learning approach to predicting smoking cessation treatment outcomes. Nicotine & Tobacco Research, 22(3), 415–422. 10.1093/ntr/nty259 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Cunningham CL, Fitzgerald RD, & Francisco DL (1977). Excitatory and inhibitory consequences of explicitly unpaired and truly random conditioning procedures on heart rate in rats. Animal Learning & Behavior, 5(2), 135–142. [Google Scholar]
  20. Daniel TO, Stanton CM, & Epstein LH (2013). The future is now reducing impulsivity and energy intake using episodic future thinking. Psychological Science, 24(11), 2339–2342. 10.1177/0956797613488780 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Doremus-Fitzwater TL, Barreto M, & Spear LP (2012). Age-related differences in impulsivity among adolescent and adult Sprague-Dawley rats. Behavioral Neuroscience, 126(5), 735–741. 10.1037/a0029697 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Eubig PA, Noe TE, Floresco SB, Sable JJ, & Schantz SL (2014). Sex differences in response to amphetamine in adult Long-Evans rats performing a delay-discounting task. Pharmacology Biochemistry and Behavior, 118, 1–9. 10.1016/J.PBB.2013.12.021 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Evenden JL, & Ryan CN (1996). The pharmacology of impulsive behaviour in rats: The effects of drugs on response choice with varying delays of reinforcement. Psychopharmacology, 128(2), 161–170. 10.1007/S002130050121 [DOI] [PubMed] [Google Scholar]
  24. Fattore L, Melis M, Fadda P, & Fratta W (2014). Sex differences in addictive disorders. Frontiers in Neuroendocrinology, 35(3), 272–284. 10.1016/J.YFRNE.2014.04.003 [DOI] [PubMed] [Google Scholar]
  25. Felix PC, & Flagel SB (2024). Leveraging individual differences in cue–reward learning to investigate the psychological and neural basis of shared psychiatric symptomatology: The sign-tracker/goal-tracker model. Behavioral Neuroscience. Advance online publication. 10.1037/bne0000590 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Fernie G, Peeters M, Gullo MJ, Christiansen P, Cole JC, Sumnall H, & Field M (2013). Multiple behavioural impulsivity tasks predict prospective alcohol involvement in adolescents. Addiction, 108(11), 1916–1923. 10.1111/add.12283 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Garofalo S, & di Pellegrino G (2015). Individual differences in the influence of task-irrelevant Pavlovian cues on human behavior. Frontiers in Behavioral Neuroscience, 9, Article 00163. 10.3389/fnbeh.2015.00163 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Haight JL, Fuller ZL, Fraser KM, & Flagel SB (2017). A food-predictive cue attributed with incentive salience engages subcortical afferents and efferents of the paraventricular nucleus of the thalamus. Neuroscience, 340, 135–152. 10.1016/j.neuroscience.2016.10.043 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Hammerslag LR, & Gulley JM (2013). Age and sex differences in reward behavior in adolescent and adult rats. Developmental Psychobiology, 56(4), 611–621. 10.1002/dev.21127 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Harvanko AM, Strickland JC, Slone SA, Shelton BJ, & Reynolds BA (2019). Dimensions of impulsive behavior: Predicting contingency management treatment outcomes for adolescent smokers. Addictive Behaviors, 90, 334–340. 10.1016/J.ADDBEH.2018.11.031 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Hellberg SN, Levit JD, & Robinson MJF (2018). Under the influence: Effects of adolescent ethanol exposure and anxiety on motivation for uncertain gambling-like cues in male and female rats. Behavioural Brain Research, 337, 17–33. 10.1016/j.bbr.2017.09.036 [DOI] [PubMed] [Google Scholar]
  32. Hox JJ, Moerbeek M, & Van de Schoot R (2017). Multilevel analysis: Techniques and applications ( 3rd ed.). Routledge. [Google Scholar]
  33. Keefer SE, Bacharach SZ, Kochli DE, Chabot JM, & Calu DJ (2020). Effects of limited and extended Pavlovian training on devaluation sensitivity of sign- and goal-tracking rats. Frontiers in Behavioral Neuroscience, 14, 3, Article 00003. 10.3389/fnbeh.2020.00003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Khurana A, Romer D, Betancourt LM, Brodsky NL, Giannetta JM, & Hurt H (2013). Working memory ability predicts trajectories of early alcohol use in adolescents: The mediational role of impulsivity. Addiction, 108(3), 506–515. 10.1111/add.12001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Kluwe-Schiavon B, Viola TW, Sanvicente-Vieira B, Lumertz FS, Salum GA, Grassi-Oliveira R, & Quednow BB (2020). Substance related disorders are associated with impaired valuation of delayed gratification and feedback processing: A multilevel meta-analysis and meta-regression. Neuroscience & Biobehavioral Reviews, 108, 295–307. 10.1016/j.neubiorev.2019.11.016 [DOI] [PubMed] [Google Scholar]
  36. Krank MD, O’Neill S, Squarey K, & Jacob J (2008). Goal-and signal-directed incentive: conditioned approach, seeking, and consumption established with unsweetened alcohol in rats. Psychopharmacology, 196, 397–405 [DOI] [PubMed] [Google Scholar]
  37. Lee B, Gentry RN, Bissonette GB, Herman RJ, Mallon JJ, Bryden DW, Calu DJ, Schoenbaum G, Coutureau E, Marchand AR, Khamassi M, & Roesch MR (2018). Manipulating the revision of reward value during the intertrial interval increases sign tracking and dopamine release. PLoS Biology, 16(9), Article e2004015. 10.1371/journal.pbio.2004015 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. López P, Alba R, & Orduña V (2018). Individual differences in incentive salience attribution are not related to suboptimal choice in rats. Behavioural Brain Research, 341, 71–78. 10.1016/j.bbr.2017.12.028 [DOI] [PubMed] [Google Scholar]
  39. Lukkes JL, Thompson BS, Freund N, & Andersen SL (2015). The developmental inter-relationships between activity, novelty preferences, and delay discounting in male and female rats; the developmental inter-relationships between activity, novelty preferences, and delay discounting in male and female rats. Developmental Psychobiology, 58(2), 231–242. 10.1002/dev.21368 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. MacKillop J, & Kahler CW (2009). Delayed reward discounting predicts treatment response for heavy drinkers receiving smoking cessation treatment. Drug and Alcohol Dependence, 104(3), 197–203. 10.1016/J.DRUGALCDEP.2009.04.020 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Madden GJ, Mahmoudi S, & Brown K (2023). Pavlovian learning and conditioned reinforcement. Journal of Applied Behavior Analysis, 56(3), 498–519. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Mahmoudi S, Peck S, & Madden GJ (2023). Effects of inter-trial interval on sign-tracking and conditioned reinforcer efficacy in female rats. Behavioural Processes, 210, Article 104911. 10.1016/j.beproc.2023.104911 [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Marshall AT, Smith AP, & Kirkpatrick K (2014). Mechanisms of impulsive choice: I. Individual differences in interval timing and reward processing. Journal of the Experimental Analysis of Behavior, 102(1), 86–101. 10.1002/jeab.88 [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Meyer PJ, Cogan ES, & Robinson TE (2014). The form of a conditioned stimulus can influence the degree to which it acquires incentive motivational properties. PLoS ONE, 9(6), Article 0098163. 10.1371/journal.pone.0098163 [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Meyer PJ, Lovic V, Saunders BT, Yager LM, Flagel SB, Morrow JD, & Robinson TE (2012). Quantifying individual variation in the propensity to attribute incentive salience to reward cues. PLoS ONE, 7(6), Article 0038987. 10.1371/journal.pone.0038987 [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Odum AL (2011). Delay discounting: I’m a k, you’re a k. Journal of the Experimental Analysis of Behavior, 96(3), 427–439. 10.1901/jeab.2011.96-423 [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Odum AL, Madden GJ, Badger GJ, & Bickel WK (2000). Needle sharing in opioid-dependent outpatients: Psychological processes underlying risk. Drug and Alcohol Dependence, 60(3), 259–266. 10.1016/S0376-8716(00)00111-3 [DOI] [PubMed] [Google Scholar]
  48. Orsini CA, & Setlow B (2017). Sex differences in animal models of decision making. Journal of Neuroscience Research, 95(1–2), 260–269. 10.1002/JNR.23810 [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Panfil K, Bailey C, Davis I, Mains A, & Kirkpatrick K (2020). A time-based intervention to treat impulsivity in male and female rats. Behavioural Brain Research, 379, Article 112316. 10.1016/j.bbr.2019.112316 [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Pavlov IP (1927). Conditioned reflexes: An investigation of the physiological activity of the cerebral cortex. Annals of Neurosciences, 17(3), 136–141. 10.5214/ans.0972-7531.1017309 [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Peck S, Rung JM, Hinnenkamp JE, & Madden GJ (2020). Reducing impulsive choice: VI. Delay-exposure training reduces aversion to delay-signaling stimuli. Psychology of Addictive Behaviors, 34(1), 147–155. 10.1037/adb0000495 [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Perry JL, Nelson SE, Anderson MM, Morgan AD, & Carroll ME (2007). Impulsivity (delay discounting) for food and cocaine in male and female rats selectively bred for high and low saccharin intake. Pharmacology Biochemistry and Behavior, 86(4), 822–837. 10.1016/J.PBB.2007.03.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Perry JL, Nelson SE, & Carroll ME (2008). Impulsive choice as a predictor of acquisition of IV cocaine self-administration and reinstatement of cocaine-seeking behavior in male and female rats. Experimental and Clinical Psychopharmacology, 16(2), 165–177. 10.1037/1064-1297.16.2.165 [DOI] [PubMed] [Google Scholar]
  54. Peterson JR, & Kirkpatrick K (2016). The effects of a time-based intervention on experienced middle-aged rats. Behavioural Processes, 133, 44–51. 10.1016/j.beproc.2016.11.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Pitchers KK, Flagel SB, O’Donnell EG, Solberg Woods LC, Sarter M, & Robinson TE (2015). Individual variation in the propensity to attribute incentive salience to a food cue: Influence of sex. Behavioural Brain Research, 278, 462–469. 10.1016/j.bbr.2014.10.036 [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. R Core Team (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/ [Google Scholar]
  57. Renda CR, & Madden GJ (2016). Impulsive choice and pre-exposure to delays: III. Four-month test-retest outcomes in male Wistar rats. Behavioural Processes, 126, 108–112. 10.1016/j.beproc.2016.03.014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Renda CR, Rung JM, Hinnenkamp JE, Lenzini SN, & Madden GJ (2018). Impulsive choice and pre-exposure to delays: IV. Effects of delay- and immediacy-exposure training relative to maturational changes in impulsivity. Journal of the Experimental Analysis of Behavior, 109(3), 587–599. 10.1002/jeab.432 [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Renda CR, Rung JM, Peck S, & Madden GJ (2021). Reducing impulsive choice VII: Effects of duration of delay-exposure training. Animal Cognition, 24(1), 11–21. 10.1007/s10071-020-01412-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Rescorla RA (1969). Pavlovian conditioned inhibition. Psychological Bulletin, 72(2), 77–94. 10.1037/H0027760 [DOI] [Google Scholar]
  61. Robinson MJF, Anselme P, Fischer AM, & Berridge KC (2014). Initial uncertainty in Pavlovian reward prediction persistently elevates incentive salience and extends sign-tracking to normally unattractive cues. Behavioural Brain Research, 266, 119–130. 10.1016/j.bbr.2014.03.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Robinson MJF, Clibanoff C, Freeland CM, Knes A, Cote JR, & Russell TI (2019). Distinguishing between predictive and incentive value of uncertain gambling-like cues in a Pavlovian autoshaping task. Behavioural Brain Research, 371, Article 111971. 10.1016/j.bbr.2019.111971 [DOI] [PubMed] [Google Scholar]
  63. Robinson TE, & Flagel SB (2009). Dissociating the predictive and incentive motivational properties of reward-related cues through the study of individual differences. Biological Psychiatry, 65(10), 869–873. 10.1016/J.BIOPSYCH.2008.09.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Rung JM, Buhusi CV, & Madden GJ (2018). Reducing impulsive choice: V. The role of timing in delay-exposure training. Behavioural Processes, 157, 557–561. 10.1016/j.beproc.2018.04.018 [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Rung JM, & Madden GJ (2018). Experimental reductions of delay discounting and impulsive choice: A systematic review and meta-analysis. Journal of Experimental Psychology: General, 147(9), 1349–1381. 10.1037/xge0000462 [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Sangarapillai N, Ellenberger M, Wöhr M, & Schwarting RKW (2021). Ultrasonic vocalizations and individual differences in rats performing a Pavlovian conditioned approach task. Behavioural Brain Research, 398, Article 112926. 10.1016/j.bbr.2020.112926 [DOI] [PubMed] [Google Scholar]
  67. Sheffer CE, Christensen DR, Landes R, Carter LP, Jackson L, & Bickel WK (2014). Delay discounting rates: A strong prognostic indicator of smoking relapse. Addictive Behaviors, 39(11), 1682–1689. 10.1016/J.ADDBEH.2014.04.019 [DOI] [PMC free article] [PubMed] [Google Scholar]
  68. Smith AP, Marshall AT, & Kirkpatrick K (2015). Mechanisms of impulsive choice: II. Time-based interventions to improve self-control. Behavioural Processes, 112, 29–42. 10.1016/j.beproc.2014.10.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  69. Smith T, Fitch A, Deavours A, & Kirkpatrick K (2024). Active and passive waiting in impulsive choice: Effects of fixed-interval and fixed-time delays. Learning & Behavior. 10.3758/s13420-023-00622-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. Smith T, Panfil K, Bailey C, & Kirkpatrick K (2019). Cognitive and behavioral training interventions to promote self-control. Journal of Experimental Psychology: Animal Learning and Cognition, 45(3), 259–279. 10.1037/xan0000208 [DOI] [PMC free article] [PubMed] [Google Scholar]
  71. Smith TR, Panfil K, & Kirkpatrick K (2022). Generalizability of time-based interventions: Effects of choice procedure and smaller-sooner delay. Behavioural Processes, 196, Article 104584. 10.1016/j.beproc.2022.104584 [DOI] [PMC free article] [PubMed] [Google Scholar]
  72. Snider SE, LaConte SM, & Bickel WK (2016). Episodic future thinking: Expansion of the temporal window in individuals with alcohol dependence. Alcoholism: Clinical and Experimental Research, 40(7), 1558–1566. 10.1111/acer.13112 [DOI] [PMC free article] [PubMed] [Google Scholar]
  73. Stein JS, Smits RR, Johnson PS, Liston KJ, & Madden GJ (2013). Effects of reward bundling on male rats’ preference for larger-later food rewards. Journal of the Experimental Analysis of Behavior, 99(2), 150–158. 10.1002/jeab.11 [DOI] [PMC free article] [PubMed] [Google Scholar]
  74. Stein JS, Wilson AG, Koffarnus MN, Daniel TO, Epstein LH, & Bickel WK (2016). Unstuck in time: Episodic future thinking reduces delay discounting and cigarette smoking. Psychopharmacology, 233(21–22), 3771–3778. 10.1007/s00213-016-4410-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  75. Sze YY, Stein JS, Bickel WK, Paluch RA, & Epstein LH (2017). Bleak present, bright future: Online episodic future thinking, scarcity, delay discounting, and food demand. Clinical Psychological Science, 5(4), 683–697. [DOI] [PMC free article] [PubMed] [Google Scholar]
  76. Thomas BL, & Papini MR (2020). Shifts in intertrial interval duration in autoshaping with rats: Implications for path dependence. Learning and Motivation, 72, Article 101687. 10.1016/j.lmot.2020.101687 [DOI] [Google Scholar]
  77. Tunstall BJ, & Kearns DN (2015). Sign-tracking predicts increased choice of cocaine over food in rats. Behavioural Brain Research, 281, 222–228. 10.1016/J.BBR.2014.12.034 [DOI] [PMC free article] [PubMed] [Google Scholar]
  78. van Haaren F, van Hest A, & van de Poll NE (1987). Acquisition and reversal of a discriminated autoshaped response in male and female rats: Effects of long or short and fixed or variable intertrial interval durations. Learning and Motivation, 18(2), 220–233. 10.1016/0023-9690(87)90012-9 [DOI] [Google Scholar]
  79. Weafer J, & de Wit H (2014). Sex differences in impulsive action and impulsive choice. Addictive Behaviors, 39(11), 1573–1579. 10.1016/J.ADDBEH.2013.10.033 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

The full data set and analysis files are available on the Open Science Framework at https://osf.io/cxjaz/?view_only=90eef855c9344ae38356c4428a152d28.

RESOURCES