Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2024 Aug 1.
Published in final edited form as: Behav Processes. 2023 Jul 3;210:104911. doi: 10.1016/j.beproc.2023.104911

Effects of Inter-trial Interval on Sign-Tracking and Conditioned Reinforcer Efficacy in Female Rats

Saba Mahmoudi 1, Sara Peck 2, Gregory J Madden 3
PMCID: PMC10528028  NIHMSID: NIHMS1918551  PMID: 37406869

Abstract

Previous nonhuman studies have reported that sign-tracking to a conditioned stimulus (CS) is increased when the intertrial interval (ITI) duration is increased. Separate studies indicate that individual differences in sign-tracking (vs. goal-tracking) at a fixed ITI (and CS duration) is predictive of the conditioned reinforcer efficacy of the CS. The present study evaluates, for the first time, if increasing the ITI increases rats’ sign-tracking and the conditioned reinforcing efficacy of the CS. Forty-five female rats were randomly assigned to one of three groups that completed appetitive Pavlovian training with ITIs of 14, 24, or 96 s. Subsequently, they completed tests of conditioned reinforcement. Replicating previous findings, longer ITIs increased sign-tracking to a lever-CS and, extending the literature, conditioned reinforcer efficacy of that CS was highest at the longest ITI used during Pavlovian training. Implications for behavioral interventions using conditioned reinforcement are discussed.

Keywords: conditioned reinforcement, Pavlovian conditioning, sign-tracking, intertrial interval, translational behavior analysis

1. Introduction

Conditioned reinforcers are widely used in clinical settings to establish and maintain adaptive behavior (e.g., token economies used in schools and community-based group homes; (Ivy et al., 2017). Pavlovian learning is critical in the acquisition of new conditioned reinforcers (Fantino, 1977; Shahan & Cunningham, 2015; Williams, 1994), so Pavlovian-learning principles and procedures are relevant to effective clinical practice (Madden et al., 2023).

During Pavlovian conditioning with an appetitive unconditioned stimulus (US), the neutral stimulus signaling an imminent US onset acquires a conditioned stimulus (CS) function. That is, it evokes conditioned responding when presented without the US (Pavlov, 1927). In addition, the CS often acquires a conditioned-reinforcing function. That is, if access to the CS is contingent on an operant response, a conditioned-reinforcement effect is demonstrated if operant responding increases above a baseline level (Bersh, 1951; Marx & Knarr, 1963; Stubbs & Cohen, 1972).

Variables influencing conditioned reinforcer efficacy are clinically important (e.g., faster acquisition and better maintenance of adaptive operant behavior). Several rodent studies using lever-autoshaping methods report that rats prone to sign-tracking (i.e., they approach, lick, nibble, and/or bite the lever-CS during Pavlovian training) also find the CS to be a strong conditioned reinforcer, relative to rats that goal-track to the feeder during CS presentations (Flagel et al., 2011; Lomanowska et al., 2011; Meyer et al., 2012; Robinson & Flagel, 2009; Singer et al., 2016; Villaruel & Chaudhri, 2016). These experiments are focused on individual differences, so Pavlovian training parameters are held constant between rats. To our knowledge, only one experiment has manipulated an environmental variable to evaluate if it impacts the prevalence of sign/goal-tracking and conditioned-reinforcer efficacy (Beckmann & Chow, 2015). They reported that restricting rats’ ability to touch a lever-CS decreased sign-tracking and the conditioned-reinforcer efficacy of that CS. This finding opens the possibility that other environmental variables might prove effective in promoting sign-tracking and increasing the conditioned reinforcing efficacy of the CS, a line of research with translational potential.

One such variable is the intertrial interval (ITI) arranged during Pavlovian training. Several studies report that sign-tracking to the CS increases with longer ITI durations (Lee et al., 2018; Thomas & Papini, 2020; van Haaren et al., 1987) but none evaluated if this also increased the conditioned-reinforcer efficacy of the CS. There is good reason to hypothesize that ITI duration will also increase conditioned-reinforcer efficacy. For example, the delay-reduction theory (DRT) of conditioned reinforcement holds that a conditioned reinforcer is more effective when its onset signals that the average time between backup reinforcers (approximating the ITI) is large relative to the interval between conditioned-reinforcer onset and backup-reinforcer delivery (Fantino, 1977; Shahan & Cunningham, 2015). To the best of our knowledge, no experiments have experimentally manipulated ITI duration to determine if this systematically impacts sign-tracking and conditioned reinforcer efficacy. In the present experiment, ITI duration was parametrically manipulated during Pavlovian training across three groups of female rats. We hypothesized that longer ITIs would increase the prevalence of sign-tracking and increase the conditioned-reinforcer efficacy of the CS.

2. Method

2.1. Subjects

Female Long-Evans rats (N = 45; 60-days old) were bred and reared in house from animals purchased from Envigo Laboratories (Indianapolis, IN). Animals were weaned at 21 days and were pair-housed by sex until 50 days old. Subjects were maintained at 85% of free feeding throughout the experiment. Sample size was selected based on a power analysis (alpha = .05; power = 0.8; d =.888) with effect size based on group differences in conditioned reinforcement efficacy in Robinson and Flagel (2009). The sample size was increased by 9 rats per group to also detect interactions and account for unexpected mortality.

2.2. Materials and Apparatus

Sessions were conducted in Med-Associates (St. Albans, VT) operant chambers equipped with two front-panel retractable levers installed 3.1 cm above the floor and 2.5 cm to the left and right of a Med-Associates pellet receptacle which was equipped with a head-entry detector. A white noise speaker (80 dB) was on the rear wall. Food pellets were 45-mg grain-based pellets (#F0165, Bio-Serv®, Flemington, NJ). During the test of conditioned reinforcement, the left and right levers were removed and replaced with left and right Med-Associates nose-poke ports; one lever was installed on the center panel in place of the feeder (Beckmann & Chow, 2015; Robinson & Flagel, 2009).

2.3. Procedures

The study was approved by the Institutional Animal Care and Use Committee at Utah State University (protocol # 11747). Throughout the experiment, the onset and offset of the white noise signaled the beginning and end of the session, respectively. Sessions ended after the last food delivery.

2.3.1. Magazine Training and Group Assignment

During magazine training, 40 food deliveries were programmed according to a variabletime (VT) 120-s schedule (rectangular distribution; range = 20–220 s). Magazine training was completed when the rat entered the receptacle following 18 of the final 20 food deliveries. Subjects were block-randomly assigned to one of three groups (ITI = 14, 28, or 96) based on the number of sessions needed to complete magazine training.

2.3.2. Pavlovian Training

All rats, regardless of group assignment, completed eight Pavlovian training sessions, each composed of 25-trials (Beckmann & Chow, 2015; Robinson & Flagel, 2009). Trials began when the front-wall lever was inserted into the chamber. The left/right location of the lever was counterbalanced between subjects within groups and fixed throughout the training phase. After 8 s, the lever was retracted, and a single pellet was delivered non-contingently. The time between CS presentations (the ITI) was controlled by a VT schedule (rectangular distribution) that varied across groups: 14 s (range 9–19 s), 28 s (range 20–36 s), or 96 s (range 30–162 s).

Sign-tracking and goal-tracking during Pavlovian training were measured by recording lever-presses and feeder-entries and were quantified using the Pavlovian Conditioned Approach (PCA) Index. PCA Index values range from −1 to 1, with values > 0.5 indicating sign-tracking and values < −0.5 indicating goal-tracking. PCA Index values are the average of three scores, each assessed during CS presentations: (a) response bias, (b) probability, and (c) latency (Meyer et al., 2012).

2.3.3. Conditioned Reinforcement Test

To assess the conditioned-reinforcing efficacy of the CS, all rats completed two 30-min sessions during which no food was delivered. In this phase, the food cup was removed and replaced with the CS lever, which was moved to the feeder location (Beckmann & Chow, 2015; Robinson et al., 2019; Robinson & Flagel, 2009). Two nose-pokes were installed on either side of the lever. Sessions began with the illumination of both nose-poke ports. During the first session, a single response in the active nose-poke resulted in the offset of both nose-poke lights and the insertion of the lever (active nose-poke location was randomly assigned). After 8 s, the lever was retracted and both nose-pokes were illuminated again. Responses in the inactive nose-poke turned off both nose-pokes for 0.01s, a nominally detectable control stimulus change. Responses on the lever were recorded but had no programmed consequences. The second session was identical with the exception that a variable-ratio (VR) 2 schedule, controlled CS lever presentation (Villaruel & Chaudhri, 2016).

2.4. Data Analysis

Statistical analyses were performed using GraphPad Prism version 8.3.0 for Windows (GraphPad Software, San Diego, California USA, www.graphpad.com) and R 4.2.2 (R Core Team, 2022). Multilevel model (MLM) analyses were conducted in R using the lme4 package (Bates et al., 2015). Shapiro-Wilk tests were used to evaluate distributional properties (alpha = .05); assumptions of parametric statistics were met. False discovery rate (FDR) corrections for multiple comparisons were used. The full dataset and analysis files are available on Open Science Framework at https://osf.io/hnbvt/?view_only=43844eb5ce984aa68663f6c4272ce567

PCA Index scores (and its components) were analyzed using MLM analyses with group and session as fixed effects (level 2) and individual subjects as random effect (level 1). Because a different reinforcement schedule (FR 1 and VR 2) was arranged in the two tests of conditioned reinforcement, separate 3 (group) × 2 (port: active vs. inactive) mixed-model ANOVAs were conducted. Finally, the correlation between sign-tracking and conditioned-reinforcing efficacy (percent active nosepokes across sessions) was calculated using Spearman’s r.

3. Results

In Figure 1A, the MLM revealed significant main effects of session (p < .001) and ITI duration (p < .001) on the acquisition of CS-evoked sign- and goal-tracking topographies. A likelihood ratio test determined that the session × group interaction was not significant (χ2(2) = 3.57, p = .168); therefore, it was not included in the final model. Post-hoc group comparisons (collapsed across sessions) revealed significantly more sign-tracking (higher PCA Index) in the ITI 96 group than the ITI 28 (p = .008) and ITI 14 (p < .001) group, and significantly more sign-tracking in ITI 28 than ITI 14 group (p < .004). Results of statistical analyses of PCA Index components are shown in Figure 1B–D and are available in Open Science Framework.

Figure 1.

Figure 1

Open in a new tab

Mean and SEM for PCA Index (Panel A) and PCA Index components (Panel B: lever bias, Panel C: latency, Panel D: probability) during training for groups completing Pavlovian training with the ITI durations (sec) shown in the legend.

Note: PCA Index is composed of three components; lever bias (lever presses – feeder entries) / (lever presses + feeder entries); latency (average latency to enter the feeder – average latency to press the lever)/CS duration; and probability (trials with a lever press / total trials) – (trials with a feeder entry / total trials).

* <.05. ** <.01. *** <.001.

In the first test of conditioned reinforcement (FR 1, Figure 2A), the ANOVA revealed a significant port × group interaction (F(2,42) = 7.456; p = .002, η2 = .10), revealing that greater allocation of responding to the active port (the conditioned-reinforcement effect) was dependent upon the duration of the ITI during Pavlovian training. The main effect of port (F(1,42) = 6.186; p = .017, η2 = .04) was largely driven by significantly higher rates of active-port responding among ITI 96 rats, relative to ITI 28 (post-hoc p < .001) and ITI 14 rats (post-hoc p = .017). Only rats in the ITI 96 group made significantly more responses in the active port (post-hoc p < .001). There were no significant differences between groups in the rate of inactive nose-pokes (post-hoc ps > .8) and the main effect of group was not significant (F(2,42) = 2.66; p = .116, η2 = .06).

Figure 2.

Figure 2

Open in a new tab

Mean (+ SEM) and individual subject responses per conditioned-reinforcement test session in each nose-poke port; panel A: session 1, FR 1; panel B: session 2, VR 2. Panel C: correlations between PCA Index values during Pavlovian training and percent active nose-poke responding in the FR1 (dashed line) VR 2 (solid line) tests of conditioned reinforcement.

Note: * <.05. ** <.01. *** <.001. **** p < .0001

Similar, but not identical outcomes were obtained in the second conditioned-reinforcement test session (VR 2, Figure 2B). The port × group interaction was significant (F(2,42) = 5.638; p = .007, 𝜂2 = .06) and there were significant main effects of port (F(1,42) = 52.71; p < .0001, 𝜂2 = .30) and group (F(1,42) = 5.759; p = .006, η2 = .08). There were significantly higher rates of active-port responding among ITI 96 rats, relative to ITI 28 (post-hoc p < .0001) and ITI 14 rats (post-hoc p = .004) and no difference between ITI 28 and ITI 14 group (post-hoc p = .608). There were no significant differences between groups in the rate of inactive nose-pokes (post-hoc ps > .9). The ITI 96 group responded significantly more in the active than the inactive nose-poke port (post-hoc p < .001). This difference was also significant for the ITI 14 group, but only in this second test session (post-hoc p < .002); response allocation was undifferentiated between ports in both sessions for the ITI 28 group (post-hoc p = .128).

Figure 2C plots all rats’ percent active nose-pokes across the tests of conditioned reinforcer efficacy against their PCA Index scores. Disregarding group, significant positive correlations were observed in the first (r .35, p= .017; 95% CI = 0.06, 0.5) and second (r = .31, p =.035; 95% CI = 0.01, 0.6) conditioned-reinforcement test sessions.

4. Discussion

Increasing ITI durations significantly and systematically increased sign-tracking in female rats. Rats in the ITI 96 group predominantly sign-tracked and ITI 14 rats predominantly goal-tracked, with ITI 24 rats showing a mix of both topographies. In the tests of conditioned reinforcement, ITI 96 rats responded more to produce brief access to the lever-CS – the lever acquired a conditioned-reinforcer function. Shorter-duration ITIs did not consistently produce a conditioned-reinforcement effect. That is, rats in the ITI 24 group never preferred the nose-poke port that produced brief access to the lever-CS, and the ITI 14 rats favored that port in only one test session. Thus, this experiment offers some, but not entirely consistent evidence that longer ITI durations increase sign-tracking and the conditioned-reinforcing efficacy of the CS.

The present results may have implications for using conditioned reinforcers in applied behavioral research and practice. Arranging longer ITI durations during Pavlovian training may increase client attention to (sign-tracking; (Bucker & Theeuwes, 2017, 2018; Garofalo & di Pellegrino, 2015) and the incentive value of the conditioned reinforcer. Applied studies often use very brief ITIs, which may explain the inconsistent effects of Pavlovian training when establishing new conditioned reinforcers (for review see Madden et al., 2023).

Two limitations of the present study should be noted. First, an unpaired/control group was omitted because many published studies have demonstrated that these control conditions do not generate sign- or goal-tracking, nor do they establish a CS that functions as a conditioned reinforcer (Burns & Domjan, 2000; Robinson & Flagel, 2009; Srey et al., 2015). Second, female rats were exclusively used. It will be important to evaluate if the ITI effect on sign-tracking and conditioned-reinforcer efficacy is replicable in males.

Highlights.

  • Sign-tracking is predictive of conditioned reinforcer efficacy.

  • Intertrial intervals during Pavlovian training were manipulated (14, 24, or 96 s).

  • Longer intertrial intervals systematically increase sign-tracking.

  • Longer intertrial intervals yielded the most effective CS/conditioned reinforcer.

Acknowledgments

This research was supported by a grant from the National Institute on Drug Abuse (DA052467) awarded to the third author. None of the authors have any real or potential conflict(s) of interest, including financial, personal, or other relationships with organizations or pharmaceutical companies that may inappropriately influence the research and interpretation of the findings. The authors would like to thank Katherine C. Garland, Gabrielle M. Sutton, Joshua I. Jones, Sophia S. Sperber, Christine T. Layne, and Kelsey B. Smith for their assistance in conducting the experiment.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Contributor Information

Saba Mahmoudi, Department of Psychology, Utah State University, USA.

Sara Peck, College of Arts & Sciences, Western New England University, USA.

Gregory J. Madden, Department of Psychology, Utah State University, USA

References

  1. Bates D, Maechler M, Bolker B, & Walker S (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. doi: 10.18637/jss.v067.i01. [DOI] [Google Scholar]
  2. Beckmann JS, & Chow JJ (2015). Isolating the incentive salience of reward-associated stimuli: Value, choice, and persistence. Learning and Memory, 22(2), 116–127. 10.1101/lm.037382.114 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bersh PJ (1951). The influence of two variables upon the establishment of a secondary reinforcer for operant responses. Journal of Experimental Psychology, 41(1), 62–73. 10.1037/h0059386 [DOI] [PubMed] [Google Scholar]
  4. Bucker B, & Theeuwes J (2017). Pavlovian reward learning underlies value driven attentional capture. Attention, Perception, and Psychophysics, 79(2), 415–428. 10.3758/s13414-016-1241-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bucker B, & Theeuwes J (2018). Stimulus-driven and goal-driven effects on Pavlovian associative reward learning. Visual Cognition, 26(2), 131–148. 10.1080/13506285.2017.1399948 [DOI] [Google Scholar]
  6. Burns M, & Domjan M (2000). Sign tracking in domesticated quail with one trial a day: Generality across CS and US parameters. Animal Learning and Behavior, 28(1), 109–119. 10.3758/BF03199776 [DOI] [Google Scholar]
  7. Fantino E (1977). Conditioned Reinforcement: Choice and information. In Honig WK & Staddon JER (Eds.), Handbook of operant behavior (pp. 313–339). Prentice-Hall. [Google Scholar]
  8. Flagel SB, Clark JJ, Robinson TE, Mayo L, Czuj A, Willuhn I, Akers CA, Clinton SM, Phillips PEM, & Akil H (2011). A selective role for dopamine in stimulus–reward learning. Nature, 469, 53–57. 10.1038/nature09588 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Garofalo S, & di Pellegrino G (2015). Individual differences in the influence of task-irrelevant Pavlovian cues on human behavior. Frontiers in Behavioral Neuroscience, 9(JUNE). 10.3389/fnbeh.2015.00163 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Hox JJ, Moerbeek M, & Van de Schoot R (2017). Multilevel analysis: Techniques and applications (3rd ed.). Routledge. [Google Scholar]
  11. Ivy JW, Meindl JN, Overley E, & Robson KM (2017). Token economy: A systematic review of procedural descriptions. Behavior Modification, 41(5), 708–737. 10.1177/0145445517699559 [DOI] [PubMed] [Google Scholar]
  12. Lee B, Gentry RN, Bissonette GB, Herman RJ, Mallon JJ, Bryden DW, Calu DJ, Schoenbaum G, Coutureau E, Marchand AR, Khamassi M, & Roesch MR (2018). Manipulating the revision of reward value during the intertrial interval increases sign tracking and dopamine release. PLoS Biology, 16(9). 10.1371/journal.pbio.2004015 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Lomanowska AM, Lovic V, Rankine MJ, Mooney SJ, Robinson TE, & Kraemer GW (2011). Inadequate early social experience increases the incentive salience of reward-related cues in adulthood. Behavioural Brain Research, 220(1), 91–99. 10.1016/J.BBR.2011.01.033 [DOI] [PubMed] [Google Scholar]
  14. Madden GJ, Mahmoudi S, & Brown K (2023). Pavlovian learning and conditioned reinforcement. Journal of Applied Behavior Analysis. 10.1002/jaba.1004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Marx MH, & Knarr FA (1963). Long-term development of reinforcing properties of a stimulus as a function of temporal relationship to food reinforcement. Journal of Comparative and Physiological Psychology, 56(3), 546–550. 10.1037/h0046942 [DOI] [PubMed] [Google Scholar]
  16. Meyer PJ, Lovic V, Saunders BT, Yager LM, Flagel SB, Morrow JD, & Robinson TE (2012). Quantifying individual variation in the propensity to attribute incentive salience to reward cues. PLoS ONE, 7(6). 10.1371/journal.pone.0038987 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Pavlov IP (1927). Conditioned reflexes: An investigation of the physiological activity of the cerebral cortex. Annals of Neurosciences, 17(3), 136. 10.5214/ans.0972-7531.1017309 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. R Core Team (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/ [Google Scholar]
  19. Robinson MJF, Clibanoff C, Freeland CM, Knes A, Cote JR, & Russell TI (2019). Distinguishing between predictive and incentive value of uncertain gambling-like cues in a Pavlovian autoshaping task. Behavioural Brain Research, 371, 111971. https://doi.org/ 10.1016/j.bbr.2019.111971 [DOI] [PubMed] [Google Scholar]
  20. Robinson TE, & Flagel SB (2009). Dissociating the predictive and incentive motivational properties of reward-related cues through the study of individual differences. Biological Psychiatry, 65(10), 869–873. 10.1016/J.BIOPSYCH.2008.09.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Shahan TA, & Cunningham P (2015). Conditioned reinforcement and information theory reconsidered. Journal of the Experimental Analysis of Behavior, 103(2), 405–418. 10.1002/jeab.142 [DOI] [PubMed] [Google Scholar]
  22. Singer BF, Bryan MA, Popov P, Scarff R, Carter C, Wright E, Aragona BJ, & Robinson TE (2016). The sensory features of a food cue influence its ability to act as an incentive stimulus and evoke dopamine release in the nucleus accumbens core. Learning and Memory, 23(11), 595–606. 10.1101/lm.043026.116 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Srey CS, Maddux JMN, & Nadia C (2015). The attribution of incentive salience to Pavlovian alcohol cues: A shift from goal-tracking to sign-tracking. Frontiers in Behavioral Neuroscience, 9(MAR). 10.3389/fnbeh.2015.00054 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Stubbs DA, & Cohen SL (1972). Second-order schedules: Comparison of different procedures for scheduling paired and nonpaired brief stimuli. Journal of the Experimental Analysis of Behavior, 18(3), 403–413. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Thomas BL, & Papini MR (2020). Shifts in intertrial interval duration in autoshaping with rats: Implications for path dependence. Learning and Motivation, 72(November), 101687. 10.1016/j.lmot.2020.101687 [DOI] [Google Scholar]
  26. van Haaren F, van Hest A, & van de Poll NE (1987). Acquisition and reversal of a discriminated autoshaped response in male and female rats: Effects of long or short and fixed or variable intertrial interval durations. Learning and Motivation, 18(2), 220–233. 10.1016/0023-9690(87)90012-9 [DOI] [Google Scholar]
  27. Villaruel FR, & Chaudhri N (2016). Individual Differences in the Attribution of Incentive Salience to a Pavlovian Alcohol Cue. Frontiers in Behavioral Neuroscience, 10(DEC), 238. 10.3389/fnbeh.2016.00238 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Williams BA (1994). Conditioned reinforcement: Experimental and theoretical issues. The Behavior Analyst, 17(2), 261–285. 10.1007/BF03392675 [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES