Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2023 Apr 1.
Published in final edited form as: Behav Processes. 2022 Mar 14;197:104622. doi: 10.1016/j.beproc.2022.104622

Reducing Impulsive Choice: VIII. Effects of Delay-Exposure Training in Female Rats

Sara Peck 1,*, Emma Preston 2, Kelsey B Smith 1, Gregory J Madden 1
PMCID: PMC9013280  NIHMSID: NIHMS1792301  PMID: 35301066

Abstract

Impulsive choice may play an important role in serious health-related decisions, like addiction tendencies. Thus, there is merit in exploring interventions that reduce impulsive choice. Delay-exposure training involves extended experience with delayed reinforcement. Following training, delay-exposed rats make fewer impulsive choices than control rats. The reducing effects of delay exposure training on impulsive choice have been replicated in male rats seven times. For the first time, this study evaluated the effects of delay exposure training in female rats. Thirty-six rats were randomly assigned to either delay-exposure or immediacy-exposure training. Then, rats underwent two impulsive choice assessments in which they chose between one immediate pellet or three delayed pellets. In the first assessment, delays increased within-sessions, across trial blocks from 0, 8, 16, to 32 s. In the second assessment, delays to the larger reward increased between-sessions, from 8, 16, 32, to 4 s. Unlike findings with male rats, delay-exposure training produced a reduction in impulsive choice only in the initial five sessions in female rats. Possible reasons for the lack of lasting effect in female rats are discussed and future research directions are identified.

Keywords: Impulsivity, Delay discounting, Impulsive choice, Delay-exposure training, Female rats, Replication

1. Introduction

Problematic behaviors impacting human health are characterized by devaluation of delayed outcomes and preference for immediate rewards (Odum, 2011). Delay discounting refers to this devaluation as the delay to the outcome increases (Ainslie, 1975; Madden & Johnson, 2010). Steep delay discounting is correlated with substance-use disorders, over-eating, cigarette smoking, etc. (Amlung et al., 2016; Amlung et al., 2017; Reynolds et al., 2004). Because delay discounting is predictive of substance use in longitudinal studies (e.g., Barlow et al., 2016; Kim-Spoon et al., 2014), reducing discounting may prevent unhealthy behaviors in humans (Volkow & Baler, 2015).

Considerable research has aimed to reduce delay discounting, or impulsive choice, in both humans and nonhumans (Rung & Madden, 2018; Smith et al., 2019). Impulsive choice is defined as preference for a smaller-sooner reward over a larger-later reward. Delay exposure (DE) training is one learning-based procedure which has reliably reduced impulsive choice in male rats (Peck et al., 2020; Renda et al., 2018, 2020; Renda & Madden, 2016; Rung et al., 2018; Stein et al., 2013, 2015). Response to impulsivity interventions is important to evaluate across sexes and at least one other intervention – fixed-interval training – has been replicated across sexes (e.g., Panfil et al., 2020 Stuebing et al., 2018). The current study tested the efficacy of DE training in reducing impulsive choice in female rats. Rats were randomly assigned to complete 60 sessions of either DE or immediacy exposure (IE) training, and then underwent an impulsive choice assessment; Renda et al. (2020) reported that this training duration was sufficient to produce a significant and lasting reduction in male rats’ impulsive choices.

2. Method

2.1. Subjects

Thirty-six experimentally naïve female Wistar rats (Charles River Laboratories, Wilmington, MA) began the study at approximately 25 days old. Sample size was based on previous DE-training studies in male rats (Peck et al., 2020; Renda & Madden, 2016; Renda et al., 2020; Rung et al., 2018). Rats were gradually restricted to 85% of their growth-curve projected free-feeding weight, had free access to water in their home-cages, and were single-housed with cage enrichment. Home cages were kept in a humidity/temperature-controlled colony room with a 12-hr/12-hr light/dark cycle. Sessions were conducted at the same time, six days a week. Post-session feeding occurred approximately 2 hours later. Rats were matched by weight and then block-randomly assigned to either DE (n = 18) or IE training (n = 18) (Renda & Madden, 2016).

2.2. Apparatus

Twelve identical sound-attenuating operant chambers (Med Associates, St. Albans, VT) were used. Chambers were equipped with three retractable levers, two on either side of a food receptacle on the front wall and one in the center of the back wall. Cue lights were placed above each lever. Multigrain food pellets (45 mg; Bio-Serv #F0165, Frenchtown, NJ) were delivered by a pellet dispenser. White noise was provided throughout all experimental sessions.

2.3. Procedures

This study was approved by the Institutional Animal Care and Use Committee at Utah State University (protocol #11288). An autoshaping procedure was used to establish lever-pressing on the rear wall (for details, see Peck et al., 2020). Procedures followed those employed in previous investigations of DE training on impulsive choice in male rats (e.g., Peck et al., 2020; Renda et al., 2020; Rung et al., 2018).

2.3.1. Exposure Training.

Rats completed 60 sessions of their respective DE or IE training (Renda et al., 2020). Trials began every 60 s with the insertion of either the left or right front-wall lever (pseudo-randomly selected to ensure an equal number of trials on both levers) and illumination of its cue light. For DE rats, pressing the lever once retracted the lever and, following an 18-s delay, the cue light was turned off and two food pellets were delivered. For IE rats, pressing the lever once immediately retracted the lever, turned off the cue light, and delivered two food pellets. Failure to respond within 20 s initiated a 40-s blackout; the trial was scored as an omission and then repeated. Sessions terminated after 60 completed trials or 120 min, whichever occurred first. Training sessions included 60 trials rather than the 80 trials employed in male DE studies because in pilot tests, we found that some female rats omitted trials after the 60th reinforcer delivery.

2.3.2. Amount Discrimination.

Next, rats completed an amount discrimination task, in which they chose between one and three pellets, both delivered immediately after a lever press. Reward magnitudes were assigned to either the left or right lever, counterbalanced within groups. Trials began every 90 s, with insertion of a rear, center lever and illumination of its associated cue light. Following a response on that lever, either the left or right front levers (forced-choice trials) or both front levers (free-choice trials) were inserted and associated cue lights were illuminated. Failure to press a lever within 20 s was scored as an omission, initiated a 70-s blackout and, on forced-choice trials, was then repeated until completed. Sessions were separated into four trial blocks, which began with six forced-choice trials (three on each side, order randomized), followed by nine free-choice trials. Trial blocks were separated by a 7-minute blackout. Sessions terminated following completion of all trial blocks or after 2 hours elapsed, whichever occurred first. Rats continued in amount discrimination until they chose the larger reward ≥90% of the time for two consecutive sessions1.

2.3.3. Within-Session Impulsive Choice Assessment I.

Next, rats completed an impulsive choice assessment in which delays to the larger reward increased within-session, across trial blocks (Evenden & Ryan, 1996). Sessions continued exactly as described for amount discrimination, except the three-pellet reward was delayed by 0, 8, 16, and 32 s in each respective trial block. The number and range of delays was different than in prior male DE studies (i.e., 0, 15, 30 s) because preliminary testing with a separate sample of female rats revealed greater sensitivity to the 15-s delays than with males. During the delay, the lever was retracted and the cue light remained lit until food was delivered. The one-pellet reward was always delivered immediately. Rats were returned to amount discrimination if preference for the 3-pellet reward fell below 80% in the first trial block (0-s delay) for two consecutive sessions. Rats completed a minimum of 15 sessions, and continued until visually stable and area under the impulsive-choice curve (AUC; Myerson et al., 2001) in the last five sessions did not vary by more than 15% from the five-session mean. If the quantitative stability criterion was not met by session 25, the phase was terminated when no trend in AUC was visually apparent.

2.3.4. DE/IE Retraining.

Because DE2 rats made more impulsive choices than expected, two more phases were conducted to (a) assess choice at a smaller delay to the larger reward (4 s) and (b) evaluate the hypothesis that sensitivity to the longer delays arranged in the within-session increasingly-delays procedure (16 and 32 s) may have generalized to other trial blocks, thereby increasing preference for the smaller reward in earlier trial blocks. In this phase, rats were returned to their respective DE or IE training conditions for 30 sessions.

2.3.5. Between-Sessions Impulsive Choice Assessment II.

After passing the amount discrimination phase again, rats then completed a final impulsive-choice test in which delays were changed between-, rather than within-sessions. That is, test sessions were conducted as before, except that the delay to the larger reward was fixed throughout the session. That delay (8, 16, 32, and then 4 s) remained in effect for a minimum of 5 sessions and until judged stable (5–15 sessions). A minimum of 10 sessions were conducted at the final delay (4 s) because rats had no prior experience with that delay, and to allow for additional experience following the much longer 32-s delay.

2.4. Data Analysis

Non-choice measures were averaged and compared with Mann-Whitney U tests. An rTOST test of equivalence for non-normally distributed data (equivalence package; Robinson & Robinson, 2016) was used to evaluate whether reward amount discrimination in the first trial block of Impulsive Choice Assessment I was equivalent across groups. Group differences in Impulsive Choice Assessment I were analyzed with AUC, consistent with previous research in this line (e.g., Peck et al., 2020; Renda et al., 2020; Rung et al., 2018). Because AUC and large reward choice distributions violated normality and were bimodal in shape, beta regression was used to compare groups in both impulsive choice assessments (betareg package; Cribari-Neto & Zeileis, 2010). AUC values were transformed by a constant (i.e., outcome*(n-1) + .5)/n; Smithson & Verkuilen, 2006) to comply with the range requirements of beta regression. Pseudo R2 was used to quantify the variability accounted for by training group. Analyses of choice data were conducted in R (R Core Team, 2013); data, code, and output are available at: https://osf.io/8ax24/?view_only=84be0e07cb6940c8b33d17cb64872e6d

3. Results

In training, there was no significant group difference in latency to press the lever (p = .99). Due to differences in the first five DE/IE training sessions, IE rats completed significantly more exposure training-trials per session than DE rats (p < .0001; mean difference = 3.31); after these initial five sessions, however, the difference was not significant (p = .11).

Figure 1 shows data from Impulsive Choice Assessment I. As shown in the upper-left graph, group was a significant predictor of AUC in the first five sessions (p < .007; Pseudo R2 = .19), with DE rats having higher AUC (greater preference for larger, delayed rewards). The middle and right graphs in the upper row show individual rats’ preference for the larger reward across delays in these same initial five sessions (bold lines indicate the group means). This initial group difference was not due to amount-discrimination differences, as the test of equivalence in the first trial block (0s delay) revealed that the groups were equivalent in the initial five sessions. In the next five sessions (6–10) the difference in AUC only approached significance (middle graphs: p = .08; Pseudo R2 = .09) and in the final five sessions, the effect was clearly no longer significant (bottom graphs: p = .22; Pseudo R2 = .04). This erosion of effect was due to a significant reduction in self-control choices among DE rats (first five to final five sessions; p = .05). There was no comparable change among IE rats (p = .13).

Figure 1.

Figure 1

Open in a new tab

Left column: DE (filled circles) and IE (open circles) AUC values across sessions 1–5 (top), 6–10 (middle), and final five sessions (bottom) in Impulsive Choice Assessment I. Error bars show median and IQR. The asterisk indicates a significant group difference (p = .007). Center and right columns show individual subjects’ percent large-reward choice in each trial block, averaged across sessions 1–5, 6–10, and the final 5 sessions; Bolded data line shows group mean across delays.

Figure 2 shows results from the stable sessions of the Impulsive Choice Assessment II. As before, there was no significant effect of group on AUC (p = .39; Pseudo R2 = .023). At the 8s delay (first delay tested after retraining), a group difference in proportion large-reward choice approached significance, with DE rats making fewer impulsive choices than IE rats (p = .054; Pseudo R2 = .13); However, there were no group differences observed at subsequent delays (ps > .3).

Figure 2.

Figure 2

Open in a new tab

Left panel: DE (filled circles) and IE (open circles) AUC in Impulsive Choice Assessment II. Error bars show median and IQR. Center and right columns show individual DE and IE rats’ (respectively) percent large-reward choice at each delay phase, averaged across the final five sessions; Bolded data line shows group mean across delays.

4. Discussion

Seven previous replications demonstrate DE training produces lasting reductions in impulsive choice in male rats, including at the training duration used in this experiment (i.e., 60 sessions; Renda et al., 2020). However, in this first test of DE training in female rats, an effect was only observed in the initial five sessions of an impulsive choice assessment. What accounts for the lack of a lasting effect is unknown, but we offer four speculations that might be evaluated in future research. First, female rats may need the full duration of training provided in prior DE studies with male rats (i.e., 120 sessions; e.g., Stein et al., 2013), particularly given that they could complete fewer trials per DE-training session than male rats. Second, the Renda et al. (2020) finding that 60 DE-training sessions are adequate to reduce impulsive choice has not been evaluated in a replication study. If that finding is not robust, that could account for the present failure in female rats. Third, our sample size (based on previous replications of DE training in male rats; e.g., Rung et al., 2018) may provide inadequate statistical power in female rats. We are not fond of this speculation, as it suggests that small effect sizes should be pursued with larger sample sizes, rather that better experimental control (Perone, 2019). Finally, all prior DE-training studies have used either Wistar or Long Evans rats acquired from Harlan laboratories; the present experiment was conducted with Wistar rats acquired from Charles River. Behavioral differences across suppliers within the same Wistar strain have been reported in the literature (e.g., Palm et al. 2011). The present failure to replicate either across sex or vendors suggests that other interventions that prove effective in reducing rodent impulsive choice should be evaluated for generality across sex, strains, and vendors. The timing-based approach developed in the Kirkpatrick lab, for example, has proven effective in reducing impulsive choice in both male and female Sprague Dawley rats (e.g., Panfil et al., 2020 Stuebing et al., 2018) but all studies in that research line have been acquired from a single vendor (Charles River). Although future studies should evaluate if DE training is effective in female Wistars acquired from Harlan, the failure to generalize efficacy across either sex or vendors is noteworthy.

Highlights.

  • Effects of delay-exposure training were assessed for the first time in female rats with two impulsive choice assessments.

  • Delay-exposure training, relative to immediacy-exposure training, produced only a temporary reduction in impulsive choice in female rats.

  • The difference in impulsive choice across groups did not maintain after the first five sessions of the assessment.

  • Future directions and possible explanations for the lack of lasting effect are discussed.

Acknowledgments

This research was supported by NIH grant R03 DA044927-01, awarded to Dr. Gregory J. Madden. None of the authors have any real or potential conflict(s) of interest, including financial, personal, or other relationships with organizations or pharmaceutical/biomedical companies that may inappropriately influence the research and interpretation of the findings. All authors have contributed substantively to this article and have read and approved this final manuscript.

Footnotes

1

One IE rat was excluded due to failure to pass amount discrimination.

2

One DE rat died after completing Impulsive Choice Assessment I and did not complete the Retraining phase or Impulsive Choice Assessment II.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Ainslie G (1975). Specious reward: A behavioral theory of impulsiveness and impulse control, 82(4), 463–496. 10.1037/h0076860 [DOI] [PubMed] [Google Scholar]
  2. Amlung M, Petker T, Jackson J, Balodis I, & MacKillop J (2016). Steep discounting of delayed monetary and food rewards in obesity: a meta-analysis. Psychological Medicine, 46(11), 2423–2434. 10.1017/S0033291716000866 [DOI] [PubMed] [Google Scholar]
  3. Amlung Michael, Vedelago L, Acker J, Balodis I, MacKillop J, Joseph S, & Hamilton H (2017). Steep delay discounting and addictive behavior: A meta-analysis of continuous associations. Addiction, 112(1), 51–62. 10.1111/add.13535 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Barlow P, McKee M, Reeves A, Galea G, & Stuckler D (2016). Time-discounting and tobacco smoking: A systematic review and network analysis. International Journal of Epidemiology, 46(3), 860–869. 10.1093/ije/dyw233 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Cribari-Neto F, & Zeileis A (2010). Beta regression in R. Journal of statistical software, 34(1), 1–24. 10.18637/jss.v034.i02 [DOI] [Google Scholar]
  6. Evenden JL, & Ryan CN (1996). The pharmacology of impulsive behaviour in rats: the effects of drugs on response choice with varying delays of reinforcement. Psychopharmacology, 128(2), 161–170. 10.1007/s002130050121 [DOI] [PubMed] [Google Scholar]
  7. Kim-Spoon J, Farley JP, Holmes C, Longo GS, & McCullough ME (2014). Processes linking parents’ and adolescents’ religiousness and adolescent substance use: Monitoring and self-control. Journal of Youth and Adolescence, 43(5), 745–756. 10.1007/s10964-013-9998-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Madden GJ, & Johnson PS (2010). A delay discounting primer. In Madden Gregory J. & Bickel WK (Eds.), Impulsivity: The behavioral and neurological science of discounting (pp. 11–37). Washington, D.C.: American Psychological Association. 10.1037/12069-001 [DOI] [Google Scholar]
  9. Myerson J, Green L, & Warusawitharana M (2001). Area under the curve as a measure of discounting. Journal of the experimental analysis of behavior, 76(2), 235–243. 10.1901/jeab.2001.76-235 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Odum AL (2011). Delay discounting: I’m a k, you’re a k. Journal of the Experimental Analysis of Behavior, 96(3), 427–439. 10.1901/jeab.2011.96-423 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Palm S, Hävermark Å, Meyerson BJ, Nylander I, & Roman E (2011). When is a Wistar a Wistar? Behavioral profiling of outbred Wistar rats from five different suppliers using the MCSF test. Applied Animal Behaviour Science, 135(1–2), 128–137. 10.1016/j.applanim.2011.08.010 [DOI] [Google Scholar]
  12. Panfil K, Bailey C, Davis I, Mains A, & Kirkpatrick K (2020). A time-based intervention to treat impulsivity in male and female rats. Behavioural brain research, 379, 112316. 10.1016/j.bbr.2019.112316 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Peck S, Rung JM, Hinnenkamp JE, & Madden GJ (2020). Reducing impulsive choice: VI. Delay-exposure training reduces aversion to delay-signaling stimuli. Psychology of Addictive Behaviors, 34(1), 147. 10.1037/adb0000495 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Perone M (2019). How I learned to stop worrying and love replication failures. Perspectives on Behavior Science, 42 (1), 91–108. 10.1007/s40614-018-0153-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Renda CR, Rung JM, Hinnenkamp JE, Lenzini SN, & Madden GJ (2018). Impulsive Choice and Pre-Exposure to Delays: IV. Effects of Delay-and Immediacy-Exposure Training Relative to Maturational Changes in Impulsivity. Journal of the Experimental Analysis of Behavior, 109(3), 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Renda CR, Rung JM, Peck S, & Madden GJ (2020). Reducing impulsive choice VII: effects of duration of delay-exposure training. Animal Cognition, 1, 3. 10.1007/s10071-020-01412-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Renda CR, & Madden GJ (2016). Impulsive choice and pre-exposure to delays: III. Four-month test-retest outcomes in male wistar rats. Behavioural Processes, 126. 10.1016/j.beproc.2016.03.014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Reynolds B, Richards JB, Horn K, & Karraker K (2004). Delay discounting and probability discounting as related to cigarette smoking status in adults. Behavioural processes, 65(1), 35–42. 10.1016/s0376-6357(03)00109-8 [DOI] [PubMed] [Google Scholar]
  19. Robinson A, & Robinson MA (2016). Package ‘equivalence’.
  20. Rung JM, Buhusi CV, & Madden GJ (2018). Reducing impulsive choice: V. The role of timing in delay-exposure training. Behavioural Processes, 157, 557–561. 10.1016/j.beproc.2018.04.018 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Rung JM, & Madden GJ (2018). Experimental reductions of delay discounting and impulsive choice: A systematic review and meta-analysis. Journal of Experimental Psychology: General, 147(9), 1349–1381. 10.1037/xge0000462 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Smith T, Panfil K, Bailey C, & Kirkpatrick K (2019). Cognitive and behavioral training interventions to promote self-control. Journal of Experimental Psychology: Animal Learning and Cognition, 45(3), 259–279. 10.1037/xan0000208 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Smithson M, & Verkuilen J (2006). A better lemon squeezer? Maximum-likelihood regression with beta-distributed dependent variables. Psychological methods, 11(1), 54. 10.1037/1082-989x.11.1.54 [DOI] [PubMed] [Google Scholar]
  24. Stein JS, Johnson PS, Renda C, Smits RR, Liston KJ, Shahan TA, & Madden GJ (2013). Early and prolonged exposure to reward delay: Effects on impulsive choice and alcohol self-administration in male rats. Experimental and Clinical Psychopharmacology, 21(2). 10.1037/a0031245 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Stein JS, Renda CR, Hinnenkamp JE, & Madden GJ (2015). Impulsive choice, alcohol consumption, and pre-exposure to delayed rewards: II. Potential mechanisms. Journal of the Experimental Analysis of Behavior, 103(1). 10.1002/jeab.116 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Stuebing SL, Marshall AT, Triplett A, & Kirkpatrick K (2018). Females in the forefront: Time-based intervention effects on impulsive choice and interval timing in female rats. Animal cognition, 21(6), 759–772. 10.1007/s10071-018-1208-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Team, R. C. (2013). R: A language and environment for statistical computing.
  28. Volkow ND, & Baler RD (2015). NOW vs LATER brain circuits: Implications for obesity and addiction. Trends in Neurosciences, 38(6), 345–352. 10.1016/J.TINS.2015.04.002 [DOI] [PubMed] [Google Scholar]

RESOURCES