A Behavioral Timing Intervention Upregulates Striatal Serotonergic Markers and Reduces Impulsive Action in Adult Male Mice

Abstract

Many studies support the hypothesis that time-based interventions reduce impulsive behavior in rodents. However, few studies have directly assessed 1) how such interventions affect impulsive action rather than impulsive choice, 2) if intervention effects differ by sex, and 3) how time-based interventions affect neurochemistry in regions mediating decision-making and reward. Thus, we assessed how a fixed-interval (FI) intervention initiated during late adolescence and extending into adulthood affected dopaminergic and serotonergic analytes in the frontal cortex and striatum and subsequent impulsive action in adult male and female mice. Beginning on postnatal day (PND) 45, mice were either trained on a progressive series of FI schedules (FI 20, 40, & 60 s) or remained in the home cage. Following the intervention, increases in striatal serotonergic analytes were found in FI-exposed males and females (n = 8/sex/group) with few changes found in the frontal cortex. Impulsive action was assessed in the remaining mice (n = 10/sex/group) using a fixed-ratio waiting-for-reward (FR-wait) task in which completion of an FR-25 component initiated a “free” pellet component in which pellets were delivered at increasing intervals according to a fixed delay increment that varied across sessions. Responses reset the additive delay and initiated a new FR-25 component. FI-exposed males, but not females, showed fewer delay resets and no-wait resets relative to control mice. Importantly, FI-exposure did not affect discrimination reversal performance in either sex. These data suggest that time-based interventions may reduce impulsive action in addition to impulsive choice perhaps with increased male sensitivity. Additionally, time-based interventions appear to operate through striatal serotonergic augmentation.

1. Introduction

Deficits in impulse control underlie many maladaptive behaviors such as substance misuse (Dalley et al., 2007; Perry & Carroll, 2008) and gambling (Alessi & Petry, 2003), among others. Impulsive behavior, broadly defined as acting in the present moment without consideration of future rewards, can be characterized as a preference for small, immediate reinforcers at the expense of larger, delayed reinforcers (i.e., impulsive choice (Evenden, 1999)) and/or an inability to inhibit prepotent behavior (i.e., impulsive action (Bari & Robbins, 2013). Impulsive behavior is also implicated in neurodevelopmental disorders such as Attention-Deficit Hyperactivity Disorder (ADHD) (Sonuga-Barke, 2005; Winstanley et al., 2006) in which interventions are often attempted to reduce impulsive behavior, increase attention, and promote self-control (Evans et al., 2011; Johnstone et al., 2010, 2012; Sonuga-Barke et al., 2013). Various animal models of impulsive behavior interventions have been developed to understand how these interventions bring about functional improvements in self-control. While interventions in animal models do show success in promoting self-control (Smith et al., 2019), the behavioral and neurobiological mechanisms of these interventions have yet to be fully explored.

Behavioral targets of impulse-control interventions often center on temporal processes of reward (Rung & Madden, 2018; Smith et al., 2019). This approach suggests that impulsive behavior stems from poor temporal estimation or an overestimation of delays (Wittmann & Paulus, 2008). Deficits in temporal estimation may then produce an aversion to delays such that delayed rewards are avoided (Galtress et al., 2012; Kirkpatrick et al., 2015). In support of this hypothesis, several types of time-based interventions in rodents decrease impulsive choice including exposure to fixed-interval (FI) or variable-interval (VI) schedules of reinforcement (Bailey et al., 2018; Panfil et al., 2020; Smith et al., 2015), differential-reinforcement-of-low-rate (DRL) schedules (Fox et al., 2019), or forced exposure to delayed reinforcers (Stein et al., 2013). However, the extent to which these interventions improve timing processes is unclear with some studies reporting timing improvements (Marshall et al., 2014; Peterson & Kirkpatrick, 2016; Smith et al., 2015) and others showing no timing improvement despite reduced impulsive choice (Fox et al., 2019; Rung et al., 2018) suggesting that these interventions may preferentially increase tolerance for delayed reinforcers.

While promising, these interventions have been primarily studied in male rodents to the exclusion of females. In humans, it has been suggested that women and men differ in impulsive decision-making and inhibitory control (Cross et al., 2011). For example, men tend to show decreased effortful control over prepotent responses (i.e., impulsive action) whereas women tend to show increased impulsive choice, particularly with hypothetical rewards (Cross et al., 2011; Weafer & de Wit, 2014). Animal studies also show that male rodents display greater impulsive action than females and females show greater impulsive choice than males (Weafer & de Wit, 2014). Importantly, time-based interventions in female rats are effective in reducing impulsive choice (Stuebing et al., 2018), similar to findings in male rats. More recent data, however, suggest FI interventions may produce more consistent reductions in impulsive choice in male rats relative to female rats, particularly at shorter delay intervals (Panfil et al., 2020). This interaction of biological sex and time-based interventions to reduce impulsive choice remains to be fully understood. Additionally, few studies have assessed effects of time-based interventions on tasks targeting impulsive action and delay tolerance relative to impulsive choice. Thus, one goal of the current study was to assess possible differences in the efficacy of an FI intervention in females and males to increase delay tolerance in a fixed-ratio (FR) waiting-for-reward task (FR-wait). In this task, completion of an FR component is reinforced with a food pellet immediately followed by a component where “free” pellets are delivered, contingent on withholding lever presses, according to an increasing sequence of wait or delay intervals. This task has been used in rodents to characterize delay-tolerant effects of serotonergic drugs (Bizot et al., 1988) and an impulsive phenotype induced by developmental exposure to lead (Brockel & Cory-Slechta, 1998) and air pollution (Allen et al., 2013).

In addition to a lack of research on sex-dependent efficacy of interventions for impulsive action or impulsive choice, no published studies to date have directly assessed how these time-based interventions affect neurobiological substrates of reward and impulse control. It is well-established that these processes are heavily influenced by cortico-striatal monoamine circuits (Dalley & Roiser, 2012; Dalley et al., 2008; Kable & Glimcher, 2009; Volkow & Baler, 2015). In impulse-control disorders like ADHD, activation of prefrontal and striatal subregions during delayed reinforcement tasks is reduced (Bush et al., 2005) and is associated with high task impulsivity and trait impulsivity (Schneider et al., 2010; Ströhle et al., 2008). Furthermore, increasing dopaminergic tone via methylphenidate administration consistently normalizes striatal and frontal cortex activation in ADHD populations (Spencer et al., 2013) suggesting a putative role for dopamine signaling in impulse-control disorders. While dopamine’s involvement in impulsivity is complex, it appears to play a particular role in incorporating delays to reinforcer receipt into a reinforcer’s subjective value (Pine et al., 2010). Thus, inexperience with or overestimation of reinforcer delays may produce devaluations of anticipated future reinforcers via dopamine encoding.

Imaging studies in humans have reported adaptive changes in cortico-striatal dopamine dynamics following training on cognitive tasks (Bäckman & Nyberg, 2013). Specifically, five weeks of working memory training reduces D2 receptor expression in the prefrontal cortex (McNab et al., 2009) and striatum (Bäckman et al., 2011) suggesting training increases dopamine tone. Similar findings have been reported in rats following training that happened to occur during adolescence (Soiza-Reilly et al., 2004), a developmental window in which behavioral interventions may have long-lasting effects on behavior (Fuhrmann et al., 2015; Gruss et al., 2010). However, it is unclear if similar changes in dopamine dynamics occur following interval-based training.

Among other monoamines involved in impulsivity, serotonin has long been suggested as a primary driver in promoting patience or response inhibition (Cardinal, 2006; Wogar et al., 1992). Indeed, serotonin depletion in humans and rodents increases impulsivity (Crockett et al., 2010; Eagle et al., 2009; Worbe et al., 2014), particularly impulsive action and not impulsive choice (Winstanley et al., 2004). Additionally, activation of the serotonin-rich dorsal raphe nuclei (DRN) promotes response inhibition (Fonseca et al., 2015), which is reversed by silencing DRN neurons (Miyazaki et al., 2012). Moreover, serotonergic DRN neurons show increased firing rates specifically while waiting for delayed reinforcers, activity which ceases upon reinforcer delivery (Miyazaki et al., 2011). Such findings are in large agreement with historically hypothesized roles of serotonin in promoting behavioral inhibition broadly (Soubrié, 1986). These functions are likely mediated by robust striatal and frontal cortex innervation by DRN neurons (Groenewegen & Uylings, 2000; Puig & Gulledge, 2011; Vertes, 1991). Thus, time-based interventions may promote long-term augmentation of serotonergic signaling in cortico-striatal regions, separate from or in addition to dopaminergic changes, to promote waiting behavior when presented with opportunities to receive delayed reinforcers.

Therefore, the current study was designed to address primarily: 1) the extent to which a time-based intervention reduces impulsive action as opposed to impulsive choice, 2) possible sex-dependent efficacy in the effects of time-based interventions on impulsive action, and 3) how a time-based intervention affects dopaminergic and serotonergic tone in the frontal cortex and striatum. A secondary goal was to test whether any behavioral effects of a time-based intervention generalized to cognitive processes outside impulsive action, namely behavioral flexibility, which is similarly mediated by fronto-striatal monoamine signaling (Robbins, 2007).

2. Methods

2.1. Animals

Male and female C57BL6/J mice were bred in-house for all experiments. Briefly, male and female C57BL6/J mice were purchased from Jackson Laboratories (Bar Harbor, ME) at 8 weeks of age. All breeders acclimated for one week in a temperature- (71–74°F) and humidity-controlled (35–40%) colony room operating on a 12 h light/dark cycle (lights on at 0600). Mice were then bred monogamously for 3 days, after which males were removed from the cage and dams remained singly house with their pups until weaning at postnatal day (PND) 21. Pups were removed from the dams and randomly assigned to receive FI training or no training. Litters were culled such that no more than two pups per sex/litter were used for neurotransmitter or behavioral endpoints following FI training. After weaning at postnatal (PND) 21, mice were pair-housed by sex and training group with standard rodent chow and water freely available until PND 35, at which point food access was restricted (3-h ad libitum access daily) for lever-press training at PND 40. All mice were weighed daily to ensure food restriction did not interfere with normal growth curves. All mice were used and treated via protocols approved by the University of Rochester Medical Center Institutional Animal Care and Use Committee and Committee on Animal Resources (approval # 102208 / 2010-046E), and in accordance with NIH guidelines.

2.2. Behavioral Apparatus

All behavioral testing occurred in operant-conditioning chambers for mice (ENV-307W; Med Associates, St. Albans, VT) housed in sound-attenuating chambers with fans for ventilation. The work panel of the chamber consisted of two retractable levers with a small LED lever light above each lever positioned on either side of a central food pellet receptacle where 45-mg food pellets could be accessed (Bio-Serv, Flemington, NJ). A non-retractable lever was positioned in the lower center of the rear wall opposite the work panel. A houselight centered at the top of the rear wall illuminated the interior of the chamber during sessions. All experimental events were controlled by a MED-Associates^® interface and desktop computer using MED-State notation.

2.3. Lever-press training

The experimental timeline is shown in Figure 1. All mice, regardless of group assignment, were trained to press the rear lever beginning on PND 40. An autoshaping procedure was used to establish lever pressing as described previously (Allen et al., 2014). Briefly, autoshaping consisted of one or two 6-h sessions in which food pellets were freely delivered once every 90-s on average (variable-time [VT] 90-s reinforcement schedule) for the first 30 minutes of the session. During this initial VT component, a lever press also produced one pellet. A response-contingent component began following 30 minutes of the VT component or if 20 pellets were earned via a rear lever press, whichever occurred first. During the response-contingent component, pellets were delivered according to a fixed ratio-1 (FR-1) schedule of reinforcement. Autoshaping sessions terminated following 6 h or 50 pellets earned, whichever occurred first. Autoshaping was considered complete if at least 40 pellets were earned during the session. Following autoshaping, FR-1 sessions began. During FR-1 sessions, mice were required to earn 50 pellets within 1 h for 2 consecutive sessions prior to FI training. All mice met lever-pressing criteria within 3–5 sessions.

Figure 1.

Timeline of procedures across postnatal day.

2.4. FI intervention

FI training began the day following completion of FR training on approximately PND 45, considered late adolescence in rodent brain development (Spear, 2000). The overall goal of FI training was to expose mice to incrementing time intervals across sessions to promote behavioral adjustment to longer wait intervals. Across the 30 total sessions of FI training, mice were exposed to an FI 20-s schedule (Sessions 1–10), FI 40-s schedule (Sessions 11–20), and an FI 60-s schedule (Sessions 21–30) during 30-minute testing sessions. After mice were loaded into chambers, sessions were initiated by a single lever press on the active rear lever. The first rear lever press following the elapsed criterion time (20 s, 40 s, or 60 s depending on the schedule) resulted in reinforcement and immediately started the next interval (i.e., a free-operant FI schedule) after a brief 3-s reinforcement period in which the houselight flashed to signal reinforcement. Thus, only the first interval of the session was response-initiated, as unsignaled response-initiated fixed interval schedules can disrupt timing performance (Fox & Kyonka, 2015). To avoid large reductions in overall reinforcement rate across FI schedules, sessions were programmed to allow 90 possible pellets per 30-minute session. Thus, reinforcer magnitude was increased along with each FI duration while the number of possible intervals was reduced: FI 20-s = 90 possible intervals with 1 pellet each; FI 40-s = 60 possible intervals with 2 pellets each; FI 90-s = 30 possible intervals with 3 pellets each. This resulted in a slight increase, rather than a decrease, in overall reinforcement rate across FI durations (see Supplementary Figure 1). Sessions were carried out 4–5 days per week at approximately the same time of day.

To assess timing performance during FI training, response distributions were quantified in the final session of each FI duration. For each mouse, responses were time-stamped and sorted into successive 5-s bins within each inter-reinforcer interval. Each 5-s bin was then divided by the bin with the highest response count for each mouse to yield a proportional measure for each bin with values ranging from 0–1. A Gaussian function was then fit to individual, normalized mouse FI distributions to derive timing accuracy (peak location) and precision (peak spread) parameters for statistical analysis. Bins from all mice were also averaged for male and female mice separately for data visualization.

2.5. Tissue collection and neurotransmitter quantification

The day following the final FI intervention session, brain tissue from a subset of mice (n = 8/group/sex) was collected for neurotransmitter quantification. Mice were weighed and euthanized by rapid decapitation in random order at approximately the time when behavioral sessions would have occurred. Brains were removed and the left hemisphere dissected to collect the frontal cortex and striatum. Tissue was flash-frozen in dry ice and stored at −80°C until neurotransmitter analysis.

To determine effects of the FI intervention on neurotransmitter profiles in the frontal cortex and striatum, dopamine (DA), serotonin (5-HT) and their synthesis (tryptophan [Trp] and tyrosine [Tyr]) and degradation products (5-hydroxyindoleacetic acid [5-HIAA] and 3,4-dihydroxyphenylacetic acid [DOPAC]) were quantified by the University of Rochester Mass Spectrometry Core. Tissues were thawed, weighed (mg), diluted in 75 μL of ice-cold acetonitrile (50%, v/v) and homogenized on ice for 10 s using a sonication probe (SLPe digital sonifier, Branson Ultrasonics Corp., Danbury, CT). The homogenate was centrifuged at 10,000 × g (4°C) for 20 minutes. The resulting supernatant was collected and centrifuged at 10,000 × g (4°C) for 20 minutes. The final supernatant was then collected and stored at −80°C until analysis.

Stock solutions of DA, DOPAC, Trp, 5-HT, and 5-HIAA (Sigma) were made at 5 mg/mL water, with the exception of Tyr, which was made in 0.2 M HCl. A mixture of these internal standards was created in water, with analyte concentrations varying in accordance with prior range-finding studies in order to account for region-specific variation in endogenous neurotransmitters. This stock solution was derivatized with 13C benzoyl chloride (BzCl, Sigma) using a method adapted from (Wong et al., 2016) to create internal standards for each individual neurotransmitter. The derivatized internal standard mixture was aliquoted and frozen at −80°C for long term storage. At the time of analysis, internal standard aliquots were thawed, then diluted in a 50% acetonitrile, 1% sulfuric acid solution prior to being added to the samples. Prior to analysis, samples were derivatized following the same procedure. In brief, samples were centrifuged at 16,000 × g for 5 minutes to remove any debris, then 20 μL of resulting supernatant was placed in a clean LoBind tube (Eppendorf). Next, 10 μL of 100 mM sodium carbonate, 10 μL of 2% BzCl in acetonitrile, and 10 μL of the respective internal standard was added in sequence. 50 μL of water was then added to reduce the organic concentration prior to injection. Samples were centrifuged once more to pellet any remaining protein, and the supernatant was placed in a clean autosampler vial.

LC-MS/MS analysis was carried out by coupling a Dionex Ultimate 3000 UHPLC to a Q Exactive Plus mass spectrometer (Thermo Fisher). Analytes were separated on a Waters Acquity HSS T3 column. The mobile phases were: A) 10 mM ammonium formate in 0.1% formic acid, and B) acetonitrile. The flow rate was set to 400 μL/min and the column oven was set at 27°C. After 10 μL of each sample was injected, the analytes were separated using a 12-minute multi-step gradient. The Q Exactive Plus was operated in positive mode, and a parallel reaction monitoring method (PRM) was used to detect derivatized molecules. Fragment ions were extracted with a 10 ppm mass error using the LC Quan node of the XCalibur software (Thermo Fisher). Endogenous analyte peak areas were compared to those of each internal standard to determine relative abundance. Further normalizing abundance to the wet weight of the tissue (mg) yielded mass-specific concentrations of the neurotransmitters (ng/mg). DA and 5-HT turnover was estimated by dividing the measured degradation product by the neurotransmitter (e.g., 5HIAA concentration / 5-HT concentration; 5HIAA:5HT). All values are expressed as % of sex-matched control.

2.6. FR training and FR-wait assessment

Ten days following the FI intervention, the remaining mice (n = 10/group/sex) began FR training for the FR-wait assessment. All mice began on an FR-1 schedule regardless of FI history and progressed to an FR-25 schedule across sessions. To ensure lever-press mastery in control (non FI-experienced) mice, all mice were required to earn at least 40 reinforcers in 30 minutes on the FR-1 schedule. Only five control mice required a second FR-1 session suggesting that initial autoshaping in adolescence endured into adulthood. Mice were then successively trained on an FR-3, FR-5, FR-10, FR-15, and FR-20 schedule across days to minimize ratio strain. Mice progressed to the next FR if they earned at least 30 reinforcers in 30 minutes or 3 sessions occurred, whichever occurred first. The FR-20 schedule was maintained until all mice had completed FR training at which point all mice were transitioned to the final FR-25 schedule together for 15 total sessions to ensure stable responding. All FR training sessions terminated following 30 minutes or 30 reinforcers, whichever occurred first. The FR-wait task began the day following the 15^th FR-25 session.

The FR-wait task was conducted as previously described with adaptation (Brockel & Cory-Slechta, 1998). The task consists of an incrementing wait component that follows each FR-25 reinforcer earned. This wait component allowed mice to obtain “free” pellets following completion of each FR component during a 30-minute session. These “free” pellets were delivered at increasing delay intervals based on the wait increment (e.g., 5 s) during a session as long as no lever presses occurred. For example, a 5-s wait increment would yield successive “free” pellet delays of 5 s, 10 s, 15 s, etc. If a lever press occurred during the wait component (termed a “reset”), the FR component was re-introduced and the pellet delay was reset. To increase the probability of waiting behavior, an adjusting intertrial interval (ITI) was used, such that mice could not maximize reinforcement by completing successive FR components without entering the wait component. The ITI adjusted based on how long a mouse had waited during the preceding wait component. For example, if a mouse did not wait for a “free” pellet in a given component, there was a 60-s ITI in which the chamber darkened and lever presses were not reinforced. However, if a mouse earned, for example, two “free” pellets during a wait component in a 10-s wait session (10 s + 20 s = 30 s of wait total time), the ITI would be 30 s (60 s – 30 s = 30 s). Alternatively, if the same mouse only waited for one “free” pellet after completing the subsequent FR-25 (10 s of total wait time), the next ITI would be 50 s (60 s – 10 s = 50 s).

Three different wait increments were used across 26 total FR-wait sessions in the following order: 5-s (3 sessions), 10-s (5 sessions), 15-s (5 sessions), 10-s (5 sessions), 5-s (3 sessions), 15-s (5 sessions). Thus, each wait interval was tested twice and analyzed as the grand mean of all sessions at each wait interval for each mouse. Dependent measures included total resets, no-wait resets (resets in which no wait interval was completed), maximum wait time (the longest mean wait interval reached in session), response rate (total responses divided by session time), run rate (total responses divided by time in FR-25 components), and total pellets earned. FR-25 training was also analyzed to assess pre-FR-wait response differences between groups.

2.7. Spatial discrimination training and reversal assessment

Performance in a spatial discrimination reversal task was assessed following the FR-wait task to determine if effects of the FI intervention would generalize to cognitive flexibility or reversal learning as distinct from delay tolerance. Training began by reinforcing responses according to an FR-1 schedule on either the left or right front lever (counterbalanced across mice). If the left lever was first for a given mouse, then responses to the right lever were reinforced the following session and vice versa for mice assigned to the right lever first. Following left and right lever training, rear lever presses were reinforced the following session. Once at least 40 pellets were earned in a single 60-minute session for each lever (left, right, rear) chain lever press training began. Chain lever press training required a two-response sequence in which a press on the rear lever within 15 s of trial initiation caused either the front left or right lever to extend into the chamber (counterbalanced across mice). A front lever press within 15-s of lever extension was reinforced with a single pellet followed by a 15-s ITI in which the chamber was darkened with levers retracted. Failure to press within the 15-s rear or front lever limited hold ended the trial and initiated the ITI. Sessions consisted of 60 total trials. Only one chain (either back-left or back-right) was active during a chain session. Once a mouse earned at least 50 reinforcers during chain training one side (e.g., back-left), the opposite chain was in effect the following session (e.g., back-right). Discrimination training began following completion of chain training.

Each spatial discrimination session consisted of 60 total trials separated by a 15-s ITI. During the original discrimination, a rear lever press within 15 s caused both front levers to extend into the chamber. A press on the “correct” lever (left or right counterbalanced across mice) within 15-s of lever extension produced one food pellet followed by the ITI. A press on the “incorrect” lever (termed an error) immediately terminated the trial without pellet delivery and initiated the ITI. A trial was considered an omission if a lever press did not occur within 15 s of ITI termination (rear omission) or front lever extension (front omission). After ≥ 51 correct responses (85% accuracy) for 3 consecutive sessions, the first reversal was imposed by switching the “correct” lever to the opposite lever the following session. All mice completed the original discrimination and three discrimination reversals. Dependent measures included: 1) the number of sessions and 2) the total errors to reach mastery criterion across the original discrimination and subsequent reversals and 3) total error within the first session of each reversal.

2.8. Statistical analyses

All statistical analyses were split a priori by sex to determine any sex-specific effects of the FI intervention on neurochemical or behavioral outcomes. An exception to this was data from the FI intervention itself in which sex was a between-subjects factor to determine possible sex differences in FI performance. Neurotransmitter data were analyzed using independent-samples t tests with a Bonferroni correction for multiple comparisons within each sex and brain region (alpha adjusted to p < .0125). All repeated-measures data were analyzed using a 2 (group or sex) × 3 (time) or 2 (group) × 4 (time) repeated-measures analysis of variance (ANOVA) with group as the between-subjects factor and time (wait interval or reversal) as the within-subjects factor. Significant interactions were followed by Bonferroni post hoc tests. Additional analyses included session-by-session analysis of FR-wait resets within each FR-wait condition for male and female mice. Greenhouse-Geisser adjustments were made in the event of sphericity violations of repeated-measures data. Effect sizes for repeated-measures outcomes were estimated using generalized eta-squared (Bakeman, 2005). Pearson correlations were also conducted to assess the relation between FI-60 s post-reinforcement pausing (PRP) and subsequent FR-wait resets at each FR-wait duration. All data were analyzed in R using the stats and rstatix libraries. Behavioral outcomes were considered significant if p ≤ .05.

3. Result

3.1. FI intervention performance

During the progressive FI intervention, male and female mice adjusted to each FI schedule, as confirmed by response distributions as shown in Figure 2A & 2B. Quantitative analysis of FI timing gradients revealed relatively poor variance accounted for (VAC) by Gaussian curve fitting (range = 0.60 – 0.98; average = 0.80). Data from eight mice (4 female and 4 male) were excluded from the Gaussian analysis due to extremely poor equation fits (VAC < 0.60) yielding 14 mice of each sex included in the Gaussian analysis. Analysis of peak times revealed a significant Sex × FI duration interaction (F(2, 50) = 5.60, p = .006, η_g² = 0.18 [Figure 2C]). Post hoc analyses showed that male mice peaked closer to the FI 60-s criterion time (mean = 45.08 s) relative to female mice (mean = 36.14 s) with no differences detected at FI 20-s or FI 40-s. Despite grouped Gaussian curves suggesting wider FI distributions of females during FI 60-s training, there was no difference in peak spread between male and female mice across FI durations (F(1, 25) = 2.63, p = .11, η_g² = 0.12 [Figure 2D]). Apart from timing-specific analyses, male mice showed higher pellets earned per session (F(1, 34) = 16.94, p > .001, η_g² = 0.33) and higher response rates (F(1, 34) = 6.23, p = .01, η_g² = 0.15) across durations (Supplementary Figure 1).

Figure 2.

Top panels: Response distributions during the final session of each FI duration for all female (A) and male (B) mice. Different FI durations are noted by light grey, dark grey, and black lines. Dotted lines represent the criterion time for each FI duration. Bottom panels: parameter estimates of gaussian fits of normalized FI functions. n = 14/sex.

3.2. Neurotransmitter quantification

Brain tissue was collected the day following the final FI 60-s session of the FI intervention for neurotransmitter analysis in the striatum and frontal cortex from a subset of mice as described above. A Bonferroni correction was applied to t-test outcomes to control for multiple comparisons with an adjusted alpha criterion of 0.0125. Overall, the FI intervention appeared primarily to affect the 5-HT metabolic pathway, rather than DA, with more consistent effects detected in the striatum (Figure 3) than in the frontal cortex (Figure 4). In female mice receiving FI training, slight increases in Trp (t(14) = 2.93, p = .010) and 5HIAA (t(14) = 2.74, p = .016) were evident in the striatum (Figure 3A), with no significant increases in 5-HT (p = .047) or 5HIAA:5HT ratios (p = .23). In males (Figure 3B), FI-exposed mice showed increases in 5HIAA (t(14) = 3.33, p = .004) with inconsistent changes in Trp (p = .049), 5HIAA:5HT ratios (p = .033) and 5-HT (p = .065). FI training had almost no effect on striatal DA-related outcomes in females (Figure 3C) or males (Figure 3D), other than slight increases in Tyr in FI-exposed males (t(14) = 2.74, p = .016) (all other p’s > .08). In frontal cortex tissue, female mice showed no consistent changes in serotonergic (p’s > 0.045) or dopaminergic analytes (p’s > .33; Figure 4A&C). Similar effects were found in FI-exposed male mice, with no significant changes in serotonergic (Figure 4B; p’s > .16) or dopaminergic analytes (p’s > 0.03) in frontal cortex.

Figure 3.

Striatum serotonin (A, B) and dopamine (C, D) outcomes in female (A, C) and male (B, D) mice following the final FI 60-s session. Raw values were transformed to percent control for each analyte according to sex. Bars represent mean ± SEM. * indicates p < .0125 vs corresponding control, ~ indicates p < .05. n = 8/group.

Figure 4.

Frontal cortex serotonin (A, B) and dopamine (C, D) outcomes in female (A, C) and male (B, D) mice following the final FI 60-s session. Raw values were transformed to percent control for each analyte according to sex. Bars represent mean ± SEM. * indicates p < .05 vs corresponding control, ~ indicates p < .05. n = 8/group.

3.3. FR-wait performance

Prior to FR-wait testing, no group differences in FR-25 performance were detected during FR-25 training in male (response rate: F(1, 18) = 0.3, p = 0.59, η_g² = 0.01; reinforcement rate: F(1, 18) = 0.3, p = 0.59, η_g² = 0.01) or female (response rate: F(1, 18) = 0.05, p = 0.94, η_g² = 0.001; reinforcement rate: F(1, 18) = 0.04, p = 0.94, η_g² = 0.001; Supplementary Figure 2).

Effects of the FI intervention on FR-wait performance were primarily detected in male mice, but not female mice (Figures 5 & 6). For females and males, total resets increased across wait intervals regardless of FI training (Females: F(2, 36) = 144.31, p < .001, η_g² = 0.67 [Figure 5A]; Males: F(2, 36) = 46.63, p < .001, η_g² = 0.34 [Figure 5D]). However, males that had received FI training showed fewer resets across wait intervals relative to control males (F(1, 18) = 4.87, p = .04, η_g² = 0.18 [Figure 5D]), which was not detected in FI-exposed females (F(1, 18) = 2.06, p = .17, η_g² = 0.08). Similarly, FI-exposed male mice showed fewer no-wait resets relative to control mice (F(1, 18) = 6.54, p = .02, η_g² = 0.15 [Figure 5E]), whereas females did not (F(1, 18) = 0.98, p = .33, η_g² = 0.02 [Figure 5B]). For females and males, maximum wait time increased across wait intervals (Females: F(2, 36) = 6.60, p = .004, η_g² = 0.14 [Figure 5C]; Males: F(2, 36) = 15.35, p < .001, η_g² = 0.25 [Figure 5F]). A nonsignificant trend was detected in males such that FI-exposed males showed slightly higher maximum wait times across wait intervals, particularly at the 5-s wait interval (F(1, 18) = 3.48, p = .079, η_g² = 0.11 [Figure 5F]), with no such trend in females (F(1, 18) = 0.79, p = .38, η_g² = 0.11). Regardless of FI training, a Sex × Group interaction (F(2, 76) = 6.11, p = .003, η_g² = 0.06) revealed that male mice showed more resets relative to females at the 5- and 10-s wait times. Females also showed longer maximum waits (F(1, 38) = 7.38, p = .01, η_g² = 0.11) but not no-wait resets (F(1, 38) = 3.4, p = .08, η_g² = 0.04) relative to males.

Figure 5.

Total resets (A, D), no-wait resets (B, E), and average max wait (C, F) for female (A, B, C) and male (D, E, F) mice during FR-wait testing. Data points represent mean ± SEM. * indicates p < 0.05 vs corresponding control. ~ indicates p < 0.1 vs corresponding control. n = 10/group.

Figure 6.

Response rate (A, D), run rate (B, E), and totals pellets earned (C, F) for female (A, B, C) and male (D, E, F) mice during FR-wait testing. Data points represent mean ± SEM. * indicates p < 0.05 vs corresponding control. n = 10/group.

Overall response rate increased across wait intervals for both females and males (Females: F(2, 36) = 137.16, p < .001, η_g² = 0.62 [Figures 6A]; Males: F(2, 36) = 7.63, p = .002, η_g² = 0.08 [Figure 6D]). Male mice that had received FI training showed lower response rates than control males (F(1, 18) = 5.05, p = .037, η_g² = 0.18 [Figure 6D]), with no such trend in female mice (F(1, 18) = 2.6, p = .125, η_g² = 0.10). Similar patterns were observed in within-FR run rates, in which males that had received FI training had lower run rates than control males (F(1, 18) = 5.04, p = .037, η_g² = 0.19 [Figure 6E]), with no significant difference in females (F(1, 18) = 3.37, p = .083, η_g² = 0.10 [Figure 6B]). Total pellets earned decreased across wait intervals for females and males, as expected (Females: F(2, 36) = 6.57, p = .004, η_g² = 0.09 [Figure 6C]; Males: F(2, 36) = 67.52, p < .001, η_g² = 0.38 [Figure 6F]); however, no differences in total pellets were detected between groups for males or females (Females: F(1, 18) = 2.87, p = .11, η_g² = 0.10 [Figure 6C]; Males: F(1, 18) = 3.45, p = .08 , η_g² = 0.14 [Figure 6F]).

Correlations between FR 60-s PRPs and FR-wait resets were largely nonsignificant. PRPs and resets showed a significant negative correlation for female mice for the 5-s wait interval (r = −0.68, p = .02). This relation did not hold for the 10-s (r = −0.37, p = .29) or 15-s (r = −0.09, p = .80) for female mice or any FR-wait duration for male mice (5-s: r = 0.04, p = .91; 10-s: r = −0.17, p = .63; 15-s: r = −0.29, p = .41) (See Supplemental Figure 4).

3.4. Spatial discrimination reversal performance

Overall, no group differences were detected during spatial discrimination training or reversal performance (Figure 7). The number of sessions required to reach the mastery criterion increased from the original discrimination to subsequent reversal for females (F(3, 54) = 5.33, p = 0.003, η_g² = 0.11 [Figure 7A]) and males (F(3, 54) = 9.50, p < 0.001, η_g² = 0.26 [Figure 7D]); however, no effects of Group were detected (Females: F(1, 18) = 2.11, p = .16, η_g² = 0.04; Males: F(1, 18) = 2.56, p = .13, η_g² = 0.04). Similar effects were found in numbers of errors to criterion across reversals in which, as expected, errors increased from the original discrimination to subsequent reversals for females (F(3, 54) = 12., p = 0.003, η_g² = 0.11 [Figure 7B]) and males (F(3, 54) = 14.03, p < 0.001, η_g² = 0.39 [Figure 7E]), but no effects of Group were found (Females: F(1, 18) = 2.95, p = .10, η_g² = 0.09; Males: F(1, 18) = 0.44, p = .51, η_g² = 0.004). Lastly, neither FI-exposed females (F(1, 18) = 1.6, p = .22, η_g² = 0.02; Figure 7C) nor males (F(1, 18) = 1.9, p = .66, η_g² = 0.004; Figure 7F) showed differences in first-session reversal errors across reversals.

Figure 7.

Sessions to criterion (A, D), errors to criterion (B, E), and total error (C, F) for female (A, B, C) and male (D, E, F) mice during spatial discrimination reversal testing. Data points represent mean ± SEM. n = 10/group.

4. Discussion

The current set of experiments was designed to address possible sex differences in the effects of a time-based behavioral intervention to reduce impulsive action and the potential neurochemical changes that may underlie functional improvements. Collectively, our findings provide tentative evidence that a progressive FI intervention preferentially upregulates serotonergic profiles in the striatum of both male and female mice, but only improves waiting behavior in male mice as measured in an FR-wait task. Importantly, behavioral effects of the FI intervention did not generalize to a reversal learning paradigm suggesting possible domain selectivity of FI interventions.

In line with previous studies (Fox et al., 2019; Panfil et al., 2020; Smith et al., 2015), the FI intervention in the current study reduced impulsive action as shown by reductions in overall FR resets and no-wait resets in male mice in the FR-wait task. There was also a trend toward increased maximum wait time across wait intervals for FI-exposed males. As in delay-discounting paradigms where prolonging reinforcer delay increases preference for more immediate reinforcers, increasing the wait increment in the current FR-wait task produced more frequent resets and no-wait resets leading to more immediate access to the next FR reinforcer, albeit at a higher response cost. Thus, the current study complements and extends previous work on FI intervention effects in impulsive choice tasks to tasks measuring impulsive action.

Despite the FI intervention effects and the similarity of neurotransmitter changes to those observed in male mice, it is unclear why FI-exposed females did not also show reduced resets on the FR-wait task. One possible explanation could be related to female performance during the FI intervention. While there were no consistent differences in peak spread (i.e., timing precision) across FI duration, female mice showed earlier peak times on the FI 60-s schedule relative to males suggesting less accurate timing of the longest FI duration or the schedule that required the most waiting. These findings are opposite of what has been found in other studies assessing sex differences of interval timing finding either no sex difference or earlier responding in males (Buhusi et al., 2017; Gür et al., 2019). However, it was hypothesized that later responding in females was due to lower motivation relative to males (Gür et al., 2019). Indeed, males showed higher response rates and more reinforcers earned during the FI intervention in the current study suggesting greater motivation despite later FI 60-s peak times. These response patterns also carried over to FR training and the FR-wait task, where female mice showed fewer resets and higher maximum wait times supporting previous evidence that female rodents show greater response inhibition than males (Bayless et al., 2012). The lack of FI effect in females could also be due to the parameters of the FI intervention. The FI intervention used here consisted of a free-operant procedure in which thirty total 30-minute sessions were conducted and, on average, 40–55 pellets were earned per session. While not directly comparable, previous studies that found reductions in female impulsive choice (Panfil et al., 2020; Stuebing et al., 2018) used a total of forty-five sessions where 100 pellets could be earned in 2-hr sessions using a discrete-trial task. Thus, it is possible that a more robust intervention in the current study could have increased delay tolerance in females as has been previously suggested (Stuebing et al., 2018). Additional studies will be necessary to uncover intervention parameters that produce reductions in impulsive action independent of sex.

The FR-wait effects observed in FI-experienced males could be due to factors other than increased waiting, per se. For example, while nonsignificant, control males earned slightly more pellets across wait intervals relative to FI-exposed males, particularly at the 10-s wait interval. This is likely influenced by higher response rates leading to reduced FR completion times and more frequent resets allowing access to shorter free pellet delays, thereby increasing local reinforcement density. However, this interpretation is limited for several reasons. First, a similar nonsignificant increase in control female pellets was also observed during the 10-s wait interval as seen in male mice without any consistent differences in FR resets as observed in males. Second, control males showed relatively consistent increases in no-wait resets both at the 5- and 10-s wait intervals suggesting a somewhat durable increased likelihood of immediate resetting despite that reset producing a 60-s ITI where no reinforcement could be earned. Third, no group differences in FR performance were observed during FR training. Differences only emerged after delays to free pellets were introduced. This effect has also been reported previously (Allen et al., 2013). Importantly, these data show very little support for an interpretation of increased delay tolerance in male mice. Indeed, post-reinforcement pauses on the FI 60-s schedule did not consistently correlate with FR resets in either females or males. Experience with delayed reinforcement could increase waiting by virtue of simply learning that reinforcer delivery serves as a discriminative stimulus for subsequent delayed reinforcement. Experience with programmed delays is not present in mice with only FR training (i.e., control mice in the current study), thus this association cannot be learned. While the precise behavioral mechanism is unclear, the current data suggest a beneficial effect of FI exposure on impulsive action, which is consistent with its effects on impulsive choice (Bailey et al., 2018; Smith et al., 2015). Thus, it is possible that time-based interventions may benefit multiple types of impulsivity.

In addition to behavioral effects, we also found that FI-exposed male and female mice showed relatively specific increases in serotonergic analytes in the striatum. Changes in either serotonergic profiles in the frontal cortex or dopaminergic profiles in the striatum or frontal cortex were less consistent. This finding suggests that exposure to an FI intervention preferentially modulates serotonin rather than dopamine dynamics, which may be localized to the striatum. Interestingly, earlier findings indicate that serotonin metabolites show more consistent increases than dopamine metabolites in cerebrospinal fluid of pigeons during performance on an FI rather than FR schedule (Barrett & Hoffmann, 1991). Additionally, selective lesions of dorsal raphe nucleus (DRN) serotonin projection neurons impair acquisition of appropriate inhibition of responding in the peak-interval procedure (Morrissey et al., 1994), a modified version of an FI schedule. Coupled with the results of the FR-wait task, changes in serotonin, but not dopamine, further suggest that a primary function of FI interventions may be to increase waiting rather than improve temporal estimation, per se. This possibility is in line with previous work showing reduced impulsive choice following time-based interventions without improvements in interval timing itself (Fox et al., 2019; Rung et al., 2018). Indeed, dopamine signaling, through its participation with glutamate, greatly facilitates subjective time estimation with increases or decreases in dopamine signaling producing overestimation or underestimation of subjective duration, respectively, in interval timing tasks (MacDonald & Meck, 2005; Meck, 1996). Serotonin, rather than being a necessary component of neural timing systems (Ho et al., 2002), appears to play more general role in response inhibition or “waiting” behavior (Bizot et al., 1988; Luo et al., 2015).

Given the strong interconnections between the frontal cortex and striatum underlying response inhibition and timing (Dalley et al., 2008), as well as dense fronto-cortical and striatal serotonin innervation of DRN neurons (Descarries et al., 2010; Vertes, 1991), it was anticipated that neurotransmitter changes would be consistently present in both regions following FI training. Similarly, it has been hypothesized that specific areas of the prefrontal cortex (e.g., the orbitofrontal cortex [OFC] and medial prefrontal cortex [mPFC]) and striatum (e.g., the nucleus accumbens [NAcc]) interact with serotonergic circuits to mediate waiting (Miyazaki et al., 2012). Thus, it is unclear why more consistent effects were localized to the striatum. One possibility could be that DRN-mediated serotonin signaling may be greater in the striatum relative to the frontal cortex. For example, direct serotonergic DRN innervation appears to be more robust in the striatum (Descarries et al., 2010). DRN neurons also strongly innervate mid-brain dopamine nuclei (i.e., the ventral tegmental area and substantia nigra) (Descarries et al., 2010) from which dopamine release into the striatum can facilitate further striatal serotonin release (Dagher et al., 2022). It is also possible that regional variation in frontal cortex serotonin could have been masked by analyzing the whole region without specifically focusing on different prefrontal regions, which appear to have greater DRN innervation (Vertes, 1991). In addition, it is important to consider the possibility of local synthesis of serotonin in various subregions that may not require DRN input. Indeed, FI training increased tryptophan expression in female frontal cortex and the striatum of both sexes. Differences in tryptophan hydroxylase expression across brain regions would necessarily limit the functional relevance of increased tryptophan expression observed here (Walther et al., 2003). Future studies may consider greater regional specificity and quantification of rate-limiting enzymes for serotonin and dopamine production. These points remain speculative, however, as functional serotonin dynamics in the frontal cortex and striatum are extremely complex, varying by subregion and receptor subtypes (Nair et al., 2020; Puig & Gulledge, 2011).

Apart from altering FR-wait performance and serotonergic analyte expression, effects of the current FI intervention did not generalize to improvements in behavioral flexibility. Domain-selective strategies specific to impulsivity, such as interval or delay exposure used here and by others (Smith et al., 2015), have largely been employed to target behaviors relevant to self-control. Despite these interventions being effective, an underlying goal of impulse-control intervention is for treatments, whether in the clinic or laboratory, to generalize across contexts, response strategies, or complementary behavioral processes (Smith et al., 2019). More domain-general approaches, like environmental enrichment during development, have been shown to affect behavioral flexibility alone (Jones et al., 1991; Schrijver et al., 2004; Schrijver & Würbel, 2001) or behavioral flexibility along with impulsive choice/action (Wang et al., 2017). However, no studies have assessed the extent to which specific impulse-control interventions extend to other processes of adaptive decision-making, like behavioral flexibility. Perhaps not surprisingly, a domain-selective strategy of FI exposure appears to be specific to impulse control with little-to-no generalization to behavioral flexibility. It is notable, however, that the current design also did not control for testing order effects. The possibility remains that FI-intervention effects could be detected by assessing behavioral flexibility directly following the FI intervention. Future studies may consider explicitly testing for order effects.

The results of the current study should be interpreted with regard to several limitations that may impact overall interpretation. First, a no-delay control group was not included in the current study. Thus, reinforcement delay was not separated from overall reinforcement during the intervention. To isolate reinforcement delay as the operative variable driving changes in impulsivity, an immediate-exposure group is usually included that is trained on an FR-2 schedule for the duration of the intervention (Fox et al., 2019; Panfil et al., 2020). However, because neurochemistry is extremely sensitive to behavioral training (Cory-Slechta et al., 2013; Cory-Slechta et al., 2009), it was preferred to only establish lever pressing in control mice without continued immediate-exposure training. In terms of behavioral experience, it could be argued that control mice did serve as an immediate or “self-paced” reinforcement group prior to the FR-wait task due to progressive FR training, whereas FI-exposed mice received delayed reinforcement training followed by immediate reinforcement during FR training. Furthermore, lever press training was durable in control mice upon initial FR training and there were no differences in FR performance between groups. Apart from these concerns, exposure to immediate reinforcement during an intervention can produce increases in impulsive choice (Fox et al., 2019; Fox, 2022; Smith, Panfil, Kirkpatrick, 2022). This effect may then exacerbate differences between delay and immediate-exposure groups. It would be interesting to directly compare the effects of immediate, noncontingent, and delayed reinforcement in future studies.

Second, there was a trend toward more pellets earned by control males on the FR-wait task. This outcome may suggest an alternative behavioral strategy in which resetting more frequently, or being less patient, was more adaptive. While this is possible, several outcomes favor a delay-intolerant phenotype in these mice. Average maximum wait times were mostly decreased in control mice relative to FI mice during the 5-s condition whereas total numbers of pellets earned did not differ. Also, despite more pellets being earned during the 10-s condition, control mice showed a nearly 100% increase in run rates and no-wait resets than FI mice, suggesting a shorter time to complete the FR-25 coupled with an increased likelihood of immediately contacting the 60-s ITI due to an immediate reset. Third, it is unclear if FI-induced changes in serotonergic analytes are solely driving improved response inhibition in male mice. In addition to more robust neurochemical changes occurring in females without increased response inhibition, behavioral effects in males were observed approximately 35 days following neurochemical analyses in the subset of FI-exposed mice. Indeed, serotonergic changes may have diminished during extended FR training or other neurotransmitter systems (e.g., glutamatergic) may have been recruited. Increases in dopamine tone have been detected at PND 90 in rats following PND 30–37 training (Soiza-Reilly et al., 2004) suggesting some durability in neurotransmitter changes is possible when no additional training occurs between initial training and neurotransmitter assessment. The current experimental design, however, only allows statements about how FI exposure appears to immediately impact neurochemical profiles and, separately, how that intervention affects subsequent response inhibition. To address this limitation, future studies should assess neurochemical profiles both following intervention training and just prior to impulsivity testing.

Given these limitations, the current study provides preliminary evidence that: 1) progressive FI training can facilitate response inhibition, 2) that males may be more sensitive to this response inhibition effect than females depending on intervention parameters, and 3) that FI training appears to selectively augment serotonergic profiles in the striatum. Another novel, but perhaps unsurprising, outcome is that behavioral effects of the FI intervention as used here did not generalize to changes in behavioral flexibility. These results are in line with previously hypothesized influences of FI schedule exposure on various processes affecting self-control (Smith et al., 2019). Importantly, these data clearly show that targeted behavioral experience itself can produce significant changes in neurochemistry, which may be behaviorally relevant. In a more general sense, these findings highlight the importance of including non-behavior control groups in studies where behavioral testing precedes analyses of hypothesized brain changes. In this specific study, these changes uncover possible neurochemical contributions to commonly used interventions to promote self-control.

Supplementary Material

1

Supplemental Figure 1. Reinforcement rate (pellets per minute) across FI durations during the FI intervention for all female and male mice. Data points represent mean ± SEM. n = 18/sex.

Supplemental Figure 2. Response rate (upper) and reinforcement (lower) for all groups during FR-25 training prior to FR-wait testing. Data points represent mean ± SEM. n = 10/group.

Supplemental Figure 3. FR-wait resets across sessions for female mice (upper) and male mice (lower). FR-wait phases are separated by dotted lines and labeled above each pair of data paths. Data points represent mean ± SEM. * indicates p < .05 relative to corresponding control group. n=10/group.

Supplemental Figure 4. Relation between FI 60-s post-reinforcement pause and FR-wait resets for females (left column) and males (right column). Data points represent PRPs from the final FI 60-s session and average resets for a given FR-wait duration. The dotted lines indicate the best-fitting linear regression functions.