Sex and individual differences in the effect of chronic low-dose ethanol on behavioral strategy selection

Abstract

Background:

The development of an alcohol use disorder (AUD) involves impaired behavioral control and flexibility. Behavioral inflexibility includes the inability to shift behavior in response to changes in behavioral outcomes. Low levels of ethanol drinking may promote the formation of inflexible, habitual reward seeking, but this may further depend on ethanol exposure timing in relation to learning. The goal of this study was to determine whether a history of low-dose ethanol exposure promoted contingency-insensitive sucrose seeking and altered behavioral strategy selection.

Methods:

Male and female C57BL6/J mice were trained to perform a response (lever press) for sucrose on two different reinforcement schedules: one which is thought to promote inflexible responding (random interval (RI)) and one which maintains flexible responding (variable ratio (VR)). Following instrumental training each day, mice were exposed to saline or low-dose ethanol (0.5 g/kg; i.p.) either proximal (1 hour after) or distal (4 hours after) to learning. Mice were then tested for sensitivity to changes in contingency in a contingency degradation test.

Results:

A history of low-dose ethanol exposure shifted behavioral strategy selection, as measured by reward tracking behavior, but this depended on sex and reinforcement schedule history. Both male and female mice used different strategies depending on the reinforcement schedule, but only males exhibited ethanol-induced shifts in strategy selection. A history of low-dose ethanol exposure did not impact contingency sensitivity in males but promoted insensitivity in females specifically on the VR lever.

Conclusions:

These findings demonstrated that female mice show distinct behavioral effects of repeated, low-dose ethanol exposure as compared to males and uncovered sex differences in the use of reward tracking strategies to guide behavior. Future studies should investigate sex differences in the neural consequences of chronic low-dose ethanol exposure that may underlie behavioral changes.

Introduction

Only a small percentage of people who casually drink alcohol (~10%) go on to be diagnosed with or to develop an alcohol use disorder (AUD) (SAMHSA, 2021), but there is increasing evidence that exposure to even low doses of ethanol can impact the brain and behavior (Bryant et al., 2023; Cui & Koob, 2017; Randall et al., 2020). While it is known that chronic high-dose ethanol exposure can promote the development of inflexible behaviors such as habits (Dickinson et al., 2002; O’Tousa & Grahame, 2014; Vandaele & Ahmed, 2021) that are hallmarks of AUD (McKim et al., 2016; Sjoerds et al., 2013), the consequences of lower doses of ethanol are less well-characterized. A smaller but growing body of evidence suggests that chronic low-dose ethanol can also impact the development of inflexible reward seeking (Bryant et al., 2022; Corbit et al., 2012).

One aspect of inflexible, habitual behavior is insensitivity to changes in action-outcome contingencies, or the relationship between a behavior and its outcome (Dickinson, 1985). Chronic high-dose ethanol exposure promotes action-outcome insensitive sucrose seeking (Renteria et al., 2018), but this may depend on the timing of ethanol exposure in relation to reward learning (Barker et al., 2020; Corbit et al., 2012) or doses. Previously published findings from our lab demonstrated that a history of low-dose ethanol exposure proximal to reward learning enhanced sucrose reward motivation in male, but not female, mice, but this depended on reinforcement schedule history (Bryant et al., 2022). As different reinforcement schedules can be used to promote habitual or goal-directed behavior, with random interval (RI) schedules shown to promote habit formation and variable ratio (VR) schedules shown to maintain goal-directed action (Barker, Glen, et al., 2017; Dickinson et al., 1983; Gremel & Costa, 2013), chronic low-dose ethanol may interact with behavioral history to impact the use of goal-directed versus habitual response strategies. Thus, the current study investigated the effects of a history of low-dose ethanol exposure on behavioral strategy selection and sensitivity to action-outcome contingencies in sucrose seeking, and whether these differed by reinforcement schedule history and sex.

Materials and Methods

Subjects

Adult male and female C57BL/6J mice (9 weeks of age; 42 males, 30 females) from the Jackson Laboratory were used in these studies in accordance with the Drexel University Institutional Animal Care and Use Committee guidelines. The mice were housed in a vivarium with a standard 12-hour light/12-hour dark cycle and acclimated for 1 week to the facility before beginning any experiments. A subset of mice (24 males and 18 females) underwent stereotaxic surgery with a pAAV-hSyn-EGFP retrograde adeno-associated virus (Addgene plasmid # 50465, RRID: Addgene_50465) targeting the nucleus accumbens (NAc; AP +1.5mm ML +0.6mm DV −4.7mm) prior to beginning behavioral experiments in order to assess NAc-projecting circuits in a secondary tissue analysis study following all training and testing. There were no other differences, and all animals went through the same behavioral training and experiments as described below. Following recovery from surgery or following the acclimation period, mice were restricted to approximately 90% of their ad libitum weight and were then maintained at that weight for the length of the experiments. All mice were group housed for the duration of the study. A subset of the behavioral findings from these animals and experiments have been previously published (Bryant et al., 2022). This study presents new analyses of this published dataset in addition to unpublished findings in regard to contingency sensitivity and related behaviors.

Instrumental Training

All operant training occurred in standard Med-Associates operant boxes for mice, housed within sound attenuating chambers that included a fan for ventilation and white noise as described previously (Bryant et al., 2022). Prior to starting instrumental training, mice were habituated to the operant box and reward delivery magazine. For these sessions, the mice were placed in the operant box and a 10% liquid sucrose reinforcer (20ul, in tap water) was delivered into the magazine every 60 seconds for a total of 15 minutes. Mice only had one magazine training session per day for two days. After magazine training, mice were trained to lever press for sucrose on two separate levers (Fig. 1A). Only one lever was accessible at a time, and the levers were presented consecutively during the session. Thus, for the first half of the session the mice had access to one lever, then that lever would retract, and they would have access to the other lever for the rest of the session. Each lever was accessible for 15 minutes, and the whole session lasted 30 minutes. The order of which lever was accessible first alternated each day for each mouse and was counterbalanced across all groups and conditions.

Figure 1. Magazine checking behavior depends on reinforcement schedule.

(A) Experimental and behavioral timeline (CD = contingency degradation). In males, magazine checking was higher on the RI lever (B) than on the VR lever (C) and was lower in ethanol-exposed mice. In females, magazine checking was also higher on the RI lever (D) than the VR lever (E) but this was not impacted by ethanol exposure.

Initially, presses on both levers were reinforced on a fixed ratio 1 (FR1) schedule where each lever press resulted in reward delivery. Mice were trained on the FR1 schedule until they reached stable responding (at least 15 lever presses on each lever, maintained for three days). Mice that did not reach stable responding on both levers were excluded (7 males, 1 female). The schedules of reinforcement for each lever then diverged, such that the left lever began reinforcing on a RI schedule and the right lever began reinforcing on a VR schedule. On a RI schedule, the first press after a randomly determined interval (averaging 30s for RI30, and 60s for RI60) has elapsed was reinforced. On a VR schedule, the first lever press after a variable number of presses was reinforced (averaging 5 presses for VR5, and 8 presses for VR8). It has been shown that RI schedules promote inflexible behavior whereas VR schedules maintain flexible behavior (Adams & Dickinson, 1981; Barker, Bryant, et al., 2017; Dickinson et al., 1983; Gremel & Costa, 2013). Mice were trained for 3 days on the RI30/VR5 schedule and then for 3 days on the RI60/VR8 schedule. After training, mice were tested for inflexible behavior on a contingency degradation (CD) test. Progressive ratio (PR) testing occurred following CD testing as described and presented in a previously published manuscript (Bryant et al., 2022). As the PR test occurred following the CD test, there is no impact of this additional testing on the results presented here.

Ethanol exposure

Previous studies have shown that the effects of ethanol on learning can depend on ethanol exposure timing in relation to behavior, especially if exposure occurs within the window of protein synthesis dependent memory consolidation (e.g., 1–3 hours after learning) (Bourtchouladze et al., 1998; Hernandez & Abel, 2008). The current study tested and controlled for exposure timing-dependent effects by injecting saline or low-dose ethanol (0.5 g/kg; i.p.) daily either 1 hour (during the consolidation window, proximal to learning) or 4 hours (outside of the consolidation window, distal to learning) after behavior. No differences were observed between saline-treated mice that received injections at 1 hour vs. 4 hours, so data from saline mice were collapsed across groups. Mice were exposed to ethanol starting on the first day of FR1 training through the last day of RI60/VR8 training (Fig. 1A). There was no further ethanol exposure after the last day of training, therefore there was no ethanol exposure on CD testing days.

Contingency degradation testing

One day after the completion of instrumental training, mice were tested for sensitivity to changes in action-outcome relationships using a CD paradigm. To assess changes in behavior when the contingency was changed, the contingent relationship between performing an action and receiving an outcome was degraded by delivering sucrose independent of responding (i.e., noncontingently). Sucrose was instead delivered on a timer which matched the total reinforcer deliveries each animal received during their most recent RI60/VR8 session (i.e., when the action-outcome contingency was intact) for each lever (Barker et al., 2020). This degraded session was compared to a session in which the contingency was intact (nondegraded) that was identical to previous RI60/VR8 sessions. The order of which lever was presented first and the order of test session (nondegraded vs. degraded) was counterbalanced across exposure, sex, days to acquire, and total responding on the final training day. Following all testing, responding during the contingency degradation test was normalized for each animal for both test conditions (degradation score = number of lever presses during degraded condition/total lever presses for both conditions combined) (Barfield & Gourley, 2019; Barker et al., 2020). This produces a “degradation score” that more accurately reflects the changes in responding for each mouse and normalizes individual differences in total responding. Degradation scores close to 0.5 indicate that the animal is maintaining responding and is thus insensitive to changes in action-outcome contingency, while scores below 0.5 indicate greater reductions in responding consistent with sensitivity to contingency degradation.

Statistical analyses

GraphPad PRISM was used for all statistical analyses. A repeated measures ANOVA (rmANOVA) or mixed effects analysis (when there were missing values) was used for all training and testing data. Sidak’s, Tukey’s, and Dunnett’s corrections were used for post-hoc analyses as appropriate. Correlational analyses were performed using linear regression. Reward checking analysis, as measured by the percentage of lever presses that were followed by a magazine entry, was performed as described previously (Bryant et al., 2022) for both the training and testing sessions (for code see https://github.com/bsingh0110/Progressive-Ratio-Analysis-).

Results

Chronic low-dose ethanol and reinforcement schedule impact behavioral strategy selection

Previously published findings from our lab showed that chronic low-dose ethanol exposure had no effect on basal reward seeking during training, regardless of reinforcement schedule or sex (Bryant et al., 2022). However, a history of low-dose ethanol exposure proximal to learning increased reward seeking on a progressive ratio task, and this was associated with alterations in reward magazine checking behavior. To determine whether chronic low-dose ethanol exposure altered reward delivery tracking across training, and whether this depended on reinforcement schedule, magazine checking (quantified as the percentage of lever presses followed by a magazine entry) during training sessions was newly analyzed on this previously published training dataset in both male and female mice.

In males, magazine checking when responding on the RI lever (Fig. 1B) was not impacted by ethanol exposure [mixed-effects analysis, F (2, 32) = 1.211, p = 0.3113] or training day [Geisser-Greenhouse corrected, F (3.614, 109.1) = 0.9361, p = 0.4390], and no significant interaction [F (10, 151) = 1.293, p = 0.2394] was observed. For responding on the VR lever (Fig. 1C), a significant main effect of training day was observed [mixed-effects analysis, Geisser-Greenhouse corrected, F (2.988, 91.44) = 3.014, p = 0.0342] with no significant post-hoc results.

In order to directly compare magazine checking behavior on the RI and VR schedules, a three-way rmANOVA was also performed. Notable for this analysis, the later training schedules (RI60/VR8) are leaner than the early training schedules (RI30/VR5), meaning that there is a lower probability of reward delivery for the later schedules. Thus, comparisons can be made for responding on the “rich” (RI30/VR5) vs. “lean” (RI60/VR8) schedules. A significant main effect of schedule [RI vs. VR; mixed-effects analysis, F (1, 32) = 6.071, p = 0.0193] was observed such that magazine checking behavior on the RI schedule was higher than on the VR schedule. A schedule x schedule leanness interaction [F (1, 31) = 10.14, p = 0.0033] was also observed with post-hoc analyses revealing that checking behavior was higher on the RI60 than on the VR8 [Sidak’s post-hoc analysis, p = 0.0253] and there was a trend toward lower checking on the VR5 compared to the VR8 [p = 0.0997]. There was no difference in checking on the RI30 compared to the RI60 [p = 0.9957]. There was also a main effect of exposure observed [F (2, 32) = 3.483, p = 0.0428] and post-hoc analyses revealed higher magazine checking behavior in saline-exposed mice as compared to 1hr EtOH mice [Dunnett’s post-hoc analysis, p = 0.0498] and a trend in 4hr EtOH mice [p = 0.0678]. These findings demonstrate that male mice used different behavioral strategies depending on reinforcement schedule and as a result of chronic low-dose ethanol exposure.

In females, ethanol exposure and training day did not impact magazine checking on the RI lever (Fig. 1D) and only a trend toward a main effect of ethanol exposure was observed [mixed-effects analysis, F (2, 25) = 3.088, p = 0.0633]. A similar pattern was observed for responding on the VR lever (Fig. 1E), with a trend toward a training day x exposure interaction observed [mixed-effects analysis, F (10, 118) = 1.752, p = 0.0771]. In order to directly compare magazine checking behavior on the RI and VR schedules, a three-way rmANOVA was also performed on magazine checking behavior in females. A main effect of schedule was observed [three-way rmANOVA, F (1, 25) = 24.11, p < 0.0001] such that magazine checking behavior was higher on the RI schedules than on the VR schedules. There was no effect of schedule leanness on checking in females [no main effect, F (1, 25) = 1.307, p = 0.2637; no three-way interaction, F (2, 25) = 1.545, p = 0.2331]. These findings demonstrate that female mice used different behavioral strategies depending on reinforcement schedule, but strategy was generally independent of ethanol exposure.

Low-dose ethanol effects on action-outcome sensitivity depend on reinforcement schedule history and differ by sex

To determine whether a history of chronic low-dose ethanol exposure altered sensitivity to changes in action-outcome contingency, contingency degradation tests were performed in mice that were ethanol-naïve or had a history of ethanol exposure either 1 or 4 hours after self-administration sessions. It was also determined whether sensitivity was modulated by reinforcement schedule history or by sex. Degradation scores were calculated in order to normalize to individual differences in responding as described above. A significant difference between the scores during the non-degraded and degraded test indicated that the mice were sensitive to changes in the action-outcome contingency, whereas no significant difference would indicate that they were insensitive (Barfield & Gourley, 2019; Barker et al., 2020).

In males, a three-way ANOVA revealed that there were no effects of surgical history on degradation scores [surgery, F (1, 28) = 0.1779, p = 0.6764; ethanol x surgery x lever, F (2, 28) = 0.4222, p = 0.6597), and thus animals were collapsed across history of surgery for further analyses. Responding on the RI lever (Fig. 2A) was not impacted by change in contingency [two-way rmANOVA, no effect of degradation, F (1, 30) = 0.6231, p = 0.4361] or ethanol exposure [interaction, F (2, 30) = 0.5093, p = 0.6060], suggesting that all groups were insensitive to changes in action-outcome contingency. Similarly, responding on the VR lever (Fig. 2B) was not impacted by change in contingency [two-way rmANOVA, no effect of degradation, F (1, 30) = 0.9370, p = 0.3408] or ethanol exposure [interaction, F (2, 30) = 0.2239, p = 0.8007], again suggesting that all groups were insensitive to changes in action-outcome contingency. These results indicate that neither low-dose ethanol exposure history nor reinforcement schedule history impacted action-outcome sensitivity in male mice, though mice were as a population insensitive to changes in action-outcome contingency.

Figure 2. Action-outcome sensitivity depends on sex, ethanol exposure, and reinforcement history.

Male mice were insensitive to changes in the action-outcome contingency on both the RI lever (A) and VR lever (B) regardless of exposure history. Female were sensitive to changes in the action-outcome contingency on the RI lever (C), but sensitivity depended on exposure history on the VR lever (D) such that ethanol exposure promoted action-outcome insensitivity. (**p < 0.01, *p < 0.05)

In females, a three-way ANOVA revealed that there were no effects of surgical history on degradation scores [surgery, F (1, 23) = 0.05571, p = 0.8155; ethanol x surgery x lever, F (2, 23) = 0.1443, p = 0.8664), and thus animals were collapsed across history of surgery for further analyses. Responding on the RI lever (Fig. 2C) was sensitive to changes in contingency: a main effect of degradation was observed [two-way rmANOVA, F (1, 26) = 11.48, p = 0.0023]. This indicates that all groups were sensitive to changes in action-outcome contingency. For responding on the VR lever (Fig. 2D), a significant exposure x degradation interaction was observed [two-way rmANOVA, F (2, 26) = 3.827, p = 0.0349] with post-hoc analyses revealing that degradation scores were significantly lower in the degraded condition as compared to the non-degraded condition for saline [Bonferroni’s post-hoc analysis, p = 0.0194] but not 1hr EtOH [p = 0.6938] or 4hr EtOH [p > 0.9999] female mice. These findings demonstrate that female mice were generally sensitive to changes in action-outcome contingency on both reinforcement schedules, but a history of chronic low-dose ethanol exposure promoted action-outcome insensitivity only on the lever that was previously reinforced on a VR schedule.

Low-dose ethanol effects on reward tracking during contingency degradation testing depend on reinforcement schedule history and differ by sex

To determine whether reward tracking was affected during the contingency degradation test, magazine checking behavior was also analyzed during the contingency degradation session in male and female mice. In males, reward tracking on the RI Lever (Fig. 3A) was not modulated by changes in contingency or ethanol exposure history [mixed-effects analysis, no effect of exposure, F (2, 30) = 0.5945, p = 0.5582; no effect of degradation, F (1, 29) = 0.2015, p = 0.6569; interaction exposure x degradation, F (2, 29) = 0.3580, p = 0.7022]. For responding on the VR lever (Fig. 3B), a main effect of exposure was observed [mixed-effects analysis, F (2, 28) = 5.067, p = 0.0132] with post-hoc analyses revealing that saline male mice had higher magazine checking behavior than 1hr EtOH [Dunnett’s post-hoc analysis, p = 0.0214] and 4hr EtOH [p = 0.0226] male mice. Consistent with the findings during training, male mice with a history of chronic low-dose ethanol exposure exhibited lower reward tracking behavior on the VR, but not RI, lever. To determine whether magazine checking behavior was associated with action-outcome sensitivity, degradation scores from the degraded test session were correlated with checking behavior for both the RI (Fig. 3C) and VR (Fig. 3D) lever. A comparison of fits test was used to determine whether data could be collapsed across groups for each graph. As the null hypothesis was not rejected in any instance, each dataset was analyzed as a single group (black line), but individual regression lines are also shown for each group (colored lines). There was no correlation between action-outcome sensitivity and magazine checking behavior on either the RI [R² = 0.0050, F (1, 31) = 0.1580, p = 0.6937] or VR [R² = 0.0106, F (1, 26) = 0.2786, p = 0.6021] lever in male mice and this was not impacted by ethanol exposure.

Figure 3. Behavioral strategy relates to action-outcome sensitivity in female mice.

In male mice, there was no effect of ethanol or contingency degradation on overall checking on the RI lever (A). However, on the VR lever (B) checking was reduced in ethanol-exposed mice as compared to saline-exposed. There was no relationship between checking and action-outcome sensitivity on either the RI (C) or VR (D) lever. In female mice, there was no effect of ethanol or contingency degradation on overall magazine checking on either the RI (E) or VR (F) lever. However, there was a relationship between checking and action-outcome sensitivity on both the RI (G) and VR (H) lever. (*p < 0.05)

In females, reward tracking was not modulated by ethanol exposure or contingency change when responding on the RI [Fig. 3E; two-way rmANOVA, no effect of exposure, F (2, 25) = 0.9482, p = 0.4009; no effect of degradation, F (1, 25) = 0.01147, p = 0.9156; no exposure x degradation interaction, F (2, 25) = 0.3722, p = 0.6930] or VR lever [Fig. 3F; mixed-effects analysis, no effect of exposure, F (2, 23) = 0.1950, p = 0.8242; no effect of degradation, F (1, 21) = 0.06102, p = 0.8073; no exposure x degradation interaction, F (2, 21) = 0.9350, p = 0.4083]. Correlational analyses between action-outcome sensitivity and magazine checking were also performed in female mice. On the RI lever (Fig. 3G), action-outcome sensitivity was associated with magazine checking behavior such that greater sensitivity to changes in the action-outcome contingency (e.g., degradation score below 0.5) was associated with a higher percentage of checking behavior [R² = 0.2980, F (1, 26) = 11.04, p = 0.0027] regardless of exposure history. Action-outcome sensitivity was also associated with magazine checking behavior on the VR lever in females in the same fashion as on the RI lever [Fig. 3H; R² = 0.2464, F (1, 22) = 7.192, p = 0.0136]. These findings suggest that females used reward tracking information and checking strategy to guide their responding, whereas males did not.

Discussion

Our findings demonstrate that chronic low-dose ethanol exposure altered behavioral strategy selection in mice. During training, male and female mice used different strategies to guide their behavior depending on reinforcement schedule, with greater tracking behavior on interval than ratio schedules. A history of ethanol exposure only impacted behavioral strategy in males, as demonstrated by reduced reward tracking/magazine training during training and CD testing on the VR lever. During CD testing, males were insensitive to changes in action-outcome contingencies, while sensitivity in females depended on reinforcement schedule and ethanol exposure history such that low-dose ethanol-exposed females exhibited reduced sensitivity to changes in action-outcome contingencies on the VR lever. Checking strategy was not impacted by CD, although ethanol-exposed male mice had lower checking behavior than ethanol-naïve male mice on the VR lever, consistent with the observed differences during training. In female – but not male – mice, reward tracking strategy was negatively correlated with action-outcome sensitivity.

In both males and females, magazine checking behavior during training depended on reinforcement schedule, such that checking behavior was higher on the RI than on the VR schedule. RI schedules have low temporal contiguity compared to VR schedules, meaning that there is less certainty about when reinforcer is going to be delivered in relation to the amount of effort exerted (Garr et al., 2020; Perez & Dickinson, 2020). This reduced contingency between pressing behavior and reward delivery may drive the increased reward checking behavior that was observed on the RI schedule compared to the VR schedule. This indicates also that the mice detected the different reinforcement schedules for each lever.

Magazine checking behavior was correlated with degradation scores in female mice, such that females which were more sensitive to changes in the action-outcome contingency had greater magazine checking behavior. These two measures were not associated in male mice. Considering that, as a population, females were contingency sensitive while males were contingency insensitive, these findings suggest that reward tracking via magazine checking is a strategy associated with flexible reward seeking. While reward tracking still occurs in inflexible reward seeking, this information may not be used to guide behavior.

Chronic low-dose ethanol exposure only shifted magazine checking behavior in males, such that checking was lower in ethanol-exposed as compared to ethanol-naïve male mice during both training and CD testing. Further, this was mainly restricted to the VR lever, on which checking was lower in ethanol-exposed male mice during CD testing. Overall magazine checking behavior did not shift as a result of changes in the action-outcome contingency (in the degraded vs. nondegraded condition) in either males or females, suggesting that the reward tracking strategy used during testing was based on prior learning and was not updated to reflect changes in this contingent relationship.

During CD testing, ethanol-naïve male mice were insensitive to changes in the action-outcome contingency regardless of exposure or reinforcement schedule history, which prevented the ability to detect whether ethanol exposure promoted contingency insensitivity in males. Ethanol-naïve female mice, however, were sensitive to changes in action-outcome contingencies regardless of reinforcement schedule history. A history of chronic low-dose ethanol exposure promoted action-outcome insensitive reward seeking only on the VR lever in female mice, though why this was only observed on the VR lever is unclear. Chronic ethanol exposure promotes sucrose habit formation (Barker et al., 2020; Corbit et al., 2012), but this depends on exposure method and timing of exposure in relation to learning. In a subset of these findings, only males exhibited facilitated habit formation following chronic ethanol exposure (Barker, Bryant, et al., 2017) while others observed increased reliance on habits in both males and females (Renteria et al., 2020), suggesting that sex may further interact with timing and dose effects. The effects of low-dose ethanol are distinct from higher doses (Cui & Koob, 2017), so it may be that the length of exposure was not sufficient to promote habit formation. Notably, there were no effects of exposure timing on contingency sensitivity or reward tracking in either males or females.

Exposure timing-dependent effects of ethanol on sucrose reward motivation have been observed under the same training and exposure conditions (Bryant et al., 2022), so this suggests that the effects of ethanol and exposure timing on behavioral strategy selection and motivation may occur via distinct underlying mechanisms. While there were no population level differences observed here, there was a lot of individual variability in sensitivity. Motivation and habit are considered separate processes, as the concept of motivation implies an inherent goal-directedness, while habit is the absence of goal-direction (Balleine & Dezfouli, 2019; Vandaele & Janak, 2018; Watson et al., 2022). However, there may be circumstances in which contingency impairment could support enhanced motivation (Gourley et al., 2016). Such comparisons and investigations between these different measures and how they are affected by ethanol exposure should be completed in the future.

Overtraining on RI schedules has historically been shown to promote action-outcome insensitivity (Dickinson et al., 1983), but in the current study only the males developed contingency insensitivity when responding on the RI schedule when looking at the whole population. Many of these seminal studies only used males, suggesting that it may not be possible to generalize these traditional findings across sex. Notably, sex differences in sucrose habit formation have been reported before, with female rats or chromosomal female mice exhibiting more rapid sucrose habit formation than males (Barker, Bryant, et al., 2017; Quinn et al., 2007). However, these previous studies used sensitivity to changes in outcome value as a measure of habit, which may suggest loss of sensitivity to outcome value and action-outcome sensitivity occurs separately, as has been previously reported and hypothesized (Corbit et al., 2002; Schreiner et al., 2019). Alternatively, it may be that the length of RI training here did not promote the formation of action-outcome insensitivity in females or that the combination of ratio and interval schedules within the same training sessions promoted the maintenance of goal-directed actions in females.

CD testing was performed on the same day for both levers, with each lever presented consecutively, as in the previous training sessions. One possibility is that action-outcome sensitivity was generalized to both levers during the test, instead of being distinct based on reinforcement schedule history. It is possible that this may occur in a sex-specific manner such that males engendered bias toward habitual behavior, and females exhibited a bias toward goal-directed actions. That the mice used different reward tracking strategies during training depending on schedule supports that this is not related to difficulties in schedule detection or dissemination on the two levers. Further, the schedules and duration of training used in these experiments are consistent with other behavioral training paradigms associated with goal-directed and habitual behavior (Garr et al., 2021; Gremel & Costa, 2013). However, these studies used outcome devaluation instead of contingency degradation as a measure of habit. One possibility is that this limited testing procedure was not sufficient for male mice to track the change in contingency, and more protracted training on these schedules may reveal differential sensitivity to contingency changes.

One way that these studies differed from other publications which use the dual lever schedule training (Barker, Glen, et al., 2017; Gremel & Costa, 2013) is that these mice underwent daily injections following training. While care was taken to minimize stress, this extended handling may have impacted habit formation in opposing direction in males and females. Others have reported either acute or chronic stress facilitates reliance on habits (Dias-Ferreira et al., 2009; Gourley et al., 2012; Schwabe et al., 2007; Schwabe & Wolf, 2009), which may have resulted in a greater contingency insensitivity in our saline-injected control mice. Notably, to our knowledge, preclinical studies have not assessed the effects of stress on habits in female mice, leaving the possibility for sex specific consequences of chronic stress on habitual response strategies.

While interval schedules are known to promote inflexible behavior and ratio schedules are known to maintain flexible behavior (Adams & Dickinson, 1981; Barker, Glen, et al., 2017; Dickinson et al., 1983; Gremel & Costa, 2013), this is not a fixed binary. Others have observed that overtraining on RI schedules yields goal-directed behavior (Garr et al., 2020) on a devaluation test. Similarly, our findings suggest that there may be additional factors which alter the detection of and/or progression to habitual behavior on RI and VR schedules. Indeed, there are multiple factors that could be interacting to produce the patterns of contingency sensitivity observed here, including sex, exposure history, training history, stress, and same day testing, all of which has been discussed individually above.

Chronic low-dose ethanol affected reward seeking in both males and females, but on different aspects of the task. Magazine checking behavior was sensitive to chronic low-dose ethanol-induced disruption in males but not females, while action-outcome sensitivity was disrupted by ethanol in females but not males. This in part may reflect sex differences that were present at baseline, especially in the CD task. Males and females may use different contextual and reward-related information to guide their behavior as females were found to have used consistent strategies during reward learning, whereas males shifted their strategies frequently (Chen et al., 2021). The findings presented here seem to support that females used a specific strategy (in this case reward tracking behavior) to guide their responding on the CD test, whereas males did not. This may be impacted by chronic low-dose ethanol exposure, leading to differences in behavioral phenotype. Male and female mice may also have generalized differences in sensitivity to low-dose ethanol effects on reward seeking (Bryant et al., 2022; Jury et al., 2017). It is also possible that higher doses are required for similar behavioral outcomes in females. This is potentially consistent with increased voluntary ethanol consumption in females than in males (Barker et al., 2010; Becker & Koob, 2016; Rhodes et al., 2007). However, we have previously observed that female mice reach higher peak blood ethanol concentrations (BECs) at this dose and route of administration (0.5 g/kg; i.p.) as compared to males (Nothem et al., 2023), which suggests that sex differences in low-dose sensitivity are not related to lower BECs.

Overall, these findings demonstrate that there are both individual and group differences in the strategies that mice use to guide their behavior. These results add to additional literature suggesting that males are generally more sensitive to the long-term effects of chronic low-dose ethanol exposure history on reward seeking than females. Low-dose ethanol may affect different reward-related circuits in males and females, which could contribute to the differences in ethanol effects observed here. Some differences in the neural circuit effects of low-dose ethanol have been observed (Bryant et al., 2023; Cui & Koob, 2017; Randall et al., 2020), but this has yet to be thoroughly investigate in both male and female animals and as a result of chronic, as opposed to acute, exposure. Identifying the neurobiological substrates impacted by chronic low-dose ethanol exposure may uncover new therapeutic targets to assist in preventing the development of inflexible behaviors that is associated with transition from casual drinking to AUD.