Habitual alcohol seeking: modeling the transition from casual drinking to addiction

Abstract

The transition from goal-directed actions to habitual ethanol seeking models the development of addictive behavior that characterizes alcohol use disorders. The progression to habitual ethanol-seeking behavior occurs more rapidly than for natural rewards, suggesting that ethanol may act on habit circuit to drive the loss of behavioral flexibility. This review will highlight recent research that has focused on the formation and expression of habitual ethanol seeking, and the commonalities and distinctions between ethanol and natural reward-seeking habits, with the goal of highlighting important, understudied research areas that we believe will lead toward the development of novel treatment and prevention strategies for uncontrolled drinking.

1. Introduction

Recent statistics from the NIAAA indicate that over 80% of Americans have used alcohol in their lifetime, and as many as 50% have consumed alcoholic beverages in the past month (SAMHSA, 2012). Despite the prevalence of alcohol use, most people who use alcohol do not go on to develop alcohol use disorders; only less than 10% of the population met these criteria in 2011-2012 (SAMHSA, 2012). Most alcohol users remain casual consumers – they seek out and drink alcohol because of its rewarding properties, but their drug taking remains controlled and goaldirected. Nonetheless, as alcohol is repeatedly consumed and paired with environmental stimuli, alcohol seeking may become habitual. That is, for some individuals, alcohol seeking is no longer exclusively mediated by the reinforcing properties of alcohol – the taste or enjoyable components of drinking – but rather by exposure to cues previously associated with alcohol. Therefore, because this behavior is unrelated to the value of the alcohol or to the outcome of the behavior, unlike goal-directed alcohol seeking, habitual alcohol seeking is no longer sensitive to changes in the reinforcing value of alcohol. If alcohol becomes devalued, possibly by association with illness, a goal-directed individual will discontinue drinking, whereas, a habitual individual will continue their alcohol-seeking behaviors even when they no longer desire alcohol. While habitual alcohol seeking itself is not diagnostic for dependence, this uncontrolled behavior may be related to unplanned excessive consumption and the inability to regulate alcohol seeking despite devaluation of the reinforcer through pairings with illness that in part characterize dependence.

In animal models of habitual alcohol seeking, animals are trained to make an instrumental response, such as nosepoking, to receive access to an ethanol reinforcer. As an instrumental response is acquired, it can shift from a goaldirected response, which is controlled by an expected outcome (i.e., an actionoutcome association is acquired) to a habitual response, which is controlled by an association between antecedent environmental stimuli and a behavior (i.e., a stimulus-response association is formed). Goal-directed instrumental learning is flexible and is believed to be under the control of the prefrontal cortex (PFC). During this time, behavior is sensitive to the value of the outcome; when an outcome is made less valuable, a reduction in response performance occurs (Colwill & Rescorla, 1985). After extended training on specific reinforcement schedules, however, executive control is diminished and the behavior is under the control of environmental stimuli. Accordingly, as training progresses, instrumental performance becomes increasingly insensitive to outcome devaluation (A. Dickinson, 1985). This criterion provides an objective means to discriminate between goal-directed actions and stimulus-driven habits: after a reinforcer has been devalued, a reduction in responding is indicative of goal-directed behavior, whereas maintenance of responding indicates habit (Dickinson, 1985).Importantly, habitual behavior is not in and of itself maladaptive. In addition, the formation of habitual behavior does not preclude the use of goal-directed strategies in reward seeking under certain conditions. It is only when these behaviors result in uncontrolled performance of undesirable behaviors such as drug seeking that habits are problematic.

Habitual behavior can be assessed using a variety of experimental methods that modify reinforcer value or the stimulus-response contingency. Most frequently, outcome devaluation and contingency degradation are used to decrease reinforcer value in rodent models of habitual behavior. In outcome devaluation, the reinforcer is made less valuable either by training the animal in a conditioned taste aversion (CTA) paradigm or by specific satiety paradigms. In CTA, animals learn to associate reinforcer consumption with LiCl-induced malaise. After learning that reinforcer consumption is related to illness, a goal-directed animal will discontinue responding for the devalued reinforcer. However, habitual animals continue to respond for the illness-paired reinforcer, even though they will typically not consume the reinforcer in a free-access situation. This typical pattern suggests that it is the reinforcer seeking that has become habitual, rather than reinforcer taking, but it is important to consider how drug seeking and taking may be differentially impacted by the transition away from goal-directed actions. In specific satiety devaluation, animals are allowed free access to either the experimental reinforcer or a control substance (e.g., different type of food) prior to the session. Animals that receive the experimental reinforcer are considered to have the reinforcer devalued and will discontinue responding when goal-directed. Animals receiving access to the control food or substance still value the experimental reinforcer and will continue responding whether goal-directed or habitual. Specific satiety devaluation has the advantage of not requiring new learning, but may be confounded in experiments involving ethanol- or drug-seeking habits. Though less frequently used, habitual behavior also can be assessed by increasing the value of the reinforcer, typically by increasing the deprivation state (e.g., Quinn, Pittenger, Lee, Pierson, & Taylor, 2013). Here, it is expected that the animals for which the value was inflated would respond at higher rates when goal-directed, but would remain insensitive to this change in value if habitual. Instrumental habits for food reinforcers can also be measured in human populations in the experimental setting. These experiments have also typically used specific satiety outcome devaluation (e.g., Tricomi, et al., 2009; Valentin, et al., 2007). Tests of habitual responding after outcome devaluation are typically performed in extinction, that is, responses are not reinforced by outcome delivery. This is done in part because outcome-mediated behaviors are then driven by the representation of outcome value based on cached outcome value which may be relevant to stimulus-driven behaviors (Schneck & Vezina, 2012). Importantly, these tests are sufficiently brief that extinction learning is not expected to drive reduction in behavior as extinction and habitual behavior appear to be distinct.

Unlike outcome devaluation, contingency degradation paradigms do not require the animal to learn about the reinforcer value or to update the value. Rather, this assessment of habitual behavior capitalizes on the habitual animal's insensitivity to changes in the action-outcome relationship to determine response strategy. A number of methods have been used to degrade action-outcome contingencies. Most commonly, this involves making action and inaction equally predictive of reinforcer delivery. Alternatively, others have provided noncontingent reinforcement without explicitly reinforcing either action or inaction. Here, a goaldirected animal is again expected to reduce responding once they have learned the new action-outcome relationship, while a habitual animal will continue responding. Sensitivity to contingency degradation has been measured either during degradation sessions in which reinforcers are available, or during a probe test in extinction conditions. Additional modifications of the action-outcome relationship have been used to investigate response strategy selection. For example, several groups (e.g., Yu, et al., 2009) use a paradigm in which the established action-outcome contingency is reversed: Animals only receive access to the reinforcer when they withhold the previously reinforced response.

In some instances, different measures of habitual responding do not correspond. For example, at least two separate groups have reported instances in which animals were differentially sensitive to outcome devaluation and contingency degradation (Corbit, et al., 2002; Lex & Hauber, 2010a; Lex & Hauber, 2010b). In these instances, animals remained sensitive to outcome devaluation, and appeared goal-directed in this measure, but were insensitive to the degradation of the action-outcome relationship. More recent work has suggested that loss sensitivity to outcome devaluation and contingency degradation may be related not to reinforcer type (i.e., drug vs natural reward) but also of the precise quality of the reinforcers (i.e., sucrose concentration) (Shillinglaw, Everitt, & Robinson, 2014). These data serve as a reminder that it is important to consider the precise aspects of behavior that are being assessed by these different tests. For a behavior to be truly habitual, one would predict insensitivity to both the change in action-outcome relationship as well as change in outcome value. Outcome devaluation does not require an animal to form a new action-outcome relationship, but rather only to update the value of the outcome. Sensitivity to contingency degradation, on the other hand, requires the animal to update their knowledge about their behavior and its result: the action-outcome relation. Thus, persistent responding during contingency degradation can result from the influence of the stimulus-response relation, or from failure to update the action-outcome relation. These alternative conclusions are equally valid and should be considered when interpreting results.

Most often, habit studies use these paradigms to investigate habits for a natural reinforcer, and much of what we know about the neurobiology of habitual responding has been determined for natural reinforcers as described below. However, a few studies looking at drug and ethanol habits in animals exist (Belin & Everitt, 2008; Miles, Everitt, & Dickinson, 2003; Zapata, Minney, & Shippenberg, 2010). Several labs have published on habitual ethanol seeking using a lithium chloride devaluation paradigm (Barker et al., 2014; Barker, Torregrossa, Arnold, & Taylor, 2010; Dickinson, Wood, & Smith, 2002), specific satiety devaluation (Corbit et al., 2012; Hay, Jennings, Zitzman, Hodge, & Robinson, 2013) or contingency degradation (Barker et al., 2014) but to our knowledge, no outcome inflation data have been published related to alcohol seeking habits. Importantly, ethanolseeking habits differ from food seeking habits in that the transition from actions to habits occurs more readily (Corbit et al., 2012; Dickinson et al., 2002). It has further been shown using a novel paradigm, that exposure to contexts paired with ethanol can disrupt otherwise goal-directed behaviors (Ostlund, et al., 2010). The mechanisms by which ethanol acts on the habit circuitry to facilitate the promotion of habitual responding are not yet understood, but work from Corbit and colleagues indicates that ethanol exposure can also facilitate the development of sucroseseeking habits (2012). Indeed, though a causal role cannot be assessed, recent data from human alcoholics suggests that aberrant habit formation is a hallmark of the disease (Sjoerds et al., 2013). We, and many others (e.g., Robbins & Everitt, 1999), expect that this area to be a fruitful line of neurobiological research that will help us to understand the precise mechanisms of the transition from casual drug use to addiction, therefore enabling the design of better methods of treatment and prevention.

2. The neurobiology of habitual behavior

Though little work has been done to elucidate the neurobiology supporting habitual responding for ethanol, rodent and human work has been performed by a number of groups to determine the neuroanatomical substrates of food habits. What data do exist explicitly investigating the neuroanatomical substrates of habitual alcohol seeking suggest similarity to natural reward (Corbit et al., 2012). Lesion studies have indicated that the competition between goal-directed and habitual behavior depends upon signaling within limbic corticostriatal systems. More recently, work in humans using fMRI has shown that homologous structures supports response strategy selection in tasks comparable to those used in animal models (Balleine & O'Doherty, 2010). In addition to anatomical determinants, key neuromodulators have been identified that are crucial for the normal goal-directed or habitual responding (e.g. Hitchcott, et al., 2007; Yu et al., 2009). Alcohol use may produce changes in synaptic functions, intracellular signaling pathways, and dendritic morphology (Mulholland & Chandler, 2007; Nestler, 2001) and may underlie aberrant plasticity (Berke & Hyman, 2000; Hyman & Malenka, 2001) within these circuits that leads to accelerated habitual responding.

Alcohol interactions with habit circuitry

Though much work investigating the loss of behavioral flexibility has focused on food reinforcers, the transition to habitual alcohol-seeking has been shown to be distinct from that seen for natural reinforcers. Dickinson et al. (2002) demonstrated that the transition from goal-directed to habitual alcohol responding occurs more rapidly than for a natural reinforcer. Importantly, this does not appear to result solely from exposure to alcohol, as the same mice remained goal-directed for a food reinforcer, suggesting that the accelerated transition to habit is specific to the response for alcohol. In contrast, additional research on habit formation has suggested that alcohol exposure can facilitate habit formation for natural rewards (Corbit et al., 2012). Importantly, the levels of alcohol consumption and the timing are distinct in these experiments, and it is important to consider that the precise timing of alcohol consumption in relationship to learning and performance of a behavior may drive the changes in the limbic corticostriatal circuits that facilitate the loss of behavioral control. In recent work, Ostlund and colleagues (2010) used a novel behavioral method to show that exposure to a context that was previously paired with alcohol can disrupt goal-directed behaviors for natural reinforcers. This suggests that contexts and cues that predict alcohol reinforcement may be sufficient to disrupt behavioral flexibility and promote the expression of stimulusresponse habits. The mechanisms by which these behaviors become automatic are less clear. Importantly, the work from Corbit and colleagues demonstrates significant overlap between the structures mediating the development of habitual alcohol seeking and indicates that, as for natural rewards, dorsolateral striatum is a critical substrate of habitual ethanol seeking.

Alcohol exposure produces a number of changes at multiple signaling systems throughout numerous brain regions, and many are consistent with promoting habit formation. For example, alcohol is known to cause changes in corticostriatal circuitry, including impaired prefrontal cortical function (Sullivan & Pfefferbaum, 2005) and possibly aberrant input to the ventral striatum (Szumlinski et al., 2007). Additionally, alcohol exposure has been shown to increase expression of the NR2B subunit of NMDA receptors in the dorsolateral striatum, potentially producing enhanced signaling consistent with habit formation (Wang et al., 2007, 2010). Bath application of alcohol on striatal slices results in decreased long-term potentiation in the dorsomedial striatum, where decreased activity is associated with habit formation (Yin, Park, Adermark, & Lovinger, 2007). In addition to changes in corticostriatal connectivity and function, alcohol exerts effects on a number of signaling systems that may interact to facilitate the loss of behavioral flexibility.

3. Neuroanatomical structures

Medial prefrontal cortex

Response strategy depends upon interactions between a number of neurocircuits, including the subregions of the prefrontal cortex (PFC) and striatum. The PFC has a critical role in inhibitory control (Roberts & Wallis, 2000) and is crucial for decision-making and response-selection, which are notably impaired in alcoholics and in drug addicts (Jentsch & Taylor, 1999; Bechara, 2003; Hildebrandt, et al., 2006; Schoenbaum, et al., 2006). In general terms, the PFC is thought to mediate cognitive, flexible actions while the dorsolateral striatum guides automatic behaviors. The medial PFC can be subdivided into the more ventral infralimbic and dorsal prelimbic regions, which in rodent studies have been shown to have distinct influences on the acquisition and expression of goal-directed actions. The prelimbic PFC, with projections to the core of the nucleus accumbens and the dorsomedial striatum, has been shown to be critical to the formation of goaldirected behaviors – lesioning or inactivating the prelimbic PFC prior to training results in the premature expression of stimulus-response habits (Balleine & Dickinson, 1998; Killcross & Coutureau, 2003). Further work from two different groups showed that while inactivating the prelimbic PFC prior to training results in insensitivity to change in outcome value, inactivations at the time of testing did not impair flexible behavior (Ostlund & Balleine, 2005; Tran-Tu-Yen, et al., 2009), indicating that the prelimbic PFC is not involved in the storage of behavioral actions. Together, these findings suggest that prelimbic function is critical for the acquisition, but not expression, of goal-directed behavior.

The infralimbic PFC, which projects to the nucleus accumbens shell (e.g., McGeorge & Faull, 1989) as well as the amygdala (Sesack, et al., 1989), has been implicated in expression of goal-directed behavior. When the infralimbic PFC is lesioned prior to acquisition of a response, stimulus-response habits do not develop (Killcross & Coutureau, 2003); after even extended training, animals remain sensitive to outcome devaluation. Importantly, later research expanded on this finding, showing that inactivation of the infralimbic PFC after extended training, at a time point where control animals are habitual, results in the restoration of flexible behavior (Coutureau & Killcross, 2003). In addition, studies using optogenetic techniques allowing for online regulation of infralimbic activity have demonstrated that inhibition of projection neurons in the infralimbic PFC is sufficient to restore goal-directed behavior (Smith, Virkud, Deisseroth, & Graybiel, 2012). Importantly, these results indicate that even after the transition from goaldirected actions to stimulus-response habits, animals retain the ability to revert to a response strategy that depends on existing understanding of the action-outcome relationship, providing support the for the hypothesis that action-outcome information is not lost, but rather that the relationship has lost the ability to control behavior.

Nucleus accumbens

The ventral portion of the striatum is ideally situated in cortico-limbic-striatal circuits to play a critical role in attaching motivational significance to emotionally relevant stimuli. Indeed, the role of the nucleus accumbens (NAc) in the establishment and control of reward seeking behavior has been well-established (L H Corbit, Muir, & Balleine, 2001; B J Everitt et al., 1999; Saddoris, Sugam, Cacciapaglia, & Carelli, 2013; Salamone & Correa, 2012; Susan R Sesack & Grace, 2010; Yin, Ostlund, & Balleine, 2008). NAc can be further divided into core (NAcC) and shell (NAcS) subregions, based in part on their separable anatomical connections, which play distinct roles in appetitive behavior. Loss of the NAcC results in insensitivity to outcome devaluation, but also a global reduction in responding (Corbit, Muir, & Balleine, 2001). Interestingly, lesions to the core did not appear to result in insensitivity to contingency degradation, suggesting that NAcC may be critical for value-guided behaviors, but not in using action-outcome contingencies to guide behavior. Alternatively, loss of the NAcS did not impact the ability to express goaldirected behavior, though its requirement for the expression of habitual behavior has not to our knowledge been assessed. Interestingly, a critical role for NAc serial connectivity with dorsal striatum structures has been identified in stimulusmediated reward seeking; functional disconnection of NAcC-midbrain-dorsolateral striatum disrupted stimulus-driven cocaine seeking (Belin & Everitt, 2008). These data suggest that NAcC input to midbrain dopamine structures, and their subsequent serial connectivity with more dorsal striatum is critical for appropriate reward seeking behaviors, suggesting that NAcC may act both to integrate input from corticolimbic inputs, but also to output that information to dorsal striatum subregions.

Dorsal striatum

The dorsal striatum can be subdivided into a dorsolateral region that contributes to a sensory network supporting habitual, stimulus-response (S-R) behaviors as well as a dorsomedial associative network that mediates flexible, action-outcome behavior (Coutureau & Killcross, 2003; Killcross & Coutureau, 2003; Yin, et al., 2004). This division is supported by behavioral (Featherstone and McDonald, 2004; Yin et al., 2004; Faure et al., 2005; Yin et al., 2005a; Yin et al., 2005b; Yin et al., 2006, Tricomi et al., 2009), electrophysiological (Barnes, et al., 2005; Carelli, et al., 1997; Jog, et al., 1999; Kimchi, et al., 2009; Stalnaker, et al., 2010; Tang, et al., 2007), and neuroanatomical data. The subregions of the dorsal striatum communicate via a series of connections with the midbrain dopamine neurons in a unidirectional fashion, such that ventral striatum impacts the activity of more dorsal regions, but the converse does not occur (Haber, et al., 2000).

The dorsomedial striatum receives projections from the prelimbic PFC (e.g., Groenewegen, et al., 1990) which, as discussed above, has been shown to be critical for the acquisition, but not expression, of goal-directed behavior (Balleine & Dickinson, 1998; Killcross & Coutureau, 2003; Ostlund & Balleine, 2005; Tran-Tu-Yen et al., 2009). The acquisition of goal-directed behaviors has recently been shown to drive cell-type specific plasticity in the dorsomedial striatum, such that opposing plasticity occurred in dopamine D1- and D2-receptor containing neurons (Shan, Ge, Christie, & Balleine, 2014). Appropriate regulation of cytoskeletal proteins within this region appear to be critical for the acquisition of new actionoutcome learning (Gourley, Olevska, Gordon, & Taylor, 2013). The effects of lesions of the dorsomedial striatum depend on the anterior-posterior location. Specifically, Yin and colleagues (2005) showed that the lesions of the posterior dorsomedial prevents the expression of goal-directed behavior, resulting in the premature formation of behavior that is insensitive to outcome devaluation or changes in the action-outcome relationship (Yin, et al., 2005). Further, inactivation of this site after training also disrupted the expression of goal-directed behavior. Lesions of the anterior portion of the dorsomedial striatum had no effect on the acquisition or expression of goal-directed behavior. It has been speculated by Yin et al. that the distinct effects of these lesions on behavior are determined by their differential connectivity to other regions known to be important for behavioral flexibility, including the basolateral amygdala.

The more lateral portion of the dorsal striatum, which receives extensive inputs from sensorimotor cortex and projections to brainstem regions (Graybiel, 1998), is critical for the formation and expression of stimulus-response habits. Lesions of the dorsolateral striatum prior to training result in persistent sensitivity to outcome devaluation (Yin, et al., 2004). Further, inactivation of the dorsolateral striatum after acquisition of a behavior restores sensitivity to contingency degradation (Yin et al., 2006). Together, these findings indicate that the dorsolateral striatum is required for both the formation and expression of stimulus-response habits, and that in its absence, animals cannot form the representation of habits, or are unable to express previously acquired habits.

Amygdala

Generally thought to be involved in emotional learning, the amygdala can be divided into two cytoarchitecturally and neuroanatomically distinct subregions that are critically involved in cue-mediated behaviors: the central nucleus (CeA) and the basolateral amygdala (BLA). Though much of the investigation of the role of the amygdala has focused on aversive learning, these nuclei also have key roles in CS associations with reward. CeA, with extensive projections to midbrain dopaminergic regions, is thought to be necessary for the attribution of incentive motivation to stimuli (e.g., Balleine & Killcross, 2006; Swanson, 1982). The BLA, likely due to its connectivity with nucleus accumbens and prefrontal regions, has been shown to have a role in maintenance of S-O relationships (e.g., Holland et al., 2002; Pickens et al., 2003), and lesions may prevent appropriate updating of outcome value (Hatfield et al., 1996). Perhaps unsurprisingly given its network connectivity and role in cue-mediated behavior, recent work has highlighted a role for the CeA in the expression of stimulus-response habits. Lingawi & Balleine (2012) demonstrated that while lesions of either anterior or posterior CeA abolish Pavlovian-to-instrumental transfer lesions of only the anterior portion of CeA prevented the transition from strategic responding to habit. To further probe the role of CeA, a disconnection strategy was used to show that CeA interaction with the dorsolateral striatum are critical for the loss of behavioral flexibility as measured in an outcome devaluation paradigm, consistent with a role of CeA in the attribution of motivating properties to reward-paired stimuli. Our own data have implicated individual differences in amygdalar serotonin signaling in the expression of alcohol seeking habits – specifically, we have seen that individual differences in epigenetic regulation of 5HT3 signaling in the amygdala are predictive of the formation of alcohol seeking habits (Barker et al., 2014).

Circuit level control of habitual behavior

As touched upon above, a number of highly interconnected neuroanatomical substrates have been implicated in the acquisition and expression of goal-directed and habitual response strategies. While these target regions have been identified, a thorough assessment of the precise circuits that are involved in habitual ethanol seeking has not, to our knowledge, been performed. The increasing popularity of disconnection strategies and selective optogenetic manipulation is enabling a more refined understanding of circuit-level activity in reward-seeking habits. While these studies are gaining traction (Belin & Everitt, 2008; Lingawi & Balleine, 2012; Smith & Graybiel, 2013, 2014; Smith et al., 2012), this level of analysis has not yet been performed in the alcohol field. Because chronic and acute ethanol exposure can differentially impact a number of these structures, it is critical to not only identify these circuits, but also to determine how alcohol impacts components of this circuitry to drive aberrant reward-seeking behavior.

It is tempting to speculate that the differential role of the infralimbic and prelimbic PFC are related to their distinct projection targets and the differential roles of these structures in the regulation of reward-seeking behaviors. NAcC and NAcS are differentially innervated by the mPFC with NAcS receiving extensive glutamatergic projections from the infralimbic PFC, as opposed to NAcC where prelimbic projections predominate. As the NAc is thought to be largely responsible for integrating reward-related information from other corticolimbic structures, it is possible that NAc inputs from the PFC as well as the amygdala and ventral hippocampus are all critical to successful use of these contingencies to develop goal-directed behavior and to use those contingencies to guide instrumental action across performance of a behavior. In addition, infralimbic and prelimbic PFC differentially innervate the amygdala, though some discrepancy regarding the precise nuclei targeted within the amygdala exist. In particular, it has been disputed whether infralimbic PFC projects to the basolateral amygdala (Cassell & Wright, 1986; Mcdonald, Mascagni, & Guo, 1996), though prelimbic projections to this nucleus are more consistently identified. In addition, infralimbic PFC is thought to project to the GABAergic cells that compose the intercalated nucleus. Prefrontal interaction with the amygdala has been shown to be critical for the expression and acquisition of both fear and reward seeking behaviors - with infralimbic PFC appearing to be critical for the acquisition and expression of extinction learning, and the prelimbic PFC performing the opposite function - though a role for these circuits in habitual behavior has not been elucidated. It is possible that rather than their separate projection targets to the NAc driving opposite functions for infralimbic and prelimbic PFC in the acquisition and expression of goal-directed behavior, that their separable projections to the amygdala with its critical role in driving stimulus-mediated behavior. In particular, as CeA interactions with the dorsolateral striatum appear to be necessary for normal habit formation, it is possible that infralimbic PFC regulates activity in this network either directly through projections to the CeA or indirectly through projections to the GABAergic intercalated nucleus with projections to the CeA. Careful analyses of these, and additional circuits, the acquisition and expression of habitual behavior, and how these targets are impacted by ethanol exposure, is likely to be very informative in understanding how ethanol seeking can become dysregulated and inflexible.

4. Neurotransmitter systems

Glutamate

Prominent theories have identified glutamate dysregulation as a primary mechanism of the development of addictive behavior (Kalivas, 2009). In particular, these theories have identified dysregulation of glutamatergic prefrontal projections to a number of the subcortical structures described above resulting in lack of behavioral control. Though in its infancy, the explicit investigation of the role of regulation of glutamate signaling in habitual behavior has been receiving growing attention given its known role on the regulation of behavioral flexibility and general learning and memory processes. For example, recent work has shown that Nacetylcysteine can rescue cocaine-facilitated habits (Laura H Corbit, Chieng, & Balleine, 2014). While its effects are not direct, N-acetylcysteine acts to increase nonsynaptic glutamate release and thus, glutamate activity at extrasynaptic receptors, implicating this signaling in the development of habitual food seeking. Aside from habitual behavior, glutamatergic PFC projections to the amygdala and nucleus accumbens are necessary for reinstatement for many drugs of abuse (McFarland et al., 2003; LaLumiere and Kalivas, 2008), but the disconnection studies have not been performed in habits. However, as discussed above, work from the Graybiel group has used optogenetic strategies to show a selective role for glutamatergic projections from IL are critical or the expression of habitual behavior (Smith et al., 2012). AMPA and NMDA signaling in the ventral striatum have also been shown to inhibit cue-induced reinstatement of ethanol seeking (Backstrom and Hyytia, 2004; Schroeder et al., 2008), but this also has not been investigated in habitual behavior.

While much of the research that forms the basis of this model has been based on work studying cocaine dependence, a significant literature has also identified dysregulation of glutamate signaling in alcoholics and in preclinical models of dependence. Chronic alcohol exposure has been shown to produce significant alterations in glutamatergic signaling. In particular, chronic ethanol is thought to induce a hyperglutamatergic state in select subregions including the nucleus accumbens(Griffin III, Haun, Hazelbaker, Ramachandra, & Becker, 2013; Kalivas, 2009). In addition to alterations in global glutamate levels, ethanol exposure also alters glutamate receptor and transporter expression and function in a timecourse dependent manner. Acutely, ethanol inhibits NMDA (and to a lesser extent, AMPA) receptor function in a number of brain regions (e.g., Lovinger, White, & Weight, 1989; Woodward, 2000), including the cortex and subcortical projection areas including the striatum and amygdala (Calton, Wilson, & Moore, 1999; Yaka, Phamluong, & Ron, 2003; Yin et al., 2007).

Data on NMDA receptor expression in alcoholics indicates increases in expression in human alcoholics in a number of brain regions, including the frontal cortex (Freund & Anderson, 1996, 1999). As alcohol inhibits NMDA activity, this increase in expression may serve as a compensatory mechanism. Expression of NMDA receptor subunits GluN2A and GluN2B have been shown to be altered after chronic ethanol exposure in rodent studies with in limbic corticostriatal circuitry. In particular, increases in GluN2B mRNA and reductions in GluN2A have been seen in rodent PFC after chronic alcohol exposure (Meinhardt et al., 2013), though these changes may not be paralleled in human alcoholics (Ridge, Ho, Innes, & Dodd, 2008). Data on NMDA receptor expression in suggest circuit specific alterations after chronic ethanol exposure that may drive alterations in behavioral flexibility (e.g., Kroener et al., 2012). Changes in expression have also been observed in the striatum and amygdala (Falco, Bergstrom, Bachus, & Smith, 2009; Floyd, Jung, & McCool, 2003; Läck, Floyd, & McCool, 2005; Obara et al., 2009), suggesting dysregulation of glutamate release and signaling throughout habit circuitry. More recent research has investigated the role of metabotropic glutamate receptors after chronic alcohol exposure. Interestingly, downregulation of mGluR2 in prefrontal projection neurons after chronic alcohol exposure has been observed in rodents and in human alcoholics, and has been causally associated with extinction of alcohol seeking (Meinhardt et al., 2013). Additionally, agonism of mGluR5 receptors in the medial PFC can similarly rescue chronic ethanol-induced deficits in extinction learning (Gass et al., 2014), though neither of these mechanisms have been investigated in habitual ethanol seeking. Under control conditions, glutamate signaling and release is tightly regulated. The large scale disruptions observed across chronic alcohol dependence and withdrawal are likely related to the loss of behavioral control that characterizes addictive behavior.

GABA

Similar to the challenges in ascribing precise roles for glutamate signaling in habit, as GABA is a major inhibitory neurotransmitter, much of the literature regarding its role in habitual behavior focuses more generally on regional inhibition (i.e., acute inactivation) rather than a selective role for GABA signaling in the development and expression of habitual reward seeking. As such, much of the work identifying the GABA system in habits has been indirect. Importantly, the principal neurons in the nucleus accumbens and dorsal striatum are GABAergic. Their role in the regulation of habitual and goal-directed behavior is well-established, and it is likely that disruption of GABA signaling in these structures can shift response strategy. As suggested by work investigating the disruption of serial connectivity between striatal subregions and midbrain dopamine neurons, outputs from these GABAergic cells, at least to dopaminergic neurons, are critically involved in the development of stimulus-mediated behaviors. Relatedly, inhibitory GABAergic interneurons regulate output in glutamatergic projection neurons throughout a number of structures including the PFC. Dysregulation of prefrontal GABA signaling likely drives a shift in activity within these structures ultimately impacting glutamatergic signaling in subcortical targets, suggesting that local or general dysregulation of GABA signaling through alcohol exposure is likely to alter the acquisition and expression of habitual behaviors.

Evidence for ethanol-induced changes in the GABAergic system is robust in both rodent models and in human alcoholics (c.f., Kumar et al., 2009; David M Lovinger & Homanics, 2007). The impact of ethanol on the GABA system depends on the time course of ethanol exposure, with different effects of acute and chronic exposure. Indeed, acute alcohol exposure appears to potentiate GABA_A receptormediated current, which may be involved in ethanol's reinforcing properties (Boehm et al., 2004; Harvey et al., 2002; Stephens, Pistovcakova, Worthing, Atack, & Dawson, 2005). Increased GABA release has been shown to result from acute ethanol exposure in a number of models, including dissociated neuronal cultures and in slice, though it appears to be unclear whether the effects of acute ethanol in vivo have site specific effects (ROBERTO 2003, 2004, 2004 Zhu and Lovinger (2006) Chronic ethanol exposure, however, may actually promote decreases in GABA_A receptor expression within the dorsal striatum (Adermark, Jonsson, Söderpalm, & Ericson, 2013), which is critical for the formation and expression of stimulus-response habits. Alterations in GABA release and GABA receptor signaling across chronic alcohol exposure are likely critically involved in both the acquisition of goal-directed and habitual behaviors, as well as their expression. However, it is not clear the direction in which simple shifts in GABA release and signaling would drive behavior. Much of the work investigating the effects of ethanol on GABA release and signaling are performed in slice or in culture, though a growing body of work has implicated changes in key structures within limbic corticostriatal circuits, including cortical and amygdalar neurons. Depending on the precise anatomical and connectivity of cells on which these ethanol-induced alterations occur, both acute and chronic ethanol exposure may act to shift the balance between goal-directed and habitual response strategies by inhibiting normal cellular plasticity required for learning and updating contingencies, or by more generally inhibiting activity in structures required for the expression of either goal-directed or habitual behaviors.

Dopamine

Dopamine has been shown to be critical for both goal-directed and habitual behavior. It is hypothesized that DA neurotransmission within these corticostriatal networks may be responsible for the ability to shift response strategy between stimulus-response habit and action-outcome behavior, and between dorsal and ventral circuits. This hypothesis is supported by considerable data. For example, Nelson and Killcross (2006) showed that sensitization to amphetamine prior to training, which alters both DA and glutamate function, facilitates the transition from goal-directed to habitual responding in an outcome devaluation test. Notably, chronic amphetamine after training did not impact sensitivity to outcome devaluation, indicating that the effects of amphetamine were specific to the acquisition of habitual responding and did not impact established goal-directed behavior.

In addition to systemic manipulations of dopamine, a number of studies have investigated the role of dopamine signaling in the corticostriatal network that supports goal-directed and habitual behavior. In order to investigate the role of dopamine in the dorsolateral striatum, Faure and colleagues (2005) lesioned the dopaminergic nigrostriatal projections prior to training animals to respond for food. Their findings indicate that in addition to delayed acquisition, animals lacking dopaminergic innervations of the lateral striatum remained sensitive to outcome devaluation after overtraining. Using paradigms that are not traditionally used to assess habitual behavior, a number of additional groups have shown using pharmacological methods that dopamine activity in the dorsolateral striatum is necessary for the expression of stimulus-response behavior (e.g., Robbins et al., 1990; Packard & White, 1991; Vanderschuren et al., 2005).

Our lab has investigated the role prefrontal dopamine in habitual behavior by infusing dopamine into the infralimbic PFC of animals that had been trained to respond habitually for food. Our findings indicate that dopamine infusions in the ventromedial, but not dorsomedial, PFC were able to restore sensitivity to outcome devaluation and reverse habitual responding (Hitchcott et al., 2007). Our more recent findings suggest that exogenous dopamine may be acting through dopamine D2 receptors in the infralimbic PFC to restore goal-directed behavior, as infusions of a D2 agonist were able to restore goal-directed behavior (Barker, Torregrossa, & Taylor, 2013). Together with the described role for dopamine in the dorsolateral striatum, these data indicate that dopaminergic signaling in distinct components of the corticostriatal circuitry promote differential response strategy and could act as a switch between goal-directed actions and habitual responding.

ERK and Dopamine-ERK interactions

Dopamine signaling may underlie both long-term potentiation and long-term depression critical for striatum-dependent learning (e.g., Faure, et al., 2005; Shen, et al., 2008). Dopamine signaling through D1 and D2 dopamine receptors acts through a number of downstream targets, including activation of ERK by phosphorylation. It has been demonstrated that increases in the p42 isoform of ERK in the striatum can enhance striatum-dependent learning processes (Ferguson, et al., 2006), identifying ERK activity as a likely target through which dopamine acts to influence behavioral response strategy. Shiflett and colleagues (2010) demonstrated that while ERK phosphorylation increases in both the posterior dorsomedial striatum and the dorsolateral striatum across the initial acquisition of a goal-directed behavior, but over extended training, this increase is maintained in the posterior dorsomedial striatum, but lost in the dorsolateral striatum. By inhibiting ERK pharmacologically, the authors demonstrated that ERK activity in the posterior dorsomedial, but not dorsolateral, striatum is required for performance of goal-directed behavior (Shiflett, et al., 2010)

Alcohol and dopamine

While substantial evidence exists linking alcohol consumption to impaired prefrontal cortical function, the specific neurobiological effects of alcohol on neuronal activity are not well understood. Human data consistently identifies abnormalities in the dopaminergic system of addicts, and in particular, Volkow and colleagues have shown human data indicating decreased expression of the D2 dopamine receptor in the striatum of alcoholics, and further that elevated levels of D2 might be protective for at-risk individuals (Volkow et al., 2006). Tu et al. (2007) have shown that alcohol inhibits persistent activity and spike firing within PFC neurons in a dopamine dependent manner. In addition, chronic alcohol exposure may cause a sensitized increase in dopamine and glutamate release in the nucleus accumbens in response to an alcohol challenge, suggesting altered prefrontal cortical input to the accumbens (Szumlinski et al., 2007). Recent work has shown that chronic ethanol exposure results in impaired dopamine D2/D4 (but not D1) receptor signaling in the vmPFC, which can impair behavioral flexibility (Trantham-Davidson et al., 2014). As our own work has shown that D1/D2-type signaling in the in infralimbic PFC is critical for the expression of habitual food seeking (Barker et al., 2013), we expect that dysregulation in vmPFC D2-like activity after chronic ethanol exposure could facilitate the expression of habitual reward seeking.

Acute alcohol and other drugs of abuse are known to independently produce increases in active ERK (pERK), involved in the transduction of dopamine signaling, in the nucleus accumbens and other brain regions (Ibba et al., 2009). Indeed, the acute upregulation of pERK seen after alcohol administration appears to be mediated, at least in part, through dopamine D1 receptors as D1 inhibition prevents dose-dependent increases in ERK phosphorylation induced by alcohol (Ibba et al., 2009). The effects of chronic alcohol exposure on ERK phosphorylation appear to depend on the type of exposure, with intermittent ethanol exposure having distinct effects from continuous ethanol exposure (Sanna, Simpson, Lutjens, & Koob, 2002). Importantly, while alcohol exposure results in a decrease in pERK expression immediately after exposure, pERK expression is elevated as compared to baseline in a number of brain regions, including the dorsal striatum and amygdala. Alcohol has been shown to block ERK-mediated LTP in the dorsomedial striatum, which is involved in goal-directed behavior (Xie et al., 2009). We hypothesize that dopamine exerts its influence on behavior in an ERK-dependent manner and that blocking ERK signaling downstream of dopamine receptors will prevent dopamine-mediated shifts in behavioral strategy, but the downstream mechanisms of dopamine's effects on habitual behavior have not yet been determined.

Endogenous opioids

Three opioid receptors types are known to be expressed in the brain: μ, δ and κ The ligands for these receptors are opioid peptides that are derived from three distinct precursors: β-endorphin is derived from pro-opiomelanocortin (POMC) and is the primary agonist for μ receptors. Prodynorphin is the precurosor for dynorphin and acts at κ-opioid receptors. Finally, proenkephalin is the precursor from which enkephalin is derived to act at δ receptors. The opioid and dopamine systems are known to interact in such a way that activation of μ- opioid receptors produces an increase of dopamine release (Spanagel, et al., 1990) and κ-opioid receptors results in a decrease of dopamine release from dopaminergic neurons (e.g. Herz, 1997). Naltrexone, a competitive antagonist at both kappa and mu opioid receptors has been shown to inhibit ethanol-induced dopamine release (Benjamin et al., 1993). Furthermore, mice deficient in μ-opioid receptors have decreased dopamine release in the nucleus accumbens.

A clear and significant role for opioid signaling in alcohol-related behaviors has been established, though how this differs from natural reward or may extend to habitual reward seeking is less clear. Although not well studied, it is likely that in addition to modulating dopamine in the nucleus accumbens, the opioid system can influence dopamine release in the brain regions involved in the development and expression of habitual behavior. Normal μ-opioid receptors are necessary for the maintenance of nucleus accumbens dopamine release, mediated through a disinhibition of VTA dopamine neurons, (Piepponen et al., 1999) and μ-opioid receptors are also required for the regulation of dopamine release in the dorsal striatum. Less is known about the role of κ-opioid receptor activity in these brain regions implicated in habit. Recent work from Wassum et al. (2009) demonstrated that when opioid receptors were blocked through the use of naloxone during instrumental training, mice developed a habit for food reinforcement more rapidly than would be expected. However, the authors saw no effect of the acute administration of naloxone, suggesting that normal opioid receptor activity is related to the acquisition of goal-directed behaviors, but not their performance. Notably, naloxone is not specific to opioid receptor subtypes and it is unclear which receptor subtypes are producing this effect. Additionally, these authors did not investigate habitual alcohol seeking and the potential interactions between alcohol and the endogenous opioid system. Indeed, through blockade of the opioid receptor system using naloxone, Wassum et al (2009) see accelerated habit formation and that compromise of this system produces inflexible habitual behavior for sucrose. It may be through alcohol's well-established interaction with the opioid system that it produces a similar shift toward rapid habit formation. Taken together, these data suggest that alcohol may act at the opioid system to increase dopamine release in the dorsolateral striatum, through enhanced μ-opioid receptor activity in the dorsolateral striatum and enhanced κ-opioid activity in the ventromedial striatum, to shift the system toward habitual responding.

Alcohol and opioids

The opioid system is sensitive to the effects of alcohol, and may modulate alcohol's effects on dopamine signaling. Mice lacking μ-opioid receptor activity, because of pharmacological antagonism or genetic knockout, have decreased alcohol-induced dopamine release in the nucleus accumbens (Job et al., 2007). In addition, chronic alcohol has been shown to cause an increase in enkephalin, the endogenous u-opioid receptor agonist, in the striatum of mice (Urayama, et al., 2006). It has also been shown that enkephalin binding decreases in the accumbens of rats after prolonged training, but there is not a change in other brain regions, including dorsal striatum (Turchan et al., 1999). This decrease in u-opioid receptor binding may translate to a decrease in dopamine release in the nucleus accumbens, while remaining unchanged in dorsal striatum. Additionally, it has been shown that alcohol can differentially influence the expression of the κ-opioid receptor, and its agonist, dynorphin, in multiple brain regions (e.g., Gulya et al., 1993; Oswald & Wand, 2004; Rosin et al., 1999) although these data are inconclusive about the direction of this change. However, it appears likely that enhanced activity of the dynorphin/ κ-opioid receptor system can decrease the risk of alcohol abuse: Mice that have lower levels of dynorphin in the nucleus accumbens, hippocampus, substantia nigra and striatum have a higher alcohol preference than a mouse strain with higher levels of dynorphin (Ploj, et al., 2000). Additionally, alcohol preferring rat strains have been shown to have lower levels of dynorphin (Guitart-Masip et al., 2006; Marinelli, et al., 2000; Nylander, et al., 1994). In addition, the use of opioid receptor antagonists such as naltrexone has been shown to decrease alcohol intake in a variety of species (Ulm, et al., 1995). In humans, naltrexone has been shown to reduce craving and alcohol selfadministration in alcoholic subjects (O'Malley, et al., 2002). Excitingly, recent work has shown that naltrexone has differential effects on alcohol seeking in animals that are trained on habit-promoting schedules. Specifically, it was observed that the ability of naltrexone to reduce cue-mediated ethanol seeking and ethanol consumption in rats was greater in rats that were trained on a ratio schedule, thought to maintain goal-directed behavior, as compared to rats trained on a habitpromoting interval schedule (Hay et al., 2013).

Adenosine and cannabinoids

The adenosine receptors form complexes with dopamine, cannabinoid and opioid receptors that can modulate glutamatergic signaling in the striatum(Ferré et al., 2007)(Ferré et al., 2007), which allows these neuromodulators to play a role in striatum-mediated behaviors. Using a striatum specific knockout of the adenosine A2A receptor, Yu et al. (2009) showed that mice lacking this receptor show goal-directed behavior at a time point at which controls are insensitive to outcome devaluation or contingency reversal, suggesting that striatal adenosine activity at the A2A receptor is critical for the formation of habitual behavior.

The cannabinoid system has also been implicated in the development of habitual behavior using a genetic mouse model (Hilario et al., 2007). In these experiments, pharmacological antagonism and genetic knockdown of the endocannabinoid receptor CB1 results in persistent goal-directed behavior, suggesting that this neuromodulator is also critical for the formation of stimulus-response habits (Hilario, et al., 2007). Importantly, neither of these reports extends the cannabinoid or adenosine findings to specific subregions of the dorsal striatum, which have been shown to have distinct effects of habitual and goal-directed behavior.

Alcohol, adenosine and endocannabinoids

An interaction between alcohol and the adenosine system has been reported repeatedly in the literatures. In particular, a role of the adenosine A2A receptor in mediating alcoholism-related behaviors has been identified, including alcohol sensitivity (Naassila, et al., 2002), withdrawal (El Yacoubi et al., 2001), operant self-administration (Arolfo, et al., 2004) and reinstatement (Adams, et al., 2008).

Given the interactions between A2A receptors and the glutamatergic system in the striatum (Ciruela et al., 2005; Ferré et al., 2007), and the established role for striatal A2A in the development of stimulus-response habits (Yu et al., 2009), it is likely that alcohol's interactions with this system may impact the rapid development of habitual alcohol-seeking.

Similarly, the endocannabinoid system has been implicated repeatedly in alcoholism-related behaviors (for review, Erdozain & Callado, 2011). The effects of alcohol on dopamine release appear to be mediated by CB1 signaling (e.g., Cohen, et al., 2002; Hungund, et al., 2003), and chronic alcohol exposure produces lasting effects on the cannabinoid system, resulting in downregulation of CB1 receptor mRNA in a number of brain regions, including the striatum (Ortiz, et al., 2004). Additional work has shown that intermittent alcohol consumption specifically results in decreased endocannabinoid signaling specifically in the dorsolateral striatum (Adermark, et al., 2011), resulting in aberrant neuroplasticity. Further, pharmacological manipulation of the CB1 system impacts alcohol selfadministration in a bidirectional manner: antagonism of CB1 results in decreased alcohol self-administration either systemically or when infused into the prefrontal cortex, while increased signaling through blockade of CB1 degradation results in decreased alcohol consumption (Hansson et al., 2007).

5. Sex differences in habit

Epidemiological studies report higher rates of alcohol dependence and abuse in men compared to women, and being male is consistently identified as a risk factor for alcohol abuse (e.g., Kalaydjian et al., 2009). Men also develop more alcoholrelated problems regardless of age or socioeconomic status, suggesting that biological factors are responsible for this disparity (Lynch, et al., 2002). Understanding sex differences in maladaptive behaviors, such as inappropriate habitual responding, is critical for prevention and treatment of alcohol-related disorders. Traditionally, the organizational (permanent) and activational (nonpermanent) influence of gonadal hormones has been thought to underlie sex differences in behavior (Arnold & Gorski, 1984), but development of the four core genotype (FCG) mouse model has enabled exploration of the distinct influence of sex chromosome complement and gonadal phenotype (Arnold, 2009; Arnold & Chen, 2009; De Vries et al., 2002). Although it is becoming clear that sex chromosome complement plays a critical role in determining neuronal activity and connectivity, the functional outcomes of such differences are not yet well understood. Using the FCG mouse model, we have shown that sex differences in propensity to habit formation are different for alcohol and food reinforcers. In particular, we have seen that though for both natural and alcohol reinforcers chromosomal sex determines habit formation: chromosomal males show accelerated formation of alcohol seeking habits (Barker et al., 2010) while the opposite is true for sucrose seeking behavior (Quinn, et al., 2007).

Though the exact neurobiological mechanisms that underlie the transition from goal-directed to habitual responding are unclear, it is likely that changes at dopaminergic and glutamatergic synapses ultimately produce this shift away from behavioral flexibility. Interestingly, human studies have identified sex differences in the role of dopamine in the expression of goal-directed behavior, suggesting that dopamine depletion biases women toward habitual responding without impacting response strategy in men (de Wit et al., 2012). As discussed above, alcohol can interact with the dopamine and opioid signaling systems in such a way that may promote the formation of stimulus–response habits. Interestingly, genetic sexdependent differences have been observed in the opioid system. Using the FCG model, Chen and colleagues (2009) investigated levels of prodynorphin, the precursor to the endogenous κ- opioid ligand, in the striatum. Their results indicated that XX mice had higher prodynorphin expression than mice with a single X chromosome. In previous work, the same lab demonstrated that sex chromosome complement mediates functional differences in the κ opioid system (Gioiosa, et al., 2008), but a role for the κ system in mediating sex differences in habit has not been explicitly investigated.

In addition to sex differences in the opioid system, there significant evidence for an interaction between sex and GABA signaling in alcohol use disorders (Alele & Devaud, 2005; Ceylan-Isik, McBride, & Ren, 2010; Finn, Beckley, Kaufman, & Ford, 2010).In rodent models, there are both sex differences in expression of GABA receptors, as well as in neuroactive steroids that act to alter GABAergic neurotransmission. Sex determines regional differences in both innate and alcohol-induced differences in GABA_A subunit expression. In particular, male rats show decreases in alpha1 subunit expression in the cortex relative to female rats at baseline while differences in the hypothalamus emerge after alcohol exposure (Devaud, Fritschy, & Morrow, 1998; Devaud, Matthews, & Morrow, 1999). In addition to differences in receptor expression, sex differences in allopregnanolone mediation of GABA signaling likely contribute to differential effects of ethanol. Allopregnanolone, a progesterone metabolite, acts to potentiate GABA signaling at GABA_A receptors. Females have markedly higher level of allopregnanolone as compared to males, which fluctuate cyclically. In addition to baseline sex differences in allopregnanolone levels, ethanol-induced allopregnanolone release appears to be distinct between sexes with males showing greater increases in allopregnanolone after ethanol injection or consumption (Finn et al., 2004). These alterations in neuroactive steroid expression may which could potentiate signaling at GABA_A receptors suggests an additional mechanism through which sex differences in ethanol seeking habits may arise.

Sex differences provide an interesting method for investigation of the role genetic traits and hormonal states in the development of addictive behavior, and greater knowledge regarding the interaction of sex chromosome complement and gonadal hormone status with multiple neurobiological systems may provide extensive insight into our understanding of the etiology of addiction.

6. Stress effects on habit

Stress has repeatedly been shown to dysregulate prefrontal control of behavior in both humans and animal models (c.f., Arnsten, 2011; McEwen, 2004). Stress has been shown to drive alterations through limbic corticostriatal circuitry, having widespread effects on behavior and cognition that may drive addictive behavior (for review, see Schwabe, Dickinson, & Wolf, 2011). It has recently been shown that stress can drive the expression of habitual behavior in humans (Schwabe & Wolf, 2009), and more specifically that this was mediated through adrenergic and glucocorticoid systems. Indeed, activation of the adrenergic and glucocorticoid systems could drive the expression of habitual behavior, while antagonism of these circuits could prevent this effect (Schwabe, Höffken, Tegenthoff, & Wolf, 2011; Schwabe, Tegenthoff, Höffken, & Wolf, 2010, 2012). Chronic stress in humans can also bias habit-like response strategies which were accompanied by enhanced activity in sensorimotor circuits as well as volumetric changes (Soares et al., 2012). Importantly for clinical implications, these stress-induced alterations in response strategy can be rescued after a removal of stress. In animal models, a history of chronic corticosterone exposure or stress facilitated the development of habitual behavior(Dias-Ferreira et al., 2009; Gourley et al., 2012). This facilitation of habit formation appears to be in part mediated by stress-induced alterations in BDNF signaling, as well as alterations in glucocorticoid receptor activity, and subsequent spine remodeling (Gourley, Swanson, & Koleske, 2013; Gourley et al., 2012; Swanson, Shapiro, Whyte, & Gourley, 2013).

While it appears clear that both acute and chronic stress exposure facilitate a general reliance on habitual response strategies, to our knowledge the direct effects of stress on alcohol habit formation are as yet unstudied. However, there is a well-established link between stress and alcohol consumption and relapse. Importantly, alcohol users report high rates of relapse under conditions of stress, which may result from a resumption of reliance on stimulus-mediated behaviors. Stress across the lifespan has been shown to increase the likelihood of alcohol use disorders (Enoch, 2011). In animal models, chronic activation of stress circuitry can promote the development of behaviors that are thought to phenotypes that are associated with early onset of drinking in humans (Hamilton, Ansell, Reynolds, Potenza, & Sinha, 2013; Torregrossa, Xie, & Taylor, 2012). In addition to driving behavioral inflexibility, prevailing theories suggest that alcohol use disorders represent reward surfeit disorders (Comings & Blum, 2000; Koob, 2013) in which an impaired ‘reward system’ promotes the development of addiction. Chronic stress has been shown to disrupt normal responses to reinforcement that may be consistent with this reward deficit model of addictive behavior (Rygula et al., 2005) potentially through effects on the dopamine system (Butts & Phillips, 2013; Del Arco & Mora, 2009; Del Arco, Segovia, Garrido, de Blas, & Mora, 2007; Feenstra, 2000). We believe that an important yet unstudied question is how an attenuated valuation of reward may drive the development of habitual, outcome insensitive behaviors, potentially driving rapid development of habitual behaviors. Critically, cycles of alcohol consumption and abstinence which are seen across alcohol use disorders drive activation of stress systems (for review, Koob & Le Moal, 2008; Koob, 2013), suggesting an additional mechanism by which alcohol may act to facilitate the expression and development of inflexible habitual behaviors.

7. Conclusion

The exact neurobiology of habitual alcohol seeking is far from being understood. Through work primarily focusing on the neurobiological substrates of habitual responding for food and independent work investigating the effects of alcohol consumption on signaling systems that are involved in habit, a number of targets for future investigation into the loss of behavioral flexibility have been identified. As we gain more knowledge about the mechanisms of alcohol seeking habits, new therapeutic strategies can be developed that help restore goal-directed behavior of alcoholics or prevent the development of habitual alcohol-seeking in at-risk individuals. Importantly, as more tools become available to assess the role of sex in alcoholism-related behaviors, sex differences in behavior and treatment strategy must also be considered.

Figure 1. Interactions between cognitive-behavioral processes that drive addiction.

A number of models have proposed that the development of addiction, including alcohol use disorders, results from a transition from controlled behavior to uncontrolled drug seeking behavior. Innate risk for addictive behavior has been shown to be related to differences in impulsivity and cue-reactivity that exist prior to any drug exposure. These differences can initiate entry into the spiral toward addiction. Across alcohol exposure, dysregulation in limbic corticostriatal circuitry is expected to drive drug seeking, which acts to exacerbate deficits in behavioral control, ultimately facilitating the disease state. In addition to impulsivity and habitual reward seeking, addiction is characterized by compulsive behaviors. Compulsivity can be defined as either behaviors that persist despite or because of the negative consequences of drug taking. In other words, compulsive behaviors continue despite negative results of drug-taking or those negative results create an aversive state that can be relieved by ongoing drug-taking. Importantly, habit models a behavior that is more similar to the former in which drug-seeking is elicited by environmental stimuli rather than a desire to seek positive or negative reinforcement. Alternatively, the aversive state created by withdrawal may come to serve as an interoceptive stimulus, driving habitual reward seeking behavior. To our knowledge, the precise relationship between these behaviors has not been elucidated (though see Everitt, 2014; Koob & Volkow, 2010).

Fig 2. Neuroanatomical substrates of inflexible behavior.

Blue regions represent those regions known to be required for the expression of goal-directed reward seeking. Red regions are required for the development or expression of stimulus-response habits. Midbrain DA signaling has been shown to play a role in both forms of behavior. The role for the nucleus accumbens in reward seeking is more complex. One possibility is that through the ‘ascending loops’ of projections from midbrain dopaminergic regions to striatal subregions, glutamate or DA signaling in the NAc shell can ultimately influence activity in dorsal striatum mediation of behavior. A role for the BLA has not yet been explicitly demonstrated, but is likely given its established role in stimulus-outcome learning and the encoding of outcome value. Only the dorsolateral striatum has been explicitly demonstrated to have the same role in habitual alcohol seeking as in habitual food seeking (Corbit et al., 2012).

Alcohol habits: modeling the transition from casual drinking to addiction. Highlights.

• Habitual ethanol seeking models uncontrolled drinking in human alcoholics

• The transition to habitual ethanol seeking is more rapid than for natural rewards

• Understanding ethanol's impact on habit circuitry will inform treatment

• Sex and stress may mediate the development of ethanol seeking habits