Habit Formation: Implications for Alcoholism Research

Abstract

Characteristics of individuals with severe alcohol use disorders include heightened cue sensitivity, compulsive seeking, craving, and continued alcohol use in the face of negative consequences. Animal models are useful for understanding behavioral and neurological mechanisms underlying problematic alcohol use. Seeking of operant reinforcers including alcohol is processed by two mechanisms, commonly referred to as “goal-directed” (action-outcome) and “habitual” (stimulus-response). As substance use disorders are characterized by continued use regardless of unfavorable outcomes, it is plausible that drug use causes an unnatural disruption of these mechanisms. We present a critical analysis of literature pertaining to behavioral neuroscience alcoholism research involving habit formation.

Traditionally, when operant behavior is unaffected by a loss of subjective value of a reinforcer (devaluation), the behavior is considered habitual. Acquisition of instrumental behavior requires corticostriatal mechanisms that depend heavily on the prefrontal cortex and ventral striatum, whereas practiced behavior is more predominantly controlled by the dorsal striatum. Dopaminergic signaling is necessary for the neurological adaptations involved in stimulus-response action, and drugs of abuse appear to facilitate habitual behavior through high levels of dopamine release. Evidence suggests that the use of alcohol as a reinforcer expedites habit formation, and that a history of alcohol use produces alterations in striatal morphology, aids habit learning for non-psychoactive reinforcers, and promotes alcohol drinking despite aversive adulterants.

In this review, we suggest directions for future alcoholism research that seeks to measure action made despite a devalued outcome, including procedural modifications and genotypic, pharmacological, or neurological manipulations. Most alcoholism models currently in use fail to reach substantial blood ethanol concentrations, a shortcoming that may be alleviated through the use of high-drinking rodent lines. Additionally, satiety, one common mechanism of devaluing reinforcers, is not recommended for alcohol research because the psychoactive effects of alcohol depress response rates, mimicking devaluation effects. Overall, further research of habit formation and potentially related perseverative behaviors could be invaluable in discovering genetic variance, traits that correlate with persistent alcohol seeking, implicated neural structures and processes of alcohol use, and eventually novel pharmacological treatment for alcoholism.

Introduction

Alcoholism is a chronic disease of uncontrollable alcohol use. Alcoholics usually oscillate between abstinence and relapsed heavy use; accordingly, abstinence “survival” curves demonstrate that the percentage of successfully treated alcoholics in a given sample decreases over time (Kirshenbaum, Olsen, & Bickel, 2009). Prolonged use of alcohol is correlated with greater resistance to treatment, including therapy and community-based interventions, prompting recent experimentation with procedures using medication and/or deep-brain stimulation (e.g., Müller et al., 2009; Pastor, Jones, & Currie, 2012). Consistent with this idea, naltrexone, an opiate antagonist used to treat some alcohol use disorders (AUDs), is relatively ineffective in chronic, severe alcoholics (Krystal et al., 2001), a finding suggesting that prolonged use of alcohol may alter the neural substrates affected by its use. However, the mechanisms by which treatment-resistant alcoholism differs from less persistent AUDs are unclear. Elucidation of these mechanisms could lead to treatments that are more successful.

Substance use is linked to craving and positive expectancies about outcomes, associating it with intentional behavior (Robinson & Berridge, 2003). However, the inability to quit despite an intention to do so is a criterion for clinical diagnosis of substance dependence (American Psychiatric Association, 2013), implicating use as a process that persists in spite of a desire for other outcomes. These conflicting but not mutually exclusive notions suggest activation of two processes, called goal-directed and habitual behavior (Hogarth & Chase, 2011). Goal-directed behavior is defined as an action (e.g., an instrumental response) mediated by its association with a desirable outcome (Dickinson, 1985). Habitual behavior, in contrast, is defined by responding with equal strength to an associated desired or aversive outcome alike, indicating indifference to the present value of reinforcement. Habitual responses are quickly elicited by a stimulus linked to a lengthy reinforcement history, also known as a “trigger” or “cue” in the environment (Schulte et al., 2012). It is suggested that drug exposure subverts natural habit learning toward an automatized drug-driven condition, creating ingrained stimulus-response habits that are resistant to attempts at behavioral change (Belin, Jonkman, Dickinson, Robbins, & Everitt, 2009; Everitt & Robbins, 2005; Robinson & Berridge, 2003).

The susceptibility of a dependent individual to a cue that instills craving for and seeking of a substance that causes adverse consequences is phenomenologically similar to a stimulus provoking responding for a devalued outcome in an animal model. Habitual behavior in animal models is commonly measured using devalued outcomes through conditioned aversion or specific satiety, as described in detail below. Recently, there has been a great interest in habit formation using alcohol as a reinforcer, or facilitation of habitual responding for another reinforcer during concurrent alcohol consumption. Both of these are exciting prospects, given recent findings regarding the analogous behavior of human alcoholics, particularly cue sensitivity predicting relapse susceptibility and lack of planning and/or goal-directed perseveration (Coskunpinar, Dir, & Cyders, 2013; Schulte et al., 2012). Traditional habit formation and other manifestations of perseverative behavior may be critical mechanisms for the understanding and treatment of chronic drug abuse. An in-depth examination of how these concepts relate to alcoholism may prove valuable to the goal of developing improved interventions. The intention of this manuscript is to review human and animal literature in support of this pursuit.

Behavioral Classifications Using Conventional Reinforcement

The principle that a trained animal will perform an action while in a motivated state to achieve a desired outcome, such as a food or fluid reinforcer, has been described for over 100 years (Thorndike, 1911). Seminal research investigating the effects of post-operant conditioning manipulations of subjective value of appetitive reinforcers was conducted using rodents in the early 1980s in order to better understand the associative structure underlying instrumental learning (Adams, 1980). Pairing a gustatory reinforcer with lithium chloride (LiCl)-induced illness creates an aversion to that specific reinforcer, as demonstrated both by conditioned aversion to the taste of the reward and avoidance of the location where the LiCl was presented (Adams, 1982; Chen & Amsel, 1980). Alternatively, unlimited access to the reinforcer is used to instill satiety prior to operant testing (e.g., Coutureau & Killcross, 2003). A third, less common, process of manipulating the value of a reinforcer is through classically conditioned pairing with high-speed rotation, inducing mechanical rather than chemical illness (Holland & Rescorla, 1975). These procedures are commonly referred to as reinforcer devaluation. In the operant setting, motivation to respond would logically be greatly reduced once that reinforcer is devalued. However, Adams (1980) first reported that LiCl pairing had no effect on responding during extinction conditions (i.e., without reinforcer delivery, a measure of isolated motivation). That is, when animals were not directly exposed to the reinforcer during instrumental behavior, response rates for a devalued expected outcome were the same as for one whose appetitive qualities remained intact. These experimental data are consistent with behavior that corresponds to a mechanistic stimulus-response model, in which an action is either an innate or an acquired habit that is triggered by a particular stimulus (Dickinson, 1985).

In contrast, behaviors that are sensitive to reinforcer devaluation are goal-directed, manifested according to a teleological (i.e., purposeful, or outcome-driven) model of animal behavior (Dickinson, 1985). Operant behaviors are initially more goal-directed in nature, but they tend to become primarily driven by habit following extended practice. This shift is regarded as a change in the dominant force behind behavior from response-outcome associations to stimulus-response associations (Balleine & Dickinson, 1998; Rescorla, 1994). Unlike habitual behavior, goal-directed behavior is dependent upon specific motivation for an outcome; for instance, if given a choice of two actions under extinction conditions, animals preferentially respond for a non-satiated reinforcer as opposed to a satiated one because of a higher motivational state (Balleine & Dickinson, 1998). Goal-directed behavior is also responsive to a variety of contingency manipulations. Extinction generally causes responding to quickly cease (Bouton, Winterbauer, & Todd, 2012; Milad, Rauch, Pitman, & Quirk, 2006), as do contingency degradation and omission, a condition in which a previously reinforced response is now only reinforced when it is withheld (Dickinson, Squire, Varga, & Smith, 1998). Pavlovian-to-instrumental transfer (PIT), the capacity of a Pavlovian stimulus that predicts a reinforcer to elicit or increase instrumental responses for the same reinforcer, also affects goal-directed behavior (Crombag, Galarce, & Holland, 2008; see Holmes, Marchand, & Coutureau, 2010 for a comprehensive review of PIT).

Actions that are goal-directed and habitual alike are self-initiated, motivated behaviors that are made with some knowledge of a distinct outcome, e.g., an operant reinforcer (Dickinson, 1985). However, in the case of habitual actions, the present value of a reinforcer and its contingency with a response do not control the nature or strength of the action. Thus, behaviors that fully persist in spite of post-training instrumental contingency manipulations are considered habitual (Adams, 1982; Dickinson, 1985). The defining feature of initiation of habitual responding is exposure to a conditioned stimulus associated with the response through prior contiguity and a prolonged reinforcement history. Reinforcer devaluation, then, does not abolish performance during subsequent extinction testing (Adams, 1982), relative to a non-devalued control also subjected to extinction conditions. Furthermore, contingency degradation is ineffective at reducing the expression of behavior (Dickinson et al., 1998). However, habitual behavior may actually be more sensitive to PIT than goal-directed behavior. Holland (2004) demonstrated that as overtrained rats became less sensitive to reinforcer devaluation, they showed a corresponding increase in the influence of outcome-associated stimuli on response rates. This finding supports the notion that outcome representation is involved even in trained habitual behavior, and suggests that even when a reward is devalued, Pavlovian cues associated with that reward maintain their motivational properties.

Differing training procedures can profoundly affect the development of habitual behavior. Dickinson, Nicholas, & Adams (1983) tested rats for their susceptibility to reinforcer devaluation following equivalent amounts of training with ratio vs. interval schedules. Ratio schedules are defined by a close link between a particular number of responses and the reinforcer (e.g., a fixed ratio, or FR, 3 schedule provides a reward every 3 responses; variable ratio, or VR, 10 is a reward every 10 responses on average), while interval schedules are reinforced based on responses that occur after a certain time interval has elapsed (e.g., a fixed interval, or FI, 60-sec schedule reinforces the first response occurring after a minute without reinforcement; variable interval, or VI, 30 schedule represents reward availability after varying times averaged on 30 seconds). They discovered that ratio, but not interval, schedules left behavior susceptible to reinforcer devaluation. Under interval schedules, animals may be less proficient at encoding the relationship between performance variation and outcome, developing a motor habit through repeated responses (Dickinson, 1985). Additionally, the method used to devalue a reinforcer may influence detection of a devaluation effect, or lack thereof. This much is suggested by a 1979 study by Holland & Straub demonstrating that high-speed rotation devaluation showed greater effects on activity-related behaviors in response to a conditioned stimulus, while LiCl devaluation showed a greater reduction of magazine entry and more strongly reduced reinforcer consumption. Theoretically, varying components of the sensory and motivational processes, and associated underlying neurobiological adaptations that are linked with an outcome, are affected by each method. Adaptations may leave the sensitivity of motor responding to devaluation intact in some cases but not in others. We suggest that future research of habitual behavior should consider the implications of the methodology used for devaluation.

Neurological Functioning in Learning and Theories of Addiction

Motivated behavior, whether under greater teleological or mechanistic control, involves interaction of the cortex, striatum, and midbrain. The medial prefrontal cortex (mPFC) is necessary for the acquisition of goal-directed behavior (Ostlund & Balleine, 2005). Simultaneously, the premotor cortex (analogous to the presupplementary motor area in primates) is critical in the expression of goal-directed behavior (Gremel & Costa, 2013b)). The ventral striatum, including the nucleus accumbens (NAcc), receives excitatory input from the PFC and is important in goal-directed learning, theoretically due to its role in establishing motivation for valued, salient outcomes (Brog, Salyapongse, Deutch, & Zahm, 1993; Everitt & Robbins, 2005; Wendler et al., 2013). The prelimbic region of the prefrontal cortex (PFC) and its projections to the dorsomedial striatum in rodents (DMS; analogous to the anterior caudate) are crucial for goal-directed learning (Nauta, 1989; Yin, Ostlund, Knowlton, & Balleine, 2005). In contrast, if the dorsolateral striatum (DLS; analogous to the posterior putamen) is rendered ineffective, instrumental responding is sensitive to devalued outcomes under all conditions, implicating this structure in stimulus-response behavior (Yin, Knowlton, & Balleine, 2004). All striatal regions are the target of afferents from neurotransmitter production bodies in the midbrain (Nicola, Surmeier, & Malenka, 2000). Recent research suggests that the orbitofrontal cortex (OFC) is engaged during shifts from goal-directed to habitual behavior, interacting with the striatum to promote appropriate action (Gremel & Costa, 2013a)). In summary, the NAcc controls conceptualization of salience and valuation, the DMS/caudate is involved in learning and maintenance of the relationship between actions and their outcomes, the DLS/putamen mediates stimulus-response habits, and all striatal regions interconnect with cortical and midbrain neural structures to integrate motivation, learning, and action (see Balleine & O'Doherty, 2010 for a more detailed review).

Accordingly, glutamatergic innervation from the cortex provides excitatory input to the striatum, and the activation of prefrontal cortical neurons is correlated with action-outcome performance (Histed, Pasupathy, & Miller, 2009; Smith, Bennett, Bolam, Parent, & Sadikot, 1994). Cholinergic interneurons in the striatum are implicated in reward and its prediction, noted by a pause in tonic firing in response to a stimulus (Aosaki et al., 1994). Importantly, dopaminergic inputs from the midbrain, including the ventral tegmental area (connectivity to the ventral striatum; the mesolimbic pathway) and substantia nigra (connectivity to the dorsal striatum; the nigrostriatal pathway), are fundamental in regulated control of movement and planning of action (Nicola et al., 2000). The function of the nigrostriatal dopaminergic pathway specifically is essential for stimulus-response habit formation. Striatal dopamine depletion is shown to block the learning of motor habits in rats, in agreement with an experiment that found deficits in habitual learning and action in humans with Parkinson’s disease (Faure, Haberland, Condé, & El Massioui, 2005; Knowlton, Mangels, & Squire, 1996). A_2A adenosine receptors and CB1 cannabinoid receptors are also implicated in habit formation through studies using knockout animals (Hilário, Clouse, Yin, & Costa, 2007; Yu, Gupta, Chen, & Yin, 2009). Both types of receptors are believed to reciprocally interact with dopamine receptors to modulate neurotransmission in the striatum (Ferré, Goldberg, Lluis, & Franco, 2009). For striatal physiology and pharmacology that is beyond the scope of this review, see Kreitzer, 2009.

Theoretical models are characterized which relate the development of drug addiction to instrumental learning and the processes of neurological change within the striatum implicated in each (see Belin et al., 2009 and Everitt & Robbins, 2005). Initial drug reward depends upon dopaminergic functionality in the NAcc “shell”, whereas conditioning shifts seeking and reinforcement processes toward the NAcc “core” (Belin & Everitt, 2008). The dorsal striatum is interconnected to this region in a dorsally progressive cascade (Alexander & Crutcher, 1990). Learning via this pathway is important for the survival of organisms because NAcc circuitry is necessary for processing natural rewards and behaviors, and the development of advantageous stimulus-response associations improves behavioral efficiency (Kelley & Berridge, 2002). However, addiction mechanisms are theorized to be stronger than natural stimulus-response motor learning, because drugs of abuse create an abnormally high dopaminergic signal in the ventral striatum and thus exert a strong influence on the dorsal striatum (Belin et al., 2009; Everitt & Robbins, 2005). Accordingly, there is convincing evidence that habit formation (i.e., insensitivity to devaluation) is accelerated when pharmacologically active compounds are used as reinforcers, suggesting increased dorsal striatal influence on action. Most such studies have used cocaine (e.g., Belin & Everitt, 2008; Ito, Dalley, Robbins, & Everitt, 2002; Miles, Everitt, & Dickinson, 2003; Vanderschuren, Di Ciano, & Everitt, 2005; Zapata, Minney, & Shippenberg, 2010).

Alcohol and Habit Learning in Animals

Until recently, however, little literature in the domain of habit learning using alcohol had been published. The classic manuscript examining the efficacy of alcohol in eliciting classically defined habitual behavior describes an experiment conducted using Long-Evans rats (Dickinson, Wood, & Smith, 2002). In this experiment, Dickinson et al. (2002) compared habit formation using an ethanol solution following a sucrose-fading procedure (e.g., Samson, 1986; Samson et al., 2004) to that formed concurrently using food pellets as reinforcers. Training consisted of sessions in which food pellets and sucrose/ethanol solution were presented equally between two levers and equal reinforcers of each kind per session were delivered. Sucrose concentration was gradually decreased while ethanol concentration was gradually increased. The final phase of training featured consistent delivery of ethanol solution or pellets by each lever. During the extinction test, rats demonstrated fewer lever presses on the pellet lever if pellets were devalued using LiCl than if ethanol was devalued. In contrast, pressing on the ethanol solution lever was unaffected by whether pellets or ethanol were devalued, though a suppression in responding in comparison to the control group was seen. The results suggest that rats were responding for ethanol solution according to behavior consistent with a “habit”, as altering the value of ethanol or food produced the same behavior. However, inspection of the methodology and data reveals several caveats.

First, the use of a sucrose-fading procedure is questionable. Though this procedure is notable for its face validity, as human drinkers tend to start with sweet mixed drinks or beer before progressing to stronger liquors (Simms, Bito-Onon, Chatterjee, & Bartlett, 2010), its use in habit formation is problematic because the initial stimulus-response connection occurs before alcohol is introduced, and is not based entirely upon an association with alcohol. Additionally, animals demonstrate a non-specific devaluation; responding for both the alcohol and food pellet reward is decreased compared to controls. Our main concern is that animals received no more than 0.3 mL of 10% ethanol during self-administration sessions, or about 0.09 g/kg. This dose would not be expected to result in meaningful blood alcohol levels; therefore, the rats in this study never achieved intoxication. It follows that it would be hard to defend the idea that these rats were motivated to respond by the desire to obtain the pharmacological effects of ethanol or were ever subject to any of its presumed neurological adaptations. A final concern is that a floor effect of responding may be masking differences between the devalued ethanol and pellet groups, as each group only showed about two lever presses per minute. Although this study is widely cited (105 times as of December 2013) as evidence for the idea that alcohol promotes habit learning more effectively than food pellets, we contend that the evidence for this reinforcer difference is not particularly strong.

Samson et al. (2004) utilized an appetitive-consummatory model that allows for separation of motivational behavior from intake behavior to test ethanol devaluation in Long-Evans rats. Under this paradigm, a retractable sipper tube grants unlimited access to the reinforcer for a particular duration (usually 20 min) following the required number of lever presses. Reinforcer devaluation in these experiments was performed by pairing a 1 g/kg oral gavage of ethanol with LiCl injection, allowing for alteration of the value of ethanol intoxication separate from the hedonic properties of its taste. Unpaired groups received the same LiCl and gavage history, with the two events separated by 24 h rather than 10 min as in the Paired group. Suppression of responding in extinction was seen following devaluation in the Paired groups, indicating that responding was not habitual in these rats. Interestingly, alcohol intake during a final operant session in which one lever-press conferred alcohol access did not differ between Paired and Unpaired groups, suggesting that motivation to obtain ethanol was altered, but motivation to consume it was unaffected by devaluation. Blood samples were taken on the final day of training; blood ethanol concentrations (BECs) of 53 mg/dL in the paired group and 56 mg/dL in the unpaired group represent a definite strength of this study over other experiments described in this section. Specifically, although rats encountered the intoxicating effect of alcohol during self-administration, they did not develop insensitivity to devaluation. The cause of this result remains unclear but may have to do with the paradigm and/or training length, and this study is often cited as a counter-example to the Dickinson et al. (2002) paper.

A unique study noted that alcohol-based stimulus-response action might be stronger in males than females (Barker, Torregrossa, Arnold, & Taylor, 2010). This work utilized a previously devised procedure involving deletion of the maleness-conferring SRY gene from the Y chromosome in C57BL/6 mice and its insertion onto an autosome, allowing separation of chromosomal and gonadal sex in the progeny of these animals and normal females (Quinn, Hitchcott, Umeda, Arnold, & Taylor, 2007). Training these “four core genotype” (FCG) mice using a somewhat-limited training schedule that is believed to produce habitual responding in “sensitive” populations led to a lack of a devaluation effect among groups responding for 10% ethanol solution in chromosomal males but not in chromosomal females, independent of gonadal status; these results were the opposite of those seen using food pellets (Barker et al., 2010; Quinn et al., 2007). However, higher rates of responding were seen in chromosomal males, plausibly expediting stimulus-response habit formation in these animals regardless of alcohol intake or reinforcer type. Future research, therefore, should establish parameters to equalize responding as much as possible while allowing for group differences in other variables. Importantly, though, animals consumed pharmacologically relevant quantities of alcohol during 45-min sessions, peaking at 2.5 g/kg during VI-30 training and decreasing to 1.5 g/kg during VI-60 training, although blood alcohol levels were not assessed. Overall, the two experiments using FCG mice suggest that in some populations, alcohol-seeking behaviors lose dominant control by goal-directed mechanisms faster than food-seeking behaviors, whereas the converse applies to other populations.

Several studies have very recently been published that seek to examine the effects of alcohol on habitual behavior. Mangieri, Cofresí, & Gonzales (2012) provide evidence that alcohol expedites learning based upon stimulus-response action during responding for sucrose. Specifically, they showed that LiCl pairings successfully devalued responding for 10% sucrose (10S) reinforcement after limited training on a VI schedule, but not a combined 10% sucrose and 10% ethanol (10S-10E) solution. A concurrently tested VR schedule group, responding for 10S-10E solution, revealed a devaluation effect. All three schedules, following extended training in separate animals, produced a non-significant devaluation effect on responding. These rats consumed, on average, 0.5 g/kg during a 20-min operant session, corresponding to pharmacologically relevant, though not “binge-like” blood ethanol levels. A comparison group involving sucrose reinforcement under a VR schedule would have added to the thoroughness of these experiments. Additionally, rats were given two (VR) or three (VI) operant sessions prior to the introduction of ethanol; therefore, initial experience with the stimulus occurred prior to its association with an ethanol reward, as mentioned as a caveat of Dickinson et al. (2002). Importantly, however, this work provides the first published evidence that alcohol accelerates an insensitivity to devaluation compared to a reinforcer that lacks alcohol but is otherwise identical, in agreement with findings using cocaine (Mangieri et al., 2012; Miles et al., 2003).

An additional study investigated whether a principle pharmacotherapy for alcoholism, naltrexone, exhibits different effectiveness for stimulus-response vs. outcome-based learning, drawing on previous findings that naltrexone is less effective in chronic alcoholics (Krystal et al., 2001). These investigators used either an FR-5 schedule of reinforcement to facilitate outcome-based action, or a VI-30 schedule to facilitate stimulus-response action (Hay, Jennings, Zitzman, Hodge, & Robinson, 2013). Additionally, alcohol or sucrose reinforcers were tested using each schedule. The authors assert that naltrexone effectively decreased seeking for alcohol but not sucrose; however, observed ceiling effects in fluid deliveries and lever presses under all conditions except for lever presses in VI-30 animals make this claim difficult to support. Furthermore, when tested in extinction, which nullified the ceiling effect, a decrement in seeking was seen in rats trained using an ethanol or sucrose reward under all training schedules. Devaluation testing revealed a greater relative decrease in responding in VI-30 groups than FR-5 groups; however, satiety was the devaluation mechanism, possibly producing general rate-depressant effects due to acute alcohol intoxication in alcohol-drinking rats. Acute ethanol causes dose-dependent decreases in operant responding in rats and mice (Elmer & George, 1994; George, Ritz, & Meisch, 1990). During self-administration, rats had BECs that were arguably under pharmacologically relevant levels (30 mg/dL or less), another caveat to this study. Extinction and subsequent reinstatement of reinforcer delivery was also tested; naltrexone suppressed reinstatement for both alcohol and sucrose. Notably, VI-30 sucrose animals did not demonstrate this effect as strongly as other groups, perhaps suggesting a disparity between alcohol- and sucrose-seeking processes under this reinforcement schedule. However, throughout this manuscript, there is a lack of evidence that responding for alcohol or sucrose differs in sensitivity to reinforcer devaluation, unlike Mangieri et al. (2012). The use of a high-drinking line of rodents might prove useful in generating the BECs necessary to make a definitive conclusion regarding the involvement of the pharmacological effects of alcohol in a study such as this.

Corbit, Nie, & Janak (2012) presented an interesting set of experiments testing both the time course and dorsal striatal involvement of instrumental behavior, again using Long-Evans rats. These experiments did not utilize sucrose fading as previously described, but instead involved a home-cage alcohol acclimation procedure. The paradigm featured 2 weeks of free choice 10% ethanol drinking followed by 2 weeks of 1-h access to ethanol only at the time when operant training was to be performed. Ethanol devaluation, following the various lengths of training (as will be discussed), was performed through 45 min of free-choice alcohol access while non-devalued animals were given sucrose as a control. The authors required 3 mL of ethanol solution to be consumed by an animal during pre-feeding for its inclusion in data analysis, and the average intake was about 1 g/kg. As mentioned with regard to Hay et al. (2013), satiety is not recommended for alcohol devaluation because the pharmacological effects of ethanol generally lead to an effect of decreased responding due to motor suppression. However, this procedure was the only viable option for the within-subjects experiments that were among those used to probe the time course of devaluation insensitivity. Furthermore, the devaluation effects proved effective and consistent with prior data collected using aversion conditioning.

The first within-subjects experiment demonstrated a devaluation effect, and therefore goal-directed behavior, at 1 and 2 weeks of operant training. A transition to “habitual behavior” (i.e., a non-significant devaluation effect) was seen at 4 weeks, and was seen more strongly (i.e., responding between groups was nearly equal) at 8 weeks. A between-subjects second experiment using another cohort of rats, which could have, but did not, use a taste aversion method for devaluing the ethanol, confirmed the 2- and 8-week time points as facilitating goal-directed and habitual behavior, respectively. Interestingly, a third cohort of rats showed a devaluation effect after 8 weeks using a sucrose reinforcer at baseline, but did not show the effect if the animals were concurrently drinking alcohol in the home cage during the 8 weeks of operant training. The fourth experiment utilizing DMS or DLS inactivation in a fourth cohort of rats under the 2- and 8-week drinking paradigm supported previous results using limited or extended training and inactivation of the two sub-regions of the dorsal striatum (Corbit et al., 2012; Yin et al., 2004, 2005). An important caveat to this work is that the 0.5 g/kg consumed during a 1-h operant session would not be expected to generate pharmacologically relevant BECs, which are unfortunately not reported in this paper. An additional concern is that prolonged experience with even these very low intake levels could potentially lead to tolerance to the depressant effects of alcohol in long-term drinkers. When the highest dose of alcohol is encountered because of the devaluation procedure, its sedative effects could suppress extinction responding, appearing to cause devaluation that is reinforcer-specific (i.e., sucrose satiation would not have the same pharmacological effect). We recommend LiCl devaluation to be used in conjunction with alcohol as a reinforcer. Nonetheless, the specificity of the lesion effects as predicted by prior studies on habit formation is convincing. The most noteworthy novel finding is that concurrent home-cage alcohol exposure facilitates habitual responding for sucrose, which is analogous to an experiment demonstrating that amphetamine sensitization expedites habit formation for a food reward (Nelson & Killcross, 2006).

Preliminary insight into potential neural mechanisms of the aforementioned effect is provided by a recent study using the C57Bl/6J mouse model (DePoy et al., 2013). Chronic intermittent ethanol (CIE) exposure, using vapor chambers to produce BECs of around 175 mg/dL, significantly increased free-choice ethanol consumption. Neuronal adaptations including dendritic hypertrophy, altered synaptic plasticity, and cannabinoid receptor type 1 (CB1R) down-regulation were observed in the DLS of CIE-exposed animals; increased dendritic length, prevention of long-term depression, and raised levels of 2-arachidonoylglycerol (2-AG, an endogenous cannabinoid) were observed. Behavioral testing revealed that CIE-exposed mice made fewer errors in a visual discrimination test, known to be dependent upon DLS functioning (Graybeal et al., 2011). When CIE was given after learning the task and preceding a reversal of the criterion for reinforcement, another task which is highly sensitive to DLS lesions, ethanol-exposed mice again made fewer errors (DePoy et al., 2013; Graybeal et al., 2011). This latter effect can be conceptualized as a deficit in formation of a response-outcome relationship (DePoy et al., 2013). CIE also induced a quicker decrease in conditioned approach during an omission-training schedule, a result that is inconsistent with greater DLS control over behavior because stimulus-response actions are less sensitive to omission training (Dickinson et al., 1998). Finally, CIE mice demonstrated expedited extinction of a stimulus-reward procedure, another finding lending itself to an interpretation that diverges from the hypothesis of alcohol causing stimulus-response action (DePoy et al., 2013); CIE animals in this case showed better prediction about the absence of a reward, suggesting goal-directed behavior. Overall, chronic ethanol exposure causes profound adaptations in striatal morphology, which lead to alterations in behavior, but these alterations are not neatly reconciled under theories of goal-directed and habitual actions. It is possible that under some parameters, ethanol-induced alterations in the DMS or other brain regions produce stronger effects on behavior than the alterations presently observed in the DLS, and/or non-response contingent alcohol such as CIE has different behavioral and neurological effects than alcohol as an operant reinforcer.

Although not specifically designed to assess the classical definition of habitual behavior, free-choice drinking studies demonstrate interesting parallels with research of habit learning. Recent work in primates demonstrated increased putamen activation following prolonged heavy drinking and relapse (Cuzon Carlson et al., 2011). Alcohol drinking in macaque monkeys became more stable and more inductive of elevated BECs with the progression of time, consistent with the progression toward ingrained behavior. After drinking sessions were completed, the brains of the monkeys were removed; electrophysiological analysis noted an increase in dendritic spine density, enhanced glutamatergic transmission, and other neuroadaptations driving increased excitability in the putamen. Furthermore, a study assessing the effects of dopaminergic depletion in Alcohol-Preferring (P) rats (see McBride & Li, 1998) may also show relevance to perseverative “habitual” action. Groups of rats were given bilateral microinjections of 6-OHDA or vehicle into the nucleus accumbens, either prior to initiating home-cage alcohol drinking or following several weeks of established drinking. 6-OHDA lesions blocked acquisition of alcohol drinking but did not diminish acquired drinking levels (Ikemoto, McBride, Murphy, Lumeng, & Li et al., 1997). These results suggest that dopamine signaling in the accumbens is more involved in the initial rewarding effects of initial ethanol intake than the maintenance of established drinking. In combination with the previously discussed work, a representation of drinking emerges as a process initially heavily influenced by ventral striatal regions but progressively more influenced by dorsal striatal mechanisms, supporting the theory of Everitt & Robbins (2005). Interestingly, free-choice drinking is nearly genetically isomorphic with operant oral self-administration (Green & Grahame, 2008). It is plausible, therefore, that home-cage drinking studies elicit perseverative seeking processes similar to those seen in the operant studies discussed previously.

Lastly, using a similar home-cage drinking design and the high-drinking inbred C57Bl/6J strain, Lesscher, van Kerkhof, & Vanderschuren (2010) similarly investigated the effect of duration of drinking experience on the ability of quinine adulteration (an unconditioned aversion) to suppress alcohol intake. They found less suppression in mice exposed to alcohol for 8 weeks as compared to 2 weeks and, interestingly, that quinine could completely block acquisition of alcohol drinking in naïve animals (Lesscher et al., 2010). A caveat to these experiments is that alcohol intakes during limited access were below levels that could be metabolized immediately, meaning that mice probably never encountered pharmacologically relevant BECs (which, as is the case for many of the studies reviewed in this section, are not reported). This experiment nonetheless supports the development of aversion-resistant and inflexible drinking behavior following a history of free-choice consumption of alcohol in a high-drinking rodent population. The absence of a response, and therefore inability to employ a devaluation procedure to test for predominantly mechanistic behavior, precludes these final three experiments from classification as studies of habitual behavior. However, we suggest that the ability of alcohol to cause neurological adaptations in brain regions associated with motor execution while giving rise to perseverative intake demonstrates a critical association between classically defined “habits” and persistent alcohol use.

A Preliminary Conceptualization of Habitual Alcohol Behavior in Humans

Considerable evidence from experimental human literature suggests that habit-learning models targeting the striatum and its associated circuitry have the potential to be useful in elucidating the mechanisms of prolonged, compulsive alcohol seeking. Given the theory of Everitt & Robbins (2005), supported by our above review of the alcohol literature, along with human evidence that exposure to drugs of abuse shifts associative encoding from the accumbens to the putamen, this crucial structure in learning is understudied in clinical alcoholism (Takahashi, Roesch, Stalnaker, & Schoenbaum, 2007). Evidence shows that cue-induced activation of the striatum correlates with relapse susceptibility and continued AUDs (Grüsser et al., 2004), cue reactivity on the whole is suggestive of continued alcohol problems (Cooney, Litt, Morse, Bauer, & Gaupp, 1997), and alcohol reward cues elicit higher left dorsal putamen activation in heavy drinkers (Schulte et al., 2012). In addition, preliminary evidence suggests that deep brain stimulation of the striatum may be an effective treatment in chronic, treatment-resistant alcoholics (Müller et al., 2009). The involvement of stimulus-response behavioral processes in drug seeking is further supported by a meta-analysis noting that “craving”, or extreme subjective desire to obtain pharmacological effects, does not explain all of the variance in compulsive drug use (Tiffany & Carter, 1998). Alcohol cues, like stimuli in instrumental conditioning experiments that elicit responding for devalued outcomes, appear to provoke stimulus-response alcohol-seeking behaviors that are more difficult to control than goal-directed actions.

Recent laboratory tasks support this conceptualization. Alcohol cues disrupt self-regulation in drinkers, demonstrating priming of subcortical processes that preempt inhibition and thereby reduce behavioral control. Using a visual probe task, attentional bias, reaction time, and ocular function were measured in a within-subjects design using social drinker participants and three doses of alcohol, 0.0 g/kg, 0.32 g/kg, and 0.64 g/kg (Miller & Fillmore, 2011). Significant fixations toward alcohol-related stimuli were observed, demonstrating attentional bias toward conditioned cues. Furthermore, oculomotor functions were disturbed by the alcohol doses, suggesting a relative inability to shift attention away from alcohol cues. Other research from this lab has used the attentional bias-behavioral activation assay, which is a modified go/no-go task that assesses attention and motor impulsivity, to measure inappropriate response tendencies following alcohol-related imagery (Weafer & Fillmore, 2012). As hypothesized, failure of inhibition following alcohol cues was significantly higher compared to neutral cues in adult alcohol drinkers. Therefore, stimuli associated with alcohol consume attentional resources and reduce behavioral inhibition, suggesting the activation of dominant stimulus-response processes analogous to “habits”.

Finally, conditioned aversion therapy has shown mixed results in various clinical populations of alcoholics. The most recent review that we were able to find was published over 20 years ago and supports its use but states a need for more definitive research (Elkins, 1991). While the induction of a conditioned aversion to alcohol flavors is consistently shown (e.g., Baker & Cannon, 1979; Howard, 2001), evidence that therapy imparts the ability to maintain sobriety is less convincing. The efficacy of emetic aversion therapy in promoting abstinence had not previously been determined using a controlled trial prior to a 1981 study by Cannon, Baker, & Wehl. Therapy involved five sessions in which a variety of alcoholic flavors was paired with nausea and vomiting induced through ipecac syrup. A 6-month follow-up showed a modest beneficial effect of emetic therapy promoting successful remission, but therapy participants demonstrated no difference from control participants during the 12-month follow-up. However, taste aversion induced with LiCl may be more effective, in agreement with results from preclinical literature discussed throughout this review. In one trial, half of the alcoholics who developed illness reactions to lithium treatment were abstinent at a six-month follow-up assessment (Boland, Mellor, & Revusky, 1978). Without follow-up assessments at later time points, the long-term efficacy of this practice cannot be discerned. Overall, given the extremely high propensity of taste aversion to alter behavior in general (Reilly & Schachtman, 2009), we suggest that mixed results support a perseverative, aversion-resistant component of alcoholism.

Discussion and Suggested Future Directions

This review explicates the process of habit formation in animal behavior, discusses its relation to drug abuse and addiction, examines the neurology implicated in these behaviors seen in animals and humans, and draws parallels between conditions. Although an understanding of habit formation is well developed, certain specificities of its processes are not clarified, such as the time course of neurological adaptation, and clarification of why factors such as genotype, reinforcer, and training affect the development of mechanistic behavior. Based on current findings, putamen inactivation, or the diminishment of cue-induced action, might appear to be a tempting target for addiction therapy. However, such a procedure would likely have effects that extend into other domains of reinforced behavior that could be devastating to adaptive, habit-based learning and execution. Nonetheless, it is possible that further experimentation would lead to an understanding of reinforcer-specific attributes of habitual action. Most importantly for the domain of alcoholism research, an animal model of a “habitual” alcoholic has not been developed, though the literature has established concepts and parameters that produce an excellent starting point in its creation. Such a model would acquire alcohol seeking and intake voluntarily and robustly, encounter the pharmacological effects of alcohol during training and testing, and maintain seeking and intake behaviors alike following alcohol devaluation.

Any experimenter that seeks to compare the formation of stimulus-response action in alcohol-seeking behavior to that for an alternate reinforcer must be aware that subjective reinforcer values are not necessarily matched prior to devaluation. Therefore, devaluation could be a comparatively stronger effect for a particular reinforcer, with goal-directed response rates in extinction altered accordingly. As making an interpretation of habitual behavior is in some sense based upon a null result (that is, observing a non-detectable effect of devaluation), a lower initial value of a reinforcer would plausibly lend itself more easily to discovering a “habit”. Researchers should be careful in noting comparable subjective valuations of alcohol and alternate reinforcers for such experiments. Additionally, the variable schedules that lend themselves to habit formation necessitate that patterns of intake are more gradual than could be obtained from an interval schedule. When FR schedules are used for operant responding for alcohol, the majority of responses are performed during the first 20 min of the session (e.g., Rodd et al., 2003). It follows that, for example, a hypothetical experiment matching total ethanol intakes between 1-h sessions under an FR-1 and VI-60 schedule could not assure that the pharmacological effects of ethanol encountered would be identical. However, we suggest that the impact of both of these issues can each be minimized by carefully choosing the population of animals on which to experiment. Given the documented differences in alcohol preference among genotypes of rat and mouse models, the selection of a candidate line of animals for habit-based alcohol research is not as simple as for similar research using psychostimulants.

Indeed, a chief concern of any animal study relying on oral self-administration of alcohol is that most strains of rodents do not reach BECs comparable to those obtained by humans during a drinking binge. Moreover, the only reviewed study that did not find a devaluation effect during responding for alcohol while simultaneously assessing BECs reported levels of just 30 mg/dL (Hay et al., 2013). Arguably, the habitual behavior being thereby modeled is far more comparable to that of a social drinker who regularly achieves BECs well under the legal driving limit for intoxication, rather than a problem drinker. While this human population may indeed form alcohol-based “habits” (e.g., an individual who sips a glass of wine each day after work), such habits would not be viewed as counterproductive, nor are these the behaviors that alcoholism research aims to understand and eventually treat. A solution to this issue may lie in selected lines such as alcohol-preferring P rats, which volitionally drink and instrumentally respond for oral ethanol in amounts that produce BECs at the legal limit for intoxication of 80 mg/dL or higher (see McBride & Li, 1998). Selected lines provide a more valid model of populations of problem alcohol drinkers than inbred strains, while enabling a more effective and practical study of alcoholism than heterogeneous stock because all animals consume relevant amounts of alcohol (Grahame, Li, & Lumeng, 1999). Multiple susceptibility genes affect variance in alcohol use (Köhnke, 2008), and a genotypic animal model fixing as many of these genes as possible is a plausible prospect for developing a model of non-forced dependence. The recently derived cHAP mice achieve daily volitional BECs around 260 mg/dL (Matson & Grahame, 2011), although their operant oral self-administration of alcohol remains unpublished. This line may be a candidate for testing habit formation, and would not require a sucrose fade as did the rats tested by Dickinson et al. (2002). Additionally, the inability to disrupt established ethanol drinking in P rats via nucleus accumbens dopamine depletion along with the neuroadaptations seen in the putamen upon chronic ethanol exposure (Cuzon Carlson et al., 2011; Ikemoto, McBride, Murphy, Lumeng, & Li, 1997) suggest that alcohol-based perseverative behaviors share neural mechanisms with stimulus-response actions; this concept could be further investigated using such a line.

Uncovering genotypic differences in susceptibility to habit-like behaviors seen in the absence of variation in goal-directed behavior, response frequency, or alcohol intake in a home cage or operant setting may elucidate the question of why some alcoholics are treatment-resistant. We also suggest that further study into habit formation could prove beneficial to domains other than alcoholism research, extending to other domains of addiction and self-injurious behavior. Alternatively, phenotypic variation in stimulus-response action could be used as selection criteria for a new genetic model (i.e., a “habit-prone” line). Correlated traits and candidate genes could then be identified. Furthermore, other genetic tools including knockout and transgenic animals have scarcely been used for habit testing, despite the neurological knowledge of the underlying processes, and have not been used in conjunction with alcohol- or drug-based habits. The dopamine transporter knockout mouse is a particularly interesting candidate given the dopaminergic innervation cascade throughout the striatum (Everitt & Robbins, 2005). These animals are shown to lack sensitivity to psychostimulants (Giros, Jaber, Jones, Wightman, & Caron, 1996). However, one would hypothesize that stimulus-response behavior would predominate under conventional reinforcement in a similar sense to a “normal” mouse under psychostimulant reinforcement. Further characterization of the FCG model is another attractive possibility (Barker et al., 2010; Quinn et al., 2007) for its potential use in clarification of the causes of chromosomal and gonadal disparities in habitual behavior.

The seeking/taking chain model, previously used in work with cocaine-seeking habits, could be valuable to develop further as a habitual assay using alcohol (Ito et al., 2002; Vanderschuren et al., 2005; Zapata et al., 2010). The similar appetitive/consummatory model did not show habitual behavior with alcohol as a reinforcer, but it is possible based upon the parameters of that particular study that more experience/training would be necessary for the development of dominant stimulus-response action (Samson et al., 2004). Also of recent interest is the relationship between subjective "craving", as mediated by the PFC and limbic areas, and habitual seeking processes (Volkow et al., 2006). These regions could be targeted in drug studies using animal models to examine what role they have in habitual drug seeking. Experimentation into effects of pharmacotherapies at different stages of operant training, aversion conditioning, or behavioral testing should also be considered. Importantly, we reiterate that LiCl or other devaluation through conditioned aversion should be used exclusively in studies that seek to manipulate the subjective value of alcohol, because satiation with alcohol can lead to depressed response rates, which may be incorrectly interpreted as a devaluation effect. Overall, we suggest that the neurological and behavioral processes of stimulus-response and response-outcome learning may answer lingering questions in the treatment and understanding of alcoholism, and therefore should be carefully pursued.