Aberrant Learning and Memory in Addiction

Abstract

Over the past several years, drug addiction has increasingly been accepted to be a disease of the brain as opposed to simply being due to a lack of willpower or personality flaw. Exposure to addictive substances has been shown to create enduring changes in brain structure and function that are thought to underlie the transition to addiction. Specific genetic and environmental vulnerability factors also influence the impact of drugs of abuse on the brain and can enhance the likelihood of becoming an addict. Long-lasting alterations in brain function have been found in neural circuits that are known to be responsible for normal appetitive learning and memory processes and it has been hypothesized that drugs of abuse enhance positive learning and memory about the drug while inhibiting learning about the negative consequences of drug use. Therefore, the addict's behavior becomes increasingly directed towards obtaining and using drugs of abuse, while at the same time developing a poorer ability to stop using, even when the drug is less rewarding or interferes with functioning in other facets of life. In this review we will discuss the clinical evidence that addicted individuals have altered learning and memory and describe the possible neural substrates of this dysfunction. In addition, we will explore the preclinical evidence that drugs of abuse cause a progressive disorder of learning and memory, review the molecular and neurobiological changes that may underlie this disorder, determine the genetic and environmental factors that may increase vulnerability to addiction, and suggest potential strategies for treating addiction through manipulations of learning and memory.

Introduction

Humans sample psychoactive substances for a variety of reasons including obtaining pleasurable feelings, relieving pain or stress, reducing anxiety, increasing energy, and enhancing sociability. Using drugs for these purposes in of themselves is not necessarily harmful to the individual or to society as a whole; rather, it is the chronic, uncontrolled use of drugs at the expense of normal activities and in spite of adverse consequences that characterizes drug addiction. Drug addiction does not occur overnight, but takes years to develop, and indeed is hypothesized to be learned. Drug use begins by learning that the substance is rewarding due to powerful interoceptive effects, thus increasing the desire to use the drug again. Over time certain cues in the environment become associated with drug use until the cues alone are sufficient to stimulate desire or craving for the drug. Learning about reward-related cues also occurs when humans learn to find food or mates, build shelters, or earn money, but drugs of abuse alter and enhance these normal learning and memory processes. In addition, drugs of abuse consumed over long periods of time cause dysfunction in neural systems mediating self-control over behavior. In other words, addictive drugs not only enhance normal learning systems involved in seeking rewards, but also inhibit the ability to control use, especially in vulnerable populations (Jentsch, & Taylor, 1999; Everitt et al., 1999).

Drugs of abuse are able to usurp normal learning and memory systems through their direct pharmacological actions on multiple neurotransmitter systems. For example, when a new event occurs in the environment a dopamine signal is generated from the midbrain to both dorsal and ventral striatum that facilitate learning about this new event (Schultz, 2010; Jay, 2003). A common feature of drugs of abuse is that, despite different mechanisms of action, they all cause an increase in dopamine release in the striatum (Torregrossa, & Kalivas, 2008; Di Chiara, 1999). Therefore, addictive substances create artificial learning signals that are of a greater magnitude and duration than what is observed neurochemically in response to natural events. Enhanced learning about a drug positive experience results in an increased likelihood to use the drug again. This effect is magnified each time the drug is consumed. Drugs of abuse not only increase learning about the positive effects of the drug but they overshadow and diminish the impact of other features of the environment resulting in increased attention toward the drug and away from normal activities.

The ability of addictive substances to alter normal glutamate transmission is another means by which drugs directly affect learning and memory processes (Kalivas, 2004). Chronic drug exposure results in long-lasting alterations in glutamate homeostatsis in the nucleus accumbens coupled with enhanced glutamatergic transmission from the prefrontal cortex upon re-exposure to the drug (McFarland, Lapish, & Kalivas, 2003; LaLumiere & Kalivas, 2008). Glutamatergic activity is also necessary for virtually all forms of learning, and is critical for prefrontal cortical control over decision making and impulsivity (Stefani, Groth, & Moghaddam, 2003; Moghaddam, Adams, Verma, & Daly, 1997). Therefore, addictive drugs have the ability to profoundly influence learning and memory circuits. Indeed, if not for this property of drugs of abuse it is not clear if they would be able to produce addictive behavior.

The clear overlap between the neurochemical and molecular processes involved in normal learning and memory and the pharmacological actions of drugs of abuse has led many scientists to hypothesize that addiction develops, at least in part, because of aberrant learning and memory (Kilts, 2001; White, 1996; Everitt, Dickinson, & Robbins, 2001; Kelley, 2004; Jentsch, & Taylor, 1999; Thomas, Kalivas, & Shaham, 2008; Hyman, Malenka, & Nestler, 2006). In this review, we will focus on how drugs of abuse alter normal learning and memory and discuss how specific vulnerability factors influence the ability of abused drugs to affect learning and memory systems. We will also describe human, clinical studies that have investigated how normal learning and memory systems are affected in addicts. Finally, we will explore how studies of learning and memory have influenced the direction of current research into novel treatment strategies for addiction.

Behavioral Effects of Addictive Drugs on Learning and Memory

While it has long been observed that strong associations develop between addictive drugs and the environmental stimuli and contexts in which they are encountered, it is increasingly appreciated that chronic drug exposure leads to long-lasting changes in the circuits underlying normal learning and memory processes. Some of the first studies demonstrating drug-induced changes in normal learning examined the effect of psychostimulants on incentive learning about reward-predictive cues. Taylor and Horger (1999) found that chronic cocaine exposure facilitates responding for a water paired-cue (conditioned reinforcer (CR)) using the acquisition of a new response procedure. In addition, chronic cocaine exposure sensitized the ability of intraaccumbens amphetamine to potentiate responding for the conditioned reinforcer. This study expanded upon older studies demonstrating that dopamine signaling in the nucleus accumbens is required for responding for a CR (Taylor, & Robbins, 1984; Taylor, & Robbins, 1986). In addition, it was one of the first studies to unequivocally demonstrate that chronic exposure to an addictive drug persistently alters normal learning about non-drug, reward-associated cues. Subsequent studies have further demonstrated that acute and chronic nicotine (Olausson, Jentsch, & Taylor, 2004a; Olausson, Jentsch, & Taylor, 2004b), chronic phencyclidine (PCP) (Jentsch, & Taylor, 2001), and chronic cocaine (Olausson et al., 2007) enhance CR responding for a natural reinforcer in rats or non-human primates.

Likewise, chronic drug exposure can enhance Pavlovian conditioning to a natural reinforcer using the conditioned Pavlovian approach paradigm. Chronic exposure to nicotine (Olausson, Jentsch, & Taylor, 2003), cocaine, or amphetamine, but not MDMA (Taylor, & Jentsch, 2001; Harmer, & Phillips, 1999; Harmer, & Phillips, 1998), enhances Pavlovian approach, while approach to the unconditioned stimulus is unchanged. In addition, Wyvell and Berridge found that chronic amphetamine exposure selectively enhanced responding energized by a Pavlovian-conditioned cue using the Pavlovian-to-instrumental transfer paradigm, while not altering the hedonic value of the reinforcer (Wyvell, & Berridge, 2001). These results indicate that chronic drug exposure sensitizes or enhances attribution of incentive value to reward-predictive stimuli, accelerating cue-associated learning and memory, while not necessarily making the reinforcer more “rewarding”. In a situation where both drug and natural reward-related cues are present, it is likely that cues paired with drug will exert much stronger motivational influences on behavior than cues that had been learned in the absence of drug. Indeed, this hypothesis agrees well with the incentive sensitization theory of addiction (Robinson, & Berridge, 1993).

The studies described above examined the ability of chronic drug exposure to enhance later learning about associations between discrete cues and rewards. Few studies have determined whether chronic drug exposure affects other forms of learning. However, one interesting study examined the ability of exposure to a drug-conditioned stimulus (in this case a context) to enhance spatial learning in the Morris water maze. A novel aspect to this study was that the researchers did not simply examine whether chronic drug exposure affected spatial learning, but determined if simply exposing rats to a drug-conditioned context prior to water maze training was sufficient to alter learning. Both learning to associate specific contexts with reinforcer availability and spatial learning require activity in the hippocampus (Sweatt, 2004; Fanselow, 2000; Holland, & Bouton, 1999). Therefore, the researchers hypothesized that exposure to drug-conditioned contextual stimuli would excite hippocampal activity and enhance learning of a hippocampus-dependent task in a drug-free state. Indeed, exposure to either a morphine- or cocaine-conditioned context prior to training sessions enhanced learning in the Morris water maze in comparison to drug-conditioned rats that were not re-exposed to the context. Exposure to a sucrose-paired context or a context associated with naloxone-precipitated withdrawal did not alter water maze learning (Zhai et al., 2007). Therefore, exposure to contexts associated with drugs can enhance other forms of spatial learning in a manner that is not achieved by contexts associated with natural rewards or aversive states. In addicted individuals, this unique property of drugs of abuse could result not only in drug context-induced craving, but also an enhanced ability to learn how and where to obtain drugs. Consequently, drugs of abuse enhance aspects of cue- and context-associated learning that support continued drug use.

Chronic exposure to drugs of abuse can also accelerate habit formation, or the learning of specific motor actions to receive rewards. Habit learning is a natural learning process that allows stereotyped, repetitive motor sequences to be performed with little to no conscious awareness. It is believed that habit learning allows cognitive resources to be freed for managing more complex actions and learning new information. Habit formation involves a switch in the control of behavior from a circuit including the ventral and medial striatum and dorsal prefrontal cortex (PFC) to a circuit involving the dorsal lateral striatum and ventral medial PFC (Killcross, & Coutureau, 2003; Yin, Knowlton, & Balleine, 2004; Yin, Ostlund, Knowlton, & Balleine, 2005). Habit learning is generally advantageous to an organism, but can become pathological when a habit is so ingrained that an individual has difficulty refraining from a habitual action even when the outcome of that action has changed for better or worse. Several investigators have hypothesized that addicts form habits related to drug use more rapidly and possibly to a greater degree than non-addicted individuals, and in comparison to habits formed for non-drug reinforcers (Everitt, & Robbins, 2005).

The formation of habits is traditionally assessed by reinforcer devaluation procedures. In these experiments, animals learn to perform an action (e.g., lever pressing) for a food reward and over time, with repeated instrumental schedules of training, the action can become habitual. If the behavior is habitual, it is relatively insensitive to changes in the value of reinforcer outcome. So, if the reinforcer is devalued by pairing with illness or by pre-feeding satiation, the animal continues to perform the action as if the value of the reinforcer was unchanged. In animals behaving in a flexible, goal-directed manner, performance of the action is appropriately reduced after reinforcer devaluation. Chronic amphetamine exposure has been shown by multiple labs to accelerate the formation of habitual responding for food at a point in training where control animals are goal-directed (Nordquist et al., 2007; Nelson, & Killcross, 2006). The effect of amphetamine was specific to newly learned actions and did not affect goal-directed behavior that was learned prior to drug exposure (Nelson, & Killcross, 2006). Therefore, chronic exposure to drugs of abuse accelerates habit formation for natural reinforcers, suggesting that drug exposure sensitizes neural systems involved in habit formation such that behaviors are more likely to become inflexible.

While it is interesting that drug exposure (at least to amphetamine) can cause long-term changes in the speed of habit formation, the critical question is: Do habits for drugs form faster than for natural rewards? If drug-seeking actions are more likely to become habits than other types of responses it might explain why addicts preferentially seek out drugs over natural reinforcers, even when the reinforcing value of the drug has been reduced and the consequences of drug use are detrimental to the individual. Researchers have attempted to address this question by comparing the rate of habit formation for an oral drug reinforcer to habit formation for an oral food reinforcer. Oral drug reinforcers were used because it is more straightforward to devalue an oral reinforcer by LiCl-induced illness than to devalue an intravenous reinforcer. In these studies, responding for both oral ethanol and oral cocaine (Dickinson, Wood, & Smith, 2002; Miles, Everitt, & Dickinson, 2003) were found to become habitual when responding for oral sucrose remained goal-directed; thus, providing evidence that actions aimed at obtaining drug reinforcers do become habitual faster than responses for non-drug reinforcers. However, it should be noted that all of these studies were conducted in male rodents, and while habitual responding for ethanol does develop more quickly than for food in chromosomal male mice, the opposite is true for chromosomal female mice, indicating that genetic sex differences may influence the effect of drugs on habit circuitry (Barker, Torregrossa, Arnold, & Taylor, 2010; Quinn, Hitchcott, Umeda, Arnold, & Taylor, 2007).

In addition, some studies have attempted to evaluate the habitual nature of drug seeking using intravenous reinforcers. In a clever study by Olmstead and colleagues (Olmstead, Lafond, Everitt, & Dickinson, 2001), animals were trained to respond for cocaine on a second order seeking-taking chained schedule. Responding on the seeking part of the chain was devalued by extinguishing responding on the taking part of the chain. Using this procedure, cocaine-seeking actions remained goal-directed in a similar manner to sucrose seeking with moderate training. A more recent study used the same procedure but with extensive training found that cocaine seeking does become habitual. In addition, the researchers found that inactivation of the dorsal lateral striatum prevented expression of a cocaine-seeking habit, supporting the view that the dorsal lateral striatum subserves habit learning (Zapata, Minney, & Shippenberg, 2010). In addition, a variation of this procedure has found that when responding on the seeking chain of the second-order schedule is devalued by presentation of a shock-paired conditioned stimulus or by a direct foot shock, responding is reduced after limited, but not prolonged cocaine self-administration experience. This was not true of sucrose seeking actions, which remained goal-directed even after prolonged training (Vanderschuren, & Everitt, 2004; Pelloux, Everitt, & Dickinson, 2007). Taken together, all of these data suggest that chronic exposure to drugs of abuse produces plasticity in neural circuits responsible for promoting habitual behavior. In addition, when compared directly, responding for drug reinforcers is much more difficult to disrupt by devaluation or punishment than a food reinforcer. Therefore, when an addict is confronted with multiple reinforcers in the environment, actions directed toward obtaining drug will be initiated more readily and will be persistent in the face of adverse consequences.

Molecular and Physiological Effects of Addictive Drugs on Learning and Memory

The ability of addictive drugs to produce enduring changes in learning about cue-reward associations and developing habits, suggest that drugs may cause long-lasting synaptic plasticity in cortico-limbic-striatal circuitry. Drugs may produce these effects via altered gene and protein expression/activity, changes in electrophysiological activity, and by modifying neural morphology. A wealth of research has clearly demonstrated that chronic use of addictive substances causes long-term changes to brain structure and function in many brain regions, including the PFC, dorsal and ventral striatum, hippocampus, hypothalamus, amygdala, and ventral tegmental area (VTA). The exact behavioral consequence of all the neurobiological changes associated with chronic drug use have not yet been determined, but certain aspects of drug-induced neuroplasticity in learning and memory circuits have been well-characterized and will be the focus of our discussion here.

Among the strongest evidence for neurobiological alterations in systems associated with reward-related learning and memory comes from a series of reports showing that chronic drug exposure increases activity of the dopamine-regulated cAMP/protein kinase A (PKA) pathway in cortico-limbic-striatal circuits (Nestler, 2004). Chronic morphine and cocaine exposure has been found to increase PKA activity in the nucleus accumbens and to some extent in the amygdala (Terwilliger, Beitner-Johnson, Sevarino, Crain, & Nestler, 1991; Pollandt et al., 2006). Likewise, stimulation of PKA within the amygdala facilitates stimulus-reward learning and mimics the facilitation of stimulus-reward learning found after prior chronic cocaine, amphetamine or nicotine exposure in rodents (Olausson, Jentsch, & Taylor, 2003; Harmer, & Phillips, 1998; Jentsch, Olausson, Nestler, & Taylor, 2002; Taylor, & Jentsch, 2001). Moreover, our laboratory has demonstrated that inhibition of amygdala PKA activity impairs the acquisition of appetitive stimulus-reward learning using the Pavlovian approach paradigm (Jentsch, Olausson, Nestler, & Taylor, 2002). Therefore, because normal appetitive learning requires PKA activity, the increased PKA activity, particularly in the amygdala, produced by chronic drug exposure is likely to be one mechanism by which drug-associated memories become abnormally strong. Indeed, we have shown that inhibiting PKA activity in the amygdala after a drug-memory reactivation (discussed below) can inhibit cue-induced reinstatement, suggesting that manipulations of PKA activity might help treat addiction (Sanchez, Quinn, Torregrossa, & Taylor, 2010).

PKA activation leads to the downstream phosphorylation and activation of CREB, which leads to the increased expression of several genes. Chronic cocaine and amphetamine exposure increase CREB phosphorylation and activity in the nucleus accumbens and the amygdala (Konradi, Cole, Heckers, & Hyman, 1994; Shaw-Lutchman, Impey, Storm, & Nestler, 2003; Mattson et al., 2005; Brenhouse, Howe, & Stellar, 2007), while chronic morphine and nicotine exposure decrease levels and activity of CREB in these regions (Pandey, Roy, Xu, & Mittal, 2001; Brunzell, Russell, & Picciotto, 2003; Widnell et al., 1996). Consequently, the specific role CREB activity plays in the development of addiction is complicated. CREB has been shown to produce opposing roles on drug-motivated behavior depending on the brain region studied. Overexpression or activation of CREB in the nucleus accumbens has been found to reduce, while reductions in CREB activity have been found to enhance, the rewarding effects of psychostimulants (Carlezon et al., 1998; Walters, & Blendy, 2001). CREB, however, is required for the rewarding properties of nicotine and morphine (Walters, Cleck, Kuo, & Blendy, 2005; Walters, & Blendy, 2001), suggesting an important role for CREB depending on the pharmacological mechanism of action and the brain region studied. However, because CREB-regulated gene transcription is essential for virtually all forms of memory consolidation, including amygdala-dependent memory (Kida et al., 2002; Josselyn et al., 2001; Lonze, & Ginty, 2002; Josselyn, & Nguyen, 2005; Lamprecht, Hazvi, & Dudai, 1997; Josselyn, Kida, & Silva, 2004; Carlezon, Duman, & Nestler, 2005), it is possible that even short-term drug-induced increases in PKA/CREB signaling in the amygdala or nucleus accumbens could contribute to the ability of stimuli to acquire enhanced conditioned reinforcing properties and increase the ability of drug-associated stimuli to drive behavior. In support of this hypothesis, our laboratory has found that nucleus accumbens infusion of BDNF, a growth factor known to activate CREB through stimulation of TrkB receptors, potentiates responding for a conditioned reinforcer, and this effect is augmented by cocaine administration (Horger et al., 1999). On the other hand, cocaine-induced increases in CREB and BDNF activity in the medial PFC appear to put a brake on non-cued progressive ratio responding for cocaine (Sadri-Vakili et al., 2010). Therefore, while CREB may have differential effects on the direct reinforcing properties of drugs, CREB activity may be important for stimuli associated with drugs to become conditioned reinforcers.

However, few studies have identified a direct link between persistent drug-induced neuroadaptive changes and altered processing of reward-related stimuli. One exception is the association between a time-dependent increase in the reinforcing effects of drug cues (i.e., incubation) and increases in extracellular signal regulated kinase (ERK) activity in the central nucleus of the amygdala and an increase in BDNF in the VTA, accumbens, and amygdala (Grimm et al., 2003; Lu et al., 2005; Li et al., 2008). Activation of ERK also leads to the activation of CREB, and CREB phosphorylation is enhanced in the central nucleus of the amygdala during incubation of morphine conditioned place preference (Li et al., 2008). In addition, blockade of ERK and CREB activation in this region can block incubation of the incentive motivational value of drug stimuli (Li et al., 2008; Lu et al., 2005). Moreover, in unpublished work from our laboratory, we have found that inhibition of PKA and CREB activity in the central amygdala can prevent the enhanced responding for conditioned reinforcement produced by cocaine, while activation of CREB mimics the effect of cocaine (Olausson et al., submitted). Therefore, CREB activity, either through activation by BDNF and ERK, or through PKA, may cause long-lasting increases in the ability of drug cues to motivate behavior, particularly through actions in the amygdala, and possibly the nucleus accumbens.

Another transcription factor that is altered in a long-lasting manner by chronic drug exposure is DeltaFosB (Nestler, Barrot, & Self, 2001). The effects of chronic drug exposure on DeltaFosB expression and function have been studied extensively, but the role DeltaFosB plays in mediating incentive learning associated with drugs is less well known. However, we have found that overexpression of DeltaFosB in the nucleus accumbens of transgenic mice, or by viral-mediated gene transfer in rats, increases instrumental responding and progressive ratio breakpoints for food in a similar manner to chronic exposure to several drugs of abuse (Olausson et al., 2006). However, mice with deletions in striatal FosB show greater locomotor activity in response to cocaine and form place preferences at lower doses (Hiroi et al., 1997). Therefore, DeltaFosB, like CREB, may affect different aspects of addiction and be temporally dependent (McClung, & Nestler, 2003), with high levels of DeltaFosB increasing normal learning and memory processes, while also counteracting some of the intrinsic pharmacological and conditioned reinforcing effects of drugs.

One mechanism by which DeltaFosB may affect addiction-related behaviors is through increasing the expression of cyclin-dependent kinase 5 (Cdk5). Both overexpression of DeltaFosB and chronic cocaine exposure increase Cdk5 mRNA, protein, and activity in the striatum (Bibb et al., 2001). Our laboratory has found that pharmacological inhibition of Cdk5 enhances the cocaine-induced increase in responding for conditioned reinforcement. In addition, Cdk5 inhibition increases cocaine-induced locomotor activity and break points for self-administered cocaine (Taylor et al., 2007). Likewise, genetic and viral-mediated reduction in Cdk5 activity increased incentive motivation for food and facilitated a cocaine conditioned place preference (Benavides et al., 2007). Therefore, chronic drug-induced increases in DeltaFosB and Cdk5 appear to be part of a feedback system that moderates the effects of drugs on behavior.

Further evidence supporting this hypothesis comes from studies of cocaine's effects on dendritic spine density in the nucleus accumbens. Chronic cocaine and amphetamine increase the density of dendritic spines and spine head size in the accumbens (Robinson, & Kolb, 1997; Robinson, & Kolb, 1999; Robinson, Gorny, Mitton, & Kolb, 2001; Shen et al., 2009). In addition, the Cdk5 inhibitor roscovitine prevents the increase in accumbens spine density produced by chronic cocaine exposure (Norrholm et al., 2003). If the increase in accumbens spine density and Cdk5 activity produced by cocaine are indeed protective against the addictive properties of cocaine, then one would expect that preventing the drug-induced changes in spine density would produce an increase in addiction-like behaviors. Accordingly, an inhibitor of actin cycling, latrunculin A, which prevents dynamic changes in spine morphology, actually increases cocaine-primed reinstatement of drug seeking (Toda, Shen, Peters, Cagle, & Kalivas, 2006), providing evidence that psychostimulant-induced changes in DeltaFosB, Cdk5, and spine morphology in the accumbens are protective against some of the effects of drugs. However, it is not clear how these molecular and morphological changes influence other learning and memory processes.

In addition to the nucleus accumbens, drug-induced changes in spine morphology have been reported in the orbital frontal cortex (OFC). However, unlike the accumbens, chronic amphetamine self-administration actually decreases spine density (Crombag, Gorny, Li, Kolb, & Robinson, 2005), while chronic morphine increases spine density (Robinson, Gorny, Savage, & Kolb, 2002) in the OFC. In addition, chronic cocaine administration disrupts reversal learning, much like lesions of the OFC (Schoenbaum, Saddoris, Ramus, Shaham, & Setlow, 2004), suggesting that the psychostimulant-induced reduction in spine density in this region may have detrimental effects on certain forms of learning. Indeed, neurons in the OFC of cocaine-treated rats failed to signal adverse outcomes in a decision-making task and these neurons lost the ability to reverse their cue-selectivity during behavioral reversals. Again, these data suggest that chronic drug use causes impairments in synaptic plasticity in learning and memory systems (Stalnaker, Roesch, Franz, Burke, & Schoenbaum, 2006). On the other hand, it is unclear whether the ability of chronic morphine to increase spine density in this region has behavioral consequences. Though, one study has reported that cue-induced reinstatement of heroin, but not sucrose seeking, activates immediate early gene expression in the OFC, which indicates a possible increase in the engagement of this region that is specific for heroin-seeking behavior (Koya et al., 2006).

The molecular and morphological changes produced by drugs of abuse can result in alterations in the physiology of learning and memory circuits (as discussed briefly above), which likely underlies changes in behavior. One area that has well-documented changes in physiology after drug exposure is the VTA. Dopaminergic activity in the VTA is not only critical for the pharmacological effects of most drugs of abuse, but its projections to the nucleus accumbens, PFC, and amygdala also underlie learning about rewarding and aversive events in the environment. Acute and chronic cocaine can induce long-term potentiation (LTP) at excitatory synapses on dopaminergic cells in the VTA that lasts several days (Ungless, Whistler, Malenka, & Bonci, 2001; Borgland, Malenka, & Bonci, 2004). LTP is a putative mechanism underlying learning and memory; however, the increase in LTP induced by cocaine in the VTA (indexed by the AMPAR/NMDAR ratio) has to date, not been correlated with specific learning effects. The AMPAR/NMDAR ratio has been shown to correlate with acute cocaine-induced locomotor activity, but not with locomotor activity after 7 days of cocaine or to a challenge dose of cocaine in a sensitization paradigm (Borgland, Malenka, & Bonci, 2004). Therefore, chronic cocaine may alter the normal association between LTP at VTA synapses and locomotor output. In contrast to experimenter-administered cocaine, self-administered cocaine has been shown to produce an enduring (at least 3 month) increase in LTP in the VTA that is resistant to disruption by extinction training. Self-administration of natural rewards also increased LTP, but only transiently (Chen et al., 2008). Therefore, chronic drug use may increase the drive to take drugs even after long periods of abstinence due to an abnormal enhancement of synaptic strength in the VTA.

In conclusion, chronic drug exposure not only produces long-lasting changes in behaviors relevant to learning and memory, but also produces enduring plasticity in the circuits and molecules that underlie learning and memory processes. Many of these changes occur in PKA, ERK, and CREB signaling cascades. In addition, some changes appear to occur as a negative feedback system to limit the effects of chronic drug use on behavior, including the observed increases in DeltaFosB and Cdk5 signaling. Interestingly, many published studies actually find opposite effects of psychostimulants and other classes of drugs, like opiates, on spine density and transcription factor manipulations. Therefore, the precise relevance of some of these molecules and morphological changes to the development of addiction still needs to be determined. Further study into the relationship between these molecular changes and effects on learning and memory processes associated with addiction is warranted because molecular changes that limit drug effects on locomotor activity and primary reward may still promote long-lasting increases in incentive learning and possibly the formation of drug-associated habits and compulsions.

Individual Vulnerability to the Effects of Addictive Drugs on Learning and Memory

We have identified many long-term effects of addictive drugs on basic learning and memory processes and hypothesized that these effects of drugs of abuse mediate, at least in part, the development of addiction. However, many people sample addictive substances, and some even use drugs or alcohol over many years, but only a small subset of the population progresses to addiction. Therefore, there must be environmental and/or genetic and biological factors that cause some individuals to be more susceptible to the addictive effects of drugs. One possibility is that certain vulnerability factors cause addictive drugs to have more profound and enduring effects on cue-related learning and memory. A variety of factors have been found to influence the basic reinforcing and/or ‘euphoric’ effects of abused drugs, but in this review we will focus our discussion on factors that may influence the effects of drugs on learning and memory processes that may facilitate the development of addiction.

Certain basic biological factors modulate the effects of drugs on the brain, such as gender, hormonal status, and age. While currently more males than females meet the clinical definition of addiction, the gap is closing, which is likely due to increased opportunities for women to be exposed to addictive drugs. In addition, women are thought to progress to addiction more quickly than males and may be more likely to relapse (Becker, 2009). Some preclinical studies have attempted to determine how drugs of abuse differentially affect males and females. In self administration studies, females are consistently found to acquire psychostimulant self administration more quickly and at lower doses than males (Lynch, & Carroll, 1999; Hu, Crombag, Robinson, & Becker, 2004; Roth, Cosgrove, & Carroll, 2004). Females also develop a conditioned place preference to several drugs of abuse at lower doses and with fewer pairings than males (Russo et al., 2003; Mathews, & McCormick, 2007; Karami, & Zarrindast, 2008; Zakharova, Wade, & Izenwasser, 2009). Circulating gonadal hormones also influence self-administration and drug-primed reinstatement. Estradiol generally enhances the sensitivity of females to drugs (Kerstetter, Aguilar, Parrish, & Kippin, 2008; Kippin et al., 2005; Hu, Crombag, Robinson, & Becker, 2004), while progesterone attenuates the motivational effects of drugs (Russo et al., 2008; Larson, Anker, Gliddon, Fons, & Carroll, 2007; Feltenstein, Byrd, Henderson, & See, 2009). While all of these studies certainly indicate that females have different responses to addictive drugs, few studies have provided strong evidence that females are differentially sensitive to the effects of drugs on learning and memory. There have been reports that females form conditioned place preferences with fewer drug-context pairings than males, providing some indication that associative memories about environmental contextual stimuli and drugs form more readily in females (Russo et al., 2003). However, females do not show major differences from males in cue-induced reinstatement to cocaine seeking, suggesting that cues are not stronger in motivating active behaviors in females (Fuchs, Evans, Mehta, Case, & See, 2005). Females do show greater cocaine-primed reinstatement, again suggesting that females are more sensitive to the pharmacological effects of the drug (Kippin et al., 2005). No studies to our knowledge have directly addressed whether chronic exposure to drugs of abuse differentially enhances incentive learning in males and females.

Our laboratory has found that chromosomal females form habits to food more rapidly than chromosomal males (Quinn, Hitchcott, Umeda, Arnold, & Taylor, 2007), but that chromosomal males more rapidly form habits for ethanol (Barker, Torregrossa, Arnold, & Taylor, 2010). In addition, gonadal females were found to drink more ethanol than males during a free drinking period. Therefore, females may be more sensitive to certain aspects of drug addiction depending on the reinforcer, and gonadal hormones influence this sensitivity. Overall, these data suggest that females are generally more sensitive to the reinforcing properties of drugs driving consumption, but are not necessarily more likely to form stronger cue-drug associations, and males may more readily form habits. In addition, few studies have examined if males and females have different biochemical responses to drugs of abuse in circuits involved in learning and memory. One study in our laboratory did find that females have enhanced PKA activity at DARPP-32 in the dorsal and ventral striatum after abstinence from cocaine, but the behavioral consequences of this sex difference are unclear (Lynch, Kiraly, Caldarone, Picciotto, & Taylor, 2007). Certainly, more research into sex differences in drug effects on learning and memory systems are warranted to develop a better understanding of differences in male and female vulnerability to addiction.

Another factor that influences the effects of drugs on behavior is age. Adolescents are much more sensitive to the locomotor stimulant effects of drugs and, like females, form conditioned place preferences at lower doses (Badanich, Adler, & Kirstein, 2006; Zakharova, Wade, & Izenwasser, 2009; Mathews, & McCormick, 2007). Adolescent drug use is also associated with an increased risk for becoming addicted as an adult (Chambers, Taylor, & Potenza, 2003). It is possible that adolescent drug use alters the development of cortical circuits such that learning and memory processes associated with drug use are enhanced. Very few studies have determined the long-term effects of adolescent drug exposure on normal learning and memory processes. Notably, we (Gourley, Koleske, & Taylor, 2009) have used the Abl-related gene (Arg) kinase knock-out mouse as a model of the dendritic simplification present in the adolescent brain to study the effects of drugs of abuse on behavior. Adult Arg deficient mice, like adolescents, are more sensitive to the locomotor stimulant effects of cocaine compared to wild-type mice, and show impairments in a medial orbital frontal cortex-dependent reversal learning task after chronic exposure to a dose of cocaine that does not impair wild-type mice. Therefore, the immature structure of the adolescent brain may make adolescents more sensitive to specific cocaine-induced learning deficits (Gourley, Taylor, & Koleske, in press). An interesting question to pursue is whether adolescent drug, or possibly stress, exposure causes changes to brain structure and function such that these individuals are permanently more sensitive to drug-induced changes in learning and memory processes. One study has demonstrated that adolescent cocaine exposure leads to increased basal levels of dopamine in the nucleus accumbens compared to adults treated with cocaine, and an enhanced accumbal dopamine response to sucrose as adults (Catlow, & Kirstein, 2007). Therefore, adolescent drug exposure does have some permanent effects of dopamine mediated learning processes. However, another study found no effect of adolescent cocaine or methylphenidate self-administration on Pavlovian approach or responding for a CR, suggesting that cue motivated learning may not be affected by adolescent drug exposure (Burton, Nobrega, & Fletcher, 2010). Other forms of cognitive control may be permanently impaired after adolescent drug exposure, such as impulsivity, behavioral flexibility, and habit formation. Much additional research is needed to understand how adolescent onset drug exposure affects learning and memory processes and predisposes individuals to addiction.

Numerous studies have examined how individual differences in personality traits or innate learning and memory influence the development of addictive behaviors. These studies allow researchers to determine if chronic drug exposure causes changes in normal learning and memory circuits or if innate differences in learning and memory result in an enhanced likelihood of developing an addiction. Trait impulsivity, response to novelty (‘sensation seeking’), and sign-tracking have all been studied extensively in rodents to determine to what extent these traits predict the later development of addiction-like behaviors. The sign-tracking model is interesting because it examines the relation of an animal's inherent reactivity to reward-associated stimuli to the development of addiction. One might hypothesize that if an individual readily attributes incentive value to cues, then drug-associated stimuli would become quite strong motivators of behavior, and habits may form more readily. In these experiments, presentation of a lever signals food reward availability. A subset of rats selectively engage with the lever stimulus and are dubbed ‘sign-trackers’, while a different subset of rats prefer to approach the food magazine and are dubbed ‘goal-trackers’ (Flagel, Watson, Robinson, & Akil, 2007). Sign-trackers show increased development of locomotor sensitization to cocaine, but other tests of addiction vulnerability such as progressive ratio responding, extinction, reinstatement, insensitivity to punishment, and habit have yet to be reported.

However, another form of individual variation, known as the high responder/low responder (HR/LR) model of sensation seeking has been shown to share characteristics with the sign-tracking/goal-tracking phenotype. High responders are animals that have increased locomotor activity in a novel environment, and have been shown to more readily acquire drug self administration (Piazza, Deminière, Le Moal, & Simon, 1989; Piazza et al., 1990). Flagel and colleagues have bred HR and LR rats over several generations and found that HRs have a very high probability of being sign-trackers while LRs have a high probability of being goal-trackers. The bred HRs also show greater sign-tracking directed toward a cocaine-associated cue, suggesting that intrinsic differences in incentive learning about reward-associated cues translates to enhanced learning about drug-associated cues. These animals also show increases in certain forms of impulsive action (Flagel et al., 2010). These animals also differ in their neurochemical response when learning to attribute incentive and/or predictive value to reward-associated cues. Sign-trackers show a robust phasic dopamine response in the accumbens core in response to a food-predictive cue that is much greater than the dopamine response elicited by the food itself. On the other hand, goal-trackers have a cue-elicited dopamine response of much lower magnitude than sign-trackers, and this response is not different from the dopamine response elicited by the food itself (Flagel et al., 2011). This study provides convincing evidence that innate differences in incentive learning about reward-predictive cues are related to differences in the neurobiological response during normal cue-reward learning. In addition, this innate difference translates to differences in vulnerability to some aspects of addiction.

In support of these findings, another study determined that outbred rats with high novelty preference (novelty-seeking) were also sign-trackers, and that these rats more readily acquired cocaine self-administration at lower doses than low novelty-seeking/goal-tracking rats (Beckmann, Marusich, Gipson, & Bardo, 2010). Therefore, genetic or environmental factors that predispose individuals to attribute incentive value to cues may overlap with factors predicting risk-taking behavior and a combination of these phenotypes may result in enhanced risk for addiction.

While attribution of incentive value to environmental cues associated with reward may constitute a risk factor for sampling drugs initially, it is not yet clear whether this phenotype increases the likelihood of developing addiction. However, trait impulsivity on the 5-choice serial reaction time task (5-CSRTT) does seem to predict an enhanced vulnerability to the compulsive component of addiction, while response to novelty does not. The most impulsive rats on the 5-CSRTT were found to have higher break points in a test of progressive ratio responding for cocaine, made more drug-seeking responses in extinction, and made more ‘compulsive’ responses when cocaine-infusions were paired with foot shock (Belin, Mar, Dalley, Robbins, & Everitt, 2008). Interestingly, in another study, rats that demonstrated these three addictive-like phenotypes were also shown to have long-lasting inhibition in the ability to elicit long-term depression (LTD) in the accumbens, while individuals that self-administered cocaine, but did not develop this “addicted” phenotype recovered the ability to elicit LTD (Kasanetz et al., 2010). Chronic cocaine self-administration experience has previously been shown to limit both LTP and LTD induction in the accumbens (Moussawi et al., 2009), and it is possible that certain individuals are more susceptible to this effect. High impulsive rats also reinstated drug seeking after punishment-induced abstinence, indicating an enhanced risk of relapse with this phenotype (Economidou, Pelloux, Robbins, Dalley, & Everitt, 2009).

Other studies of trait impulsivity, using delay-discounting, have determined that impulsiveness predicts escalation of cocaine self administration and cocaine-primed reinstatement (Anker, Perry, Gliddon, & Carroll, 2009; Perry, Nelson, & Carroll, 2008). Therefore, different personality traits may predict vulnerabilities to different aspects of addictive behavior. While impulsivity appears to predict an increased probability of developing many features of addiction, including habitual and/or compulsive behavior. It is still not clear how this personality trait affects cue-related learning and memory. In addition, the effects of these individual differences on responding for a variety of addictive drugs still needs to be assessed, as most studies have focused on psychostimulants. Indeed, a recent report indicates that impulsivity does not predict heroin self-administration, escalation of intake, or reinstatement, indicating that different traits may differentially enhance risk for developing an addiction to specific drugs of abuse (McNamara, Dalley, Robbins, Everitt, & Belin, 2010). Identifying these risk factors in humans may allow targeted therapies aimed at preventing initial sampling of drugs (e.g., HR-sign trackers) and treatment of compulsive users (e.g., High-Impulsive).

In addition, several other developmental and environmental factors may influence the effects of addictive drugs on learning and memory systems. These include, but are not limited to, prenatal stress or drug exposure, post-natal to adolescent stress, parental behavior, and environmental enrichment. All of these are important factors to understand the complicated individual effects of drugs on the brain that potentially affect learning and memory processes relevant to addiction. When a complete picture is developed of how all of these factors interact to produce increased risk or resilience to developing addiction, it may be possible to prevent the cycle of addiction before it starts.

Clinical Evidence for Aberrant Learning and Memory in Addiction

Despite the wealth of pre-clinical data favoring an aberrant learning and memory conception of addiction, there is a lack of clinical research exploring exactly how learning accrues to drug predicting stimuli in human subjects. This is perhaps a result of ethical concerns with drug administration to human subjects or technical difficulties; it is difficult to know exactly what might serve as a drug-predicting cue to a human addict. There are numerous possibilities ranging from contextual cues associated with access to drug (for example the places in which drugs were procured or consumed), the sight and smell of drugs and drug paraphernalia, or even the acquaintances with who drugs have previously been consumed. A study of smokers who recorded the place, situation, and feeling experienced during smoking, revealed that different contexts were associated with differing degrees of craving and enjoyment. Furthermore, these differences could be predicted by dependence; unsurprisingly heavier smokers smoked in more contexts (Van Gucht, Van den Bergh, Beckers, & Vansteenwegen, 2010).

But how does this learning take place? Thus far, with a few notable exceptions (Foltin & Haney 2000; Hogarth, Dickinson, & Duka, 2003; Hogarth, Mogg, Bradley, Duka, & Dickinson, 2003; Hogarth, Dickinson, Hutton, Bamborough, & Duka, 2006; Yoder et al., 2009), clinical research into cue processing in addiction has focused on exposing drug users to generic cues associated with drug use such as stimuli from the International Affective Pictures System (Lang, Bradley, & Cuthbert, 2008) or dramatized video representations of cocaine consumption (Volkow et al., 2006). While these cues have the advantage of being used extensively across numerous studies, they presumably have conditioning histories that are unique to each subject (a cocaine user may not insuflate their cocaine from a mirror, as depicted in a cocaine related stimulus for example). Nevertheless, stimuli related to the abused substance evoke neural activation in the striatum of smokers (David et al., 2005), alcoholics (Grusser et al., 2004) and cocaine users (Garavan et al., 2000), exactly as one would predict from the preclinical literature and from an aberrant learning and memory model of addiction (note however that one should be careful of making reverse inferences with functional imaging data (Poldrack, 2006); merely observing striatal activation in a subject does not mean that the subject was necessarily experiencing reward, motivation or engaging in learning).

Positron emission tomography (PET) scans with radiolabelled tracers sensitive to dopamine binding reveal dopamine release in the dorsal striatum in cocaine users in response to cocaine associated cues (Volkow et al., 2006). Alcoholics too release dopamine in the striatum in response to alcohol cues, particularly in situations when alcohol was expected but not delivered (Yoder et al., 2009). Such expectancy violations or prediction errors may provide a strong impetus to drug seeking (Lapish, Seamans, & Chandler, 2006) and may drive new learning of Pavlovian cue-drug associations and the instrumental actions involved in drug procurement (Redish, 2004). In the following we will discuss some clinical evidence in favor of the importance of learning processes for addiction.

Learning to predict important environmental events and behave adaptively towards natural rewards, aversive experiences, and other salient shifts in environmental contingencies is highly conserved across phylogeny. Single celled organisms show anticipatory predictive learning (Mitchell et al., 2009), Pavlov's dogs salivated in response to events that predicted food delivery (Pavlov, 1960), and humans acquire causal (Corlett et al., 2004) and social (Behrens, Hunt, Woolrich, & Rushworth, 2008) relationships in the same way. Such predictive associations are formed by minimizing uncertainty, that is, by reducing prediction errors (Rescorla, 1972). Drugs and drug-predicting stimuli appear to usurp the systems responsible for such adaptive anticipation, driving behavior toward anticipation and consumption of drug rewards and biasing memory consolidation toward associations between cues, drugs and the actions that procure them (Hyman, Malenka, & Nestler, 2006), perhaps by spuriously elevating reward prediction error signals (Redish, 2004; Lapish, Seamans, & Chandler, 2006) in the midbrain, striatum, and frontal cortex (Schultz & Dickinson, 2000). These aberrant errors may drive addiction-related plasticity toward a particular behavioral repertoire, but once strongly consolidated this learning becomes insensitive to updating and is maintained despite large discrepancies between expected and experienced reinforcement (Takahashi, Schoenbaum, & Niv, 2008).

During the earliest phases of drug use, individuals appear to show sensitized responses to all salient events but with repeated drug use, and hence conditioning, behavioral repertoires become more and more focused on the procurement and consumption of drug rewards (Anselme, 2009). This conditioned attentional orienting response (CAOR) is elicited by initial sensory processing of the cues and engenders a shift in the focus of attention towards the spatial location of drug-predicting cues. It is speculated further that the CAOR gates other drug-associated conditioned responses (Carter & Tiffany, 1999) and Pavlovian-to-instrumental-transfer (PIT) – the energizing effect of drug predicting cues on drug seeking behavior (Rescorla & Solomon, 1967; Lovibond, 1983), which support and constitute drug-seeking and drug-taking.

In a series of studies with smokers, Hogarth, Dickinson and Duka have established the utility of eye tracking as an outcome measure in studies of predictive learning in addiction (Hogarth, Dickinson, & Duka, 2003; Hogarth, Mogg, Bradley, Duka, & Dickinson, 2003; Hogarth, Dickinson, Hutton, Bamborough, & Duka, 2006). They trained a set of novel arbitrary visual cues as discriminative stimuli, that is, subjects were trained that instrumental key-press responses in the presence of these stimuli were always rewarded with smoke delivery. In a subsequent test phase, they found that subjects demonstrated preferential attention to those stimuli relative to cues that did not predict smoke delivery. There are competing theories of the role of attentional allocation in conditioning and suggestions from Hogarth and colleagues that attentional biases may differ depending on the stage of drug use and abuse (Hogarth, Dickinson, & Duka, 2003; Hogarth, Mogg, Bradley, Duka, & Dickinson, 2003; Hogarth, Dickinson, Hutton, Bamborough, & Duka, 2006). In brief, some learning theories suggest that we learn about and attend to unpredictable events (Pearce & Hall, 1980), whereas others suggest that we focus on the predictable, such that consistent predictors of reinforcement should engage our attention, anticipatory behaviors, and learning (Mackintosh, 1975). There is evidence for both mechanisms (Grossberg, 1982). Healthy subjects learning about abstract rewards focus their attention on unpredictable contingencies; whereas, consistent predictors of smoke delivery capture attention in moderate but not heavy smokers. Further investigations are necessary to elucidate the dynamics of attentional capture by drug cues.

One important learning and memory dimension that can be explored in humans and may be especially informative with respect to addiction is awareness. Humans can learn implicitly (below the level of conscious awareness) and explicitly (in a verbalizable, declarative sense) (Shanks & St. John, 1994). The clinical data suggest that both of these processes are important for the generation and maintenance of addictive behaviors. For example, in the studies of eyetracking in smokers, subjects need to be aware of the predictive relationships between the cues and smoke delivery to show attentional biases toward the salient smoke predicting cues (Hogarth, Dickinson, Hutton, Bamborough, & Duka, 2006). On the other hand, using functional neuroimaging and masked stimulus presentation, Childress and colleagues demonstrated that cocaine related stimuli presented below the threshold of awareness could still engage frontostriatal circuitry (Childress et al., 2008). Individual differences in that engagement predicted subsequent drug craving. This implicit/explicit distinction may map onto a crucial distinction in learning and memory systems, like that between goal-directed and habitual behaviors.

Folk Psychology conceives of drug use as a bad habit, something in which addicts indulge despite an expressed desire not to. Pre-clinical behavioral neuroscience supports this conception of addiction (Robbins & Everitt, 1999). As described above, ascertaining whether a behavior is goal-directed or habitual involves a simple yet elegant procedure (Adams, 1981) – if behavior is governed by the incentive value of a reward then, devaluing that reward (by, for example, pre-feeding to satiation or associating reward consumption with gastric illness) will decrease the vigor with which an organism engages in reward seeking behaviors. However, if behavior is habitual, it is no longer governed by a representation of reward value. Instead it involves reflexive responses to environmental cues previously associated with drug use, hence devaluing the drug reward has no effect on behavioral choice or vigor and the organism continues to engage in reward seeking. Although much of this work has been conducted in rodents, recent reports have confirmed similar behavioral (Schwabe & Wolf, 2009) and neural (Tricomi, Balleine, & O'Doherty, 2009) processes are involved in human habit learning. That is, with overtraining, devaluing a particular food will not prevent human subjects from engaging in the instrumental action that procures that food (Schwabe & Wolf, 2009), and such behavior is associated with activation in the dorsal caudate and putamen (Tricomi, Balleine, & O'Doherty, 2009), the regions aligned with habitual responding in laboratory animals (see Balleine & O'Doherty, 2010 for review).

To date, no studies have examined the neural mechanisms of human reinforcement learning using drugs as the reinforcers. However, studies using monetary rewards have revealed neural differences between addicted individuals and non-substance abusing controls. These studies indicate a more global dysfunction in learning and memory suggestive of rapid habit learning and an insensitivity of actions to their consequences. For example, in a financial investment game, smoker's brains compute a fictive error signal (indicating what they would have won had they chosen an alternative option, (Chiu, Lohrenz, & Montague, 2008)). However, they do not use that signal to guide their choices in the same way non-smokers do (Chiu, Lohrenz, & Montague, 2008). Likewise, alcoholics learning action choices generate striatal prediction errors, but those errors appear to not be communicated to the frontal cortex effectively, such that they are not used to update action choices (Park et al., 2010). Interestingly, in this study, the degree of frontostriatal dysconnectivity predicted current alcohol craving in the problem-drinking group. Of course, these effects with monetary rewards may indicate a relative lack of salience of monetary reinforcers when compared with drugs, but, nevertheless, they implicate learning and memory dysfunction in the aetiopathology of addiction. Furthermore, these studies suggest an insensitivity to the consequences of actions in addicts that resonates with the theoretical conception of habitual responding.

In summary, clinical data increasingly implicate learning and memory processes in addiction. Some learning and memory-based treatments for addiction have emerged. Psychotherapies based upon cue reactivity involve repeatedly exposing a client to the sights, smells or paraphernalia associated with a substance in the absence of actual consumption, thus attenuating the power of those cues to elicit an expectation of drug effects. In other words, they no longer readily predict a ‘high’ is coming so the cue-drug associations are extinguished (Otto, Basden, Leyro, McHugh, & Hofmann, 2007). Such therapies are often effective in the therapeutic setting but they can fail to transfer to the real world outside of the clinic (Otto, Basden, Leyro, McHugh, & Hofmann, 2007), that is, they are context dependent (Bouton, 2002; Otto, Basden, Leyro, McHugh, & Hofmann, 2007; Taylor, Olausson, Quinn, & Torregrossa, 2009). In the following section, we will discuss opportunities for targeted therapeutic approaches inspired by the learning and memory processes involved in addiction.

Developing Novel Treatments for Addiction Through Manipulations of Learning and Memory

A preponderance of evidence suggests that the development, maintenance, and persistence of addictive behaviors, even after periods of abstinence, is dependent on the strength of drug-associated cues to motivate behavior. Therefore, research in recent years has focused on ways to reduce the ability of cues to drive drug-seeking behavior. In preclinical models, researchers are primarily examining the ability of treatments to prevent reinstatement, which is an animal model of clinical relapse that can be driven by stress, re-exposure to the drug, or by drug cues (Shaham, Shalev, Lu, De Wit, & Stewart, 2003). In our discussion we will focus on cue-induced craving and relapse. In these studies, rats or mice have a neutral stimulus or context associated with self-administered or experimenter administered drug. Manipulations of the cue or context are then conducted with the goal of reducing responding or preference for the cue or context when tested in the future. Two types of learning and memory manipulations have primarily been studied as a means of reducing reinstatement – extinction and reconsolidation.

Extinction involves learning a new lack of association between a cue and an outcome. In other words, if a cue, context, or action that was previously associated with drug is repeatedly encountered or performed in the absence of drug, the response or behavior associated with that cue will diminish over time. Theoretically, after sufficient extinction training, re-exposure to the drug-paired cue should no longer elicit drug-seeking behaviors. Unfortunately, extinguishing cue memories has been ineffective in reducing relapse in both humans (Conklin and Tiffany, 2002) and rats (Crombag and Shaham, 2002), illuminating the need for alternative strategies or extinction ‘supplements’ (behavioral and/or pharmacological). The lack of effectiveness of extinction therapies is likely due to several factors that specifically apply to extinction learning. First, extinction is not permanent. Extinguished behaviors are known to reappear simply after the passage of time, a phenomenon known as ‘spontaneous recovery’ (Bouton, Westbrook, Corcoran, & Maren, 2006). Second, extinction learning is very context-dependent, such that extinction training will reduce responding in the environment where it is conducted, but behavior will be unaffected when tested in another environment, a phenomenon known as ‘renewal’ (Bouton, & King, 1983; Bouton, Westbrook, Corcoran, & Maren, 2006). Therefore, when extinction of a drug-associated cue occurs in a treatment facility the conditioned responses to the cue (e.g., increased heart rate; craving) may be reduced, but it is unlikely that these effects will transfer to the drug-taking environment. In animal models, extensive extinction training in a non-drug taking context of both the instrumental response and either a discriminative stimulus (SD) or discrete cue (CS+) associated with drug, does not significantly reduce renewal of drug-seeking (Crombag, & Shaham, 2002; Crombag, Bossert, Koya, & Shaham, 2008; Crombag, Grimm, & Shaham, 2002; Kearns, & Weiss, 2007). Finally, an extinguished behavior readily re-emerges or ‘reinstates’ after exposure to an associated cue (that was not extinguished), stress, or the unconditioned stimulus (drug), which is the basis for using the reinstatement model to test for relapse-like behavior (Shaham, Shalev, Lu, De Wit, & Stewart, 2003).

A first step in finding ways to augment extinction therapy is to first identify pharmacological agents or behavioral procedures that enhance extinction learning and prevent some forms of reinstatement. Several groups have been investigating the ability of a partial agonist at the glycine site of the NMDA receptor, D-cycloserine (DCS), to enhance extinction of drug-associated behaviors. DCS has been studied extensively because it is known to enhance several forms of learning and memory and LTP through its ability to facilitate NMDA receptor signaling (Billard, & Rouaud, 2007; Ressler et al., 2004; Ledgerwood, Richardson, & Cranney, 2004; Paolone, Botreau, & Stewart, 2008; Botreau, Paolone, & Stewart, 2006; Thanos, Bermeo, Wang, & Volkow, 2009). DCS has been shown to facilitate extinction of a cocaine conditioned place preference. The effects of DCS were long-lasting in both rats and mice (at certain doses), indicating a prevention of spontaneous recovery, and DCS treated rats were resistant to cocaine-primed reinstatement (Thanos, Bermeo, Wang, & Volkow, 2009; Paolone, Botreau, & Stewart, 2008). In addition, DCS has been shown to enhance extinction of instrumental responding for cocaine in rats, and prevent the reacquisition of cocaine self-administration in rats and monkeys (Nic Dhonnchadha et al., 2010). Likewise, the more efficacious agonist at the glycine site of the NMDA receptor, D-serine, can also augment extinction learning to prevent cocaine-primed reinstatement in rats (Kelamangalath, Seymour, & Wagner, 2009; Kelamangalath, & Wagner, 2010). Finally, DCS also facilitates extinction of alcohol self-administration and attenuates alcohol-primed reinstatement (Vengeliene, Kiefer, & Spanagel, 2008). All of these results suggest that DCS can inhibit spontaneous recovery and reinstatement, indicating that it has great potential as an adjunct pharmacotherapy to extinction training.

Notably, none of these studies examined the ability of DCS to reduce the context-specificity of extinction. In order to address this issue, we determined whether DCS administered after extinction of cocaine-paired cues in a novel environment could reduce cue-induced reinstatement when the animals were tested back in the original self-administration context. We found that DCS did cause a reduction in reinstatement, even though extinction occurred in a different environment, indicating that DCS can also inhibit renewal (Torregrossa, Sanchez, & Taylor, 2010). Therefore, DCS has many of the properties necessary to be a successful treatment for addiction when given with behavioral extinction therapy. However, one caveat is that DCS administered directly to the amygdala after 30 cocaine cue presentations has been shown to increase cue-induced reinstatement, indicating a possible enhancement of memory reconsolidation (Lee, Gardner, Butler, & Everitt, 2009), which suggests that better treatments should be identified that do not unintentionally increase the likelihood of relapse.

Unfortunately, very few other drugs have been tested for their ability to facilitate extinction and prevent reinstatement, spontaneous recovery, or renewal. However, another glutamatergic modulator, N-acetylcysteine, which reverses drug-induced reductions in basal glutamate in the nucleus accumbens, also facilitates extinction of heroin seeking and prevents reinstatement even after long periods of abstinence (Zhou, & Kalivas, 2008). In addition, AMPA receptor potentiation or beta-adrenergic receptor activation in the infralimbic prefrontal cortex can enhance the retention of extinction of cocaine seeking. However, the ability of systemic administration of these compounds to prevent reinstatement, spontaneous recovery, and renewal has yet to be tested (LaLumiere, Niehoff, & Kalivas, 2010). Overall, drugs acting at a variety of neurotransmitter systems known to enhance other forms of learning and memory, including cholinergic, cannabinoid, and monoaminergic, should be evaluated for their ability to facilitate extinction of drug-associated memories and prevent all forms of relapse-like behaviors. Due to the similarities between neural circuits mediating both extinction of fear and appetitive responses, the addiction field will benefit from testing compounds that have already been identified as effective in the fear extinction field (Peters, Kalivas, & Quirk, 2009).

The second strategy for reducing the motivational impact of drug cues is to disrupt memory reconsolidation. Reconsolidation is the process whereby a memory is re-stored into long-term memory after it has been reactivated by a ‘reminder’ event (Nader, & Hardt, 2009; Tronson, & Taylor, 2007). Some investigators have hypothesized that the function of reconsolidation is to strengthen or update memories (Lee, 2009; Lee, 2008). When a memory is reactivated it becomes labile and subject to disruption by a variety manipulations, thus reducing the strength, or possibly the existence, of the memory (Nader, & Einarsson, 2010). Recently, disruption of fear memory reconsolidation has received much attention, both in pre-clinical and clinical research, as a means of treating anxiety disorders such as post-traumatic stress disorder (PTSD) and phobias (Debiec, & LeDoux, 2006; McCleery, & Harvey, 2004; Tronson, & Taylor, 2007). Even more recently we argued it should be investigated as a means of preventing relapse in drug addiction (Taylor, Olausson, Quinn, & Torregrossa, 2009). Several preclinical studies have identified neurochemical and molecular processes that are essential for reconsolidation of both fear and drug-associated memories. We will focus next on what has been found relating to drug addiction and discuss potential avenues for future treatment development.

Reconsolidation of almost all forms of memory has been shown to require protein synthesis, gene transcription, and NMDA receptor activation, and drug-associated memories are no exception (Milekic, Brown, Castellini, & Alberini, 2006; Fuchs, Bell, Ramirez, Eaddy, & Su, 2009; Lee, Di Ciano, Thomas, & Everitt, 2005; Milton, Lee, Butler, Gardner, & Everitt, 2008; Brown, Lee, & Sorg, 2008; Itzhak, 2008; von der Goltz et al., 2009). In addition, reconsolidation of a cocaine or morphine conditioned place preference can be disrupted by systemic (Valjent, Corbillé, Bertran-Gonzalez, Hervé, & Girault, 2006) or intra-accumbens (cocaine only; (Miller, & Marshall, 2005)) inhibition of ERK, resulting in a loss of preference. Conditioned place preference has also been used to identify several other manipulations that can interfere with drug memory reconsolidation, including muscarinic acetylcholine and NMDA receptors (Kelley, Anderson, & Itzhak, 2007; Sakurai, Yu, & Tan, 2007; Zhai et al., 2008; Sadler, Herzig, & Schmidt, 2007), beta-adrenergic receptors (Robinson, & Franklin, 2010; Fricks-Gleason, & Marshall, 2008); glucocorticoid receptors (Wang, Zhao, Ghitza, Li, & Lu, 2008), calcium/calmodulin-dependent protein kinase II (CaMKII; (Sakurai, Yu, & Tan, 2007)), matrix metalloproteinases (Brown et al., 2007), and neuronal nitric oxide synthase (Itzhak, & Anderson, 2007).

Comparatively, fewer studies have determined mechanisms for disrupting reconsolidation of memories associated with self-administered drugs. However, Everitt and colleagues have found that disrupting activity of the plasticity related gene zif268 by antisense oligonucleotide infusion into the basolateral amygdala (Lee, Di Ciano, Thomas, & Everitt, 2005), antagonizing NMDA receptor activity systemically or in the amygdala (Milton, Lee, Butler, Gardner, & Everitt, 2008), or beta adrenergic antagonism with propranolol (Milton, Lee, & Everitt, 2008), all inhibit later responding for conditioned reinforcement. These studies provide further evidence that transcription of certain genes and NMDA receptor activity, particularly in the basolateral amygdala are critical for drug memory reconsolidation. We expanded upon these findings and investigated whether activation of PKA activity in the BLA was also necessary for cocaine-cue memory reconsolidation, as was found for conditioned fear (Tronson, Wiseman, Olausson, & Taylor, 2006). Rats were trained to self-administer cocaine associated with a tone+light cue, lever pressing was extinguished over several days, and then the cue was selectively reactivated by 3 non-contingent presentations in a novel environment followed by BLA infusion of the PKA inhibitor Rp-cAMPs. The rats were then tested for their propensity to reinstate lever pressing in the drug-taking context or respond for conditioned reinforcement in a third context. Rats receiving the PKA inhibitor immediately following reactivation, but not 3 hours later, or in the absence of memory reactivation, showed reduced cue-induced reinstatement and responding for conditioned reinforcement (Sanchez, Quinn, Torregrossa, & Taylor, 2010). Interestingly, Milton et al., (2008) did not see an effect on NMDAR antagonism unless the antagonist was given prior to memory reactivation, indicating that either the NMDAR-dependent phase of reconsolidation is very short, or that NMDAR antagonism interacts with memory retrieval to inhibit drug-seeking. The ability of post-retrieval PKA inhibition to block later cue motivated behavior strongly suggests that reconsolidation was selectively disrupted. In addition, disruption of reconsolidation by inhibiting PKA in a novel (treatment center-like) context was successful in attenuating cue motivated behavior in the original drug-taking context and in a third context, indicating that treatments targeting reconsolidation processes can avoid the renewal effect of extinction therapy. Notably, Rp-cAMPs treatment had no effect on cocaine-primed reinstatement, indicating that while we selectively inhibited reconsolidation of the cue memory, successful treatment of addiction may require all drug-related memories to be disrupted.

Finally, there are some promising results in clinical studies indicating that reconsolidation of fearful memories can be disrupted to later reduce fear (de Quervain, & Margraf, 2008; Kindt, Soeter, & Vervliet, 2009), indicating that disruption of reconsolidation is a feasible human treatment strategy. However, the mechanisms by which drug-cue memories are reconsolidated still need to be elucidated, and this treatment strategy remains to be explicitly tested in human drug addicts. Future studies may also use combinations of manipulations of extinction and reconsolidation to produce a more effective reduction in drug craving and relapse, as has been shown for conditioned fear (Schiller et al., 2010; Monfils, Cowansage, Klann, & LeDoux, 2009). New treatment possibilities may also develop from our continued understanding of the neurobiology of habit formation and how drugs of abuse cause long-lasting alterations in habit circuits. As of today, few studies have investigated what factors might prevent the expression of a habit, but our laboratory has found that dopaminergic activation of the medial prefrontal cortex can restore goal-directed behavior for food (Hitchcott, Quinn, & Taylor, 2007). Continued research in this exciting field is likely to produce many new possibilities for the treatment of addiction and other psychiatric disorders.

Summary

In conclusion, the past decade of drug addiction research has established that drugs of abuse and normal learning and memory processes produce similar molecular and neurochemical effects, suggesting that the development of addiction could be due in part to aberrant learning and memory about abused drugs. Drugs of abuse enhance learning about environmental cues associated with reinforcing events and may cause the rapid formation of habitual and ultimately compulsive behavior related to obtaining and using drugs. Addictive drugs also cause neuroplasticity in circuits involved in cue learning and habit formation, including changes in cAMP/PKA/CREB, DeltaFosB/Cdk5, and BDNF/ERK, among others. In addition, a growing body of literature is identifying biological and environmental factors that cause individuals to be particularly vulnerable to the effects of drugs of abuse on learning and memory systems, including age of first exposure, hormones, stress, and innate personality traits. Further research in this area will hopefully allow early identification of at risk individuals to prevent addictions from developing in the first place. For those that are already mired in the disease, manipulations of drug-associated memories may provide a novel mechanism for establishing and maintaining abstinence by reducing the ability of environmental cues to induce craving and relapse. Novel therapeutics targeting memory extinction and reconsolidation processes may produce a “clean slate”, where the learned desire to use drugs is greatly reduced. Likewise, continued research into the neural mechanisms underlying the formation of habits and compulsions may result in treatments that improve cognitive control over behavior. While many of the studies described in this review have been conducted pre-clinically, there are several studies of clinical populations that support the notion that addictive drugs alter learning and memory related to cues, especially those associated with the drug itself. Vulnerability factors associated with developing addiction have also been identified. To date, extinction therapies have not been very successful in the treatment of addiction clinically, but few adjunct pharmacotherapies have been tested (see Price et al., 2009 for one example). Moreover, clinical research into the ability of certain therapies to interfere with reconsolidation of drug-associated cues is needed to determine if this is a viable treatment strategy. These studies are just beginning, making this an exciting time to study the relationship between addiction and aberrant learning and memory.