Abstract
The purpose of this review is to describe the molecular mechanisms in the striatum that mediate reward-based learning and action control during instrumental conditioning. Experiments assessing the neural bases of instrumental conditioning have uncovered functional circuits in the striatum, including dorsal and ventral striatal sub-regions, involved in action-outcome learning, stimulus-response learning, and the motivational control of action by reward-associated cues. Integration of dopamine (DA) and glutamate neurotransmission within these striatal sub-regions is hypothesized to enable learning and action control through its role in shaping synaptic plasticity and cellular excitability. The extracellular signal regulated kinase (ERK) appears to be particularly important for reward-based learning and action control due to its sensitivity to combined DA and glutamate receptor activation and its involvement in a range of cellular functions. ERK activation in striatal neurons is proposed to have a dual role in both the learning and performance factors that contribute to instrumental conditioning through its regulation of plasticity-related transcription factors and its modulation of intrinsic cellular excitability. Furthermore, perturbation of ERK activation by drugs of abuse may give rise to behavioral disorders such as addiction.
1. Introduction
The basal ganglia are a network of subcortical nuclei that have long been known to exert a strong coordinating influence on the motor system, particularly on the feed-forward and feedback processes involved in action initiation and execution. More recent research has, however, significantly broadened our understanding of basal ganglia anatomy and the functions with which it is concerned. Although there have been many attempts to paint it as a network serving a single homogeneous function, recent evidence suggests that the basal ganglia are involved in heterogeneous motor functions ranging from simple reflexive, sensorimotor habits to deliberated, goal-directed actions. The latter function is of particular interest given recent advances in our understanding of the nature of the learning and motivational processes involved in the control of actions, in choice between actions and in decision-making more generally.
Much of this recent work has focused on the striatum, which is the largest of the basal ganglia nuclei and serves as the entry point for cortical and thalamic inputs into basal ganglia circuitry. Human and primate research has found evidence of striatal involvement in the learning and performance processes engaged by reward-related actions during decision-making tasks and disorders that affect the striatum, such as Parkinson’s disease, Huntington’s disease and substance abuse, produce impairments in these kinds of action (Balleine et al., 2007; Balleine and O'Doherty, 2010; Bechara et al., 2002; Cohen and Frank, 2009; Delgado, 2007; Hikosaka, 2007). Parallel research in rodents using paradigms derived from instrumental conditioning has identified sub-regions within the striatum that mediate distinct forms of action control based on an assessment of the learning and performance processes (Balleine et al., 2008, 2009; Balleine and Ostlund, 2007; Yin et al., 2008). In similar fashion to the primate research, rodent research has identified a network involving the dorsomedial striatum mediating the role of reward-related learning in the cognitive control of action and that has been dissociated anatomically and functionally from a more lateral network engaged during the learning and performance of habits.
How the striatum controls the influence of reward learning on the cognitive control of action selection and initiation at a cellular and molecular level is an area of active research. The primary inputs to striatum are excitatory projections from diverse regions of cortex and thalamus, and synaptic plasticity at corticostriatal synapses is thought to be critical for these aspects of action control (Costa, 2007; Di Filippo et al., 2009; Horvitz, 2002; Horvitz, 2009; Wickens et al., 2007). The striatum also receives dopaminergic input from the midbrain, and indeed, models of reward learning and action control emphasize the importance of dopamine signals in learning and decision making (Dayan and Balleine, 2002; Redgrave et al., 2008; Salamone and Correa, 2002; Schultz, 2007; Schultz and Dickinson, 2000). Dopamine and glutamate signaling interact through multiple cellular mechanisms to shape neural excitability and synaptic plasticity in striatal neurons (Di Filippo et al., 2009; Kreitzer, 2009; Lovinger, 2010; Surmeier et al., 2007). An important question for current research is to understand the role that these diverse cellular signaling mechanisms play in striatal-based reward learning and action control. In order to address this topic, it is important to have a framework for understanding the cognitive processes mediated by the striatum. Here we use concepts derived from instrumental conditioning in rodents as a model for examining the molecular mechanisms that underlie striatal function. We suggest that signaling molecules that are sensitive to combined dopamine and glutamate receptor signaling have a key role in learning and action control in the striatum and that alteration in the activity of these signaling molecules may underlie many of the abnormalities in these behavioral processes induced by genetic conditions, neurodegeneration and addiction.
2. Actions, habits and the striatum
As the main input nucleus of the basal ganglia, the striatum receives excitatory glutamatergic afferents from cortical, limbic and thalamic regions, as well as heavy dopaminergic input from the midbrain (Sesack et al., 2003; Smith et al., 1998). Traditionally, the striatum has been divided into dorsal and ventral subdivisions. The ventral subdivision contains the nucleus accumbens (NAc), which itself consists of core and shell sub-regions (Zahm, 2000; Zahm and Brog, 1992). The glutamatergic inputs to striatum are topographically organized, with limbic and ventral prefrontal regions projecting to the ventral striatum, sensorimotor cortical regions projecting to the dorsolateral striatum (DLS) and association areas of the prefrontal cortex projecting to the dorsomedial striatum (DMS) (Alexander et al., 1986; Groenewegen et al., 1990) – see Figure 1A. This pattern of connectivity has lead to the idea that cortico-basal-ganglia loops are organized into functional circuits that mediate distinct components of behavior (Alexander et al., 1990; Joel and Weiner, 1994; Pennartz et al., 2009; Voorn et al., 2004). Indeed, early studies of maze task performance in animals concluded that the dorsal striatum was necessary for implementing a “response” strategy, defined in terms of a reliance on egocentric information (e.g., turn left) or simple stimulus-response associations to encode goal locations (see (Packard and Knowlton, 2002; White, 2009; Yin and Knowlton, 2006) for review). In contrast, the ventral striatum, specifically the nucleus accumbens, was described as having a role in the expression of conditioned emotional responses to cues and contexts associated with appetitive (or aversive) events, such as access to mates, drugs of abuse, and food or liquid rewards (see Belin et al., 2009; Berridge, 2009; Cardinal et al., 2002; Day and Carelli, 2007 for review). Furthermore, behavior mediated by the striatum in these tasks were found to depend on glutamate and dopamine signaling (Burns et al., 1994; Cory-Slechta et al., 1999; Di Ciano et al., 2001; Packard, 1999; Packard and Teather, 1997; Packard and White, 1991; Smith-Roe and Kelley, 2000) suggesting that combined activation of these neurotransmitter systems in the striatum is necessary for normal striatal function.
Figure 1.
A. Corticostriatal circuits involved in decision making. The learning processes controlling the acquisition of reward-related actions are mediated by converging projections from regions of dorsomedial prefrontal cortex (dMPC) to the rodent dorsomedial striatum (DM), whereas (2) the processes mediating the acquisition of stimulus-bound actions, or habits, are thought to be mediated by projections from sensorimotor cortex (SM) to the rodent dorsolateral striatum (DL). Reward and predictors of reward are the major motivational influences on the performance of goal-directed and habitual actions and are thought to be mediated by corticostriatal circuits involving, particularly, ventral MPC and amygdala inputs (not shown) to ventral striatum (VS). These corticostriatal connections are parts of distinct feedback loops that project back to their cortical origins via ventral tegementum/substantial nigra/globus pallidus (SNr/Gpi) (blue arrows) and the mediodorsal (MD)/posterior (PO) nuclei of the thalamus and project out to premotor and motor cortices through the globus pallidus and ventral pallidum (red arrows). Dopamine (green arrows) is an important modulator of plasticity in the dorsal striatum whereas its tonic release has long been associated with the motivational processes mediated by the ventral circuit. B. The major functional divisions of the striatum. The regions shown are anatomically continuous but, recent findings suggest, functionally heterogeneous. With regard to instrumental conditioning the dorsal and ventral striatum are broadly distinguished by their involvement in learning and performance respectively; the dorsal region in the acquisition of goal-directed actions (dorsomedial region), heterogeneous chains of actions (dorsocentral region) and habits (dorsolateral region); the ventral striatum in the motivation of these actions by experienced reward (the nucleus accumbens core) and predicted reward (the nucleus accumbens shell).
More recently, the application of instrumental procedures has more precisely elucidated the various action-related functions mediated by the striatum – see Figure 1B. One of the most important aspects of this research has been the repeated demonstration that, contrary to appearance, instrumental actions, such as lever pressing, can be controlled at different times by fundamentally distinct learning and motivational processes. In particular, various instrumental procedures and tests have been used to establish whether and when an animal’s actions are deliberated or goal-directed and when they are elicited, automatic or habitual. Generally, actions are considered goal directed if it can be demonstrated both that an animal has encoded the outcome resulting from its actions and uses that information to select among potential actions. More specifically, Balleine & Dickinson (1998) argued, based on human action theory, that for any action to be labeled goal-directed it must be shown to satisfy two criteria; referred to as the goal and the contingency criteria. To satisfy the goal-criterion, performance of an action must be shown to be sensitive to changes in the encoded value of the outcome, whereas to satisfy the contingency criterion an action must be shown to depend on the contingent or causal relationship between the action its specific consequences. If the performance of an action persists in a situation where either its causal relationship to the outcome or the value of the outcome declines then it is difficult to claim it is truly goal-directed.
Using these criteria, considerable evidence has accumulated suggesting that, in rodents, actions acquired in instrumental conditioning can be goal-directed (Dickinson and Balleine, 1994, 2002). In these experiments, rats learn to perform different actions (e.g., responses on a left or right lever) for different types of outcomes (e.g., grain pellet or sucrose solution). To assess the goal criterion, one of the two outcomes is then devalued, either by feeding the outcome to satiety or pairing its consumption with illness. In a subsequent choice test, conducted in extinction (i.e. in the absence of any feedback), rats have consistently been found to show a preference for the lever that previously delivered the still valued outcome over the lever that delivered the now devalued outcome (Adams and Dickinson, 1981; Colwill and Rescorla, 1985). In addition to changes in outcome value, a goal-directed action should be sensitive to changes in the causal relationship between an action and its consequences or outcome and, again, considerable evidence suggests that rodents are highly sensitive to changes in this relationship. Thus, when a previously action-dependent outcome is delivered in a manner that is independent of that action, rodents reduce their performance of that action, an effect referred to as contingency degradation (Balleine and Dickinson, 1998; Hammond, 1980). Generally, therefore, the changes in behavior induced by contingency degradation and outcome devaluation show that rats encode information about the relationship between their actions and specific outcomes, and use these action-outcome associations, along with information about the desirability of a particular outcome - provided by the animal’s motivational system - to guide decisions about actions (Dickinson and Balleine, 2002).
Although instrumental actions can, from observation, appear to be goal-directed, they are often not; indeed, although action execution can be controlled or deliberated, often it is automatic or reflexive, under the control of internal or external states and stimuli. This sensorimotor process has been argued to form the core of a distinct functional capacity involving the formation of habitual actions. Habits are, therefore, revealed particularly in their persistence, even in the face of sometimes quite extreme negative consequences, and in their sensitivity to the motivational functions of reward-related cues (Dickinson, 1994). Recent papers have associated habit learning with various addictive behaviors (Cardinal and Everitt, 2004; Dickinson et al., 2002; Miles et al., 2003; Robbins and Everitt, 2002), suggestions that have their source in the classic theories of stimulus-response learning (Hull, 1943). From this perspective, rewarding events reinforce or strengthen associations between contiguous sensory and motor processes allowing the sensory process to elicit the motor response in a manner that is no longer regulated by its consequences. The determining features of habit learning are, therefore, quite distinct of those regulating goal-directed action, notably: (1) habitual actions have been found to be insensitive to post-training outcome devaluation; and (2) the suggestion that S-R learning is strengthened or reinforced whenever a response is rewarded under a particular stimulus - irrespective of the specific outcome delivered or other stimuli present – suggests that this learning is not regulated by the contingent relationship between action and outcome but by their contiguity.
In line with these proposed features of habits, there is considerable evidence that whenever changes in the rate of reward are constrained – e.g., at asymptote when actions are overtrained (Adams and Dickinson, 1981) or by time when trained on interval schedules of reinforcement (Holman, 1975) – instrumental performance becomes insensitive to changes in outcome value. Likewise, habitual actions have been found to be relatively insensitive to changes in the action-outcome contingency; Dickinson et al (1998) found that, when overtrained, the instrumental actions of rats are insensitive to the imposition of an omission contingency, an effect that has been replicated in mice (Frankland et al, 2004).
Importantly, a growing number of studies have found evidence that goal-directed and habitual actions rely on different regions of the striatum for their acquisition and performance, suggesting that these two distinct forms of action control depend on distinct cortical-basal ganglia networks. Beginning with work in our lab, we have found evidence that a posterior portion of the dorsomedial striatum (pDMS) in rodents together with inputs from the association, most notably prelimbic (PL), area of prefrontal cortex, is critical for the acquisition and performance of goal-directed actions (Balleine & Dickinson, 1998; Yin et al., 2005b); pretraining lesions of the PL or the pDMS render instrumental performance insensitive to outcome devaluation and contingency degradation. Unlike the prefrontal cortex, however, both pre- and post-training lesions of the pDMS have been found to render rats’ instrumental responding insensitive to these tests, indicating that this region is required not only for learning, but is also necessary for using these associations to direct performance.
In contrast to the pDMS, the dorsolateral striatum (DLS), which receives inputs from sensorimotor regions of frontal and pre-frontal cortex, is critical for both the learning and performance that mediates habitual responding. Although habits are normally insensitive to tests of goal-directed action control, the instrumental actions of rats with lesions of DLS remain sensitive to outcome devaluation and contingency degradation despite being extensively overtrained (Yin et al., 2004) indicating that DLS lesions prevented the establishment of habitual responding and preserved the goal-directed control of instrumental performance (cf. Graybiel, 2008 for review).
3. Pavlovian-instrumental interactions and the ventral striatum
In addition to goal-directed and habitual actions, animals learn through Pavlovian conditioning to associate stimuli with outcomes or other events and, as a result, develop conditioned reflexive responses to these stimulus presentations. Although they are considered separate forms of learning, Pavlovian and instrumental conditioning interact in multiple ways. For example, conditioned stimuli (CS’s) that have been associated with a particular outcome can themselves bias instrumental actions. Procedures have been developed for measuring the effect of CS’s on the choice and vigor of instrumental actions, a paradigm known as Pavlovian-instrumental transfer (PIT). When associated with an appetitive event, such as a specific food, Pavlovian CS’s can become predictors of both the sensory specific features of that food (e.g., specific taste, texture etc) and the general affective state induced by the food (Konorski, 1967). Importantly, evidence suggests that these two predictive relations can induce quite distinct kinds of effects on instrumental performance (cf. Corbit and Balleine, 2005). For example, in a choice situation, where two actions are associated with distinct instrumental outcomes, a CS associated with the same outcome as one of the two actions tends to bias choice towards that action. In paradigms of this kind, often referred to as outcome-selective PIT, rats are generally allowed to choose between actions that result in the same or a different outcome to that predicted by the CS and, in these tasks, typically perform more of the action trained with the outcome predicted by the CS than the other action.
In situations in which CS’s are presented that predict appetitive outcomes that are not made available by either available choice, performance tends to be controlled by the general appetitive significance of the outcome and, as a result, performance is generally elevated on all actions. This latter effect has been found to reflect the arousal-inducing properties of the CS gated by primary motivational state; e.g., CS’s associated with nutritive outcomes elevate responding when rats are hungry but not when they are thirsty - and when associated with fluid outcome when the rats are thirsty rather than hungry (Dickinson and Balleine, 2002). Outcome-specific PIT therefore requires that the rats use a representation of the outcome associated with the CS to direct instrumental action, whereas the general form of PIT requires that the cue elicit a state of arousal.
The ability of cues to modulate instrumental actions depends on the nucleus accumbens (NAc – refer Figure 1A and 1B), a region within the ventral striatum. Like the dorsal striatum, the NAc receives input from prefrontal cortex, as well as limbic association cortices such as the amygdala and hippocampus (Berendse et al., 1992; Brog et al., 1993; Groenewegen et al., 1999). The NAc itself consists of core and shell sub-regions. The core’s pattern of output projections is similar to the dorsal striatum, with inputs from cortex and thalamus and projections to the ventral pallidum and substantia nigra pars reticulata (SNr). The shell, in addition to the typical basal ganglia circuitry, contains projections to hypothalamic and brainstem motor regions, and has thus been considered part of the extended amygdala (Heimer et al., 1997). The NAc core and shell regions are critical for the ability of Pavlovian CS’s to influence action selection. In particular, lesions of the NAc shell render animal’s choice performance insensitive to CS presentation during outcome-specific PIT (Corbit et al., 2001 - refer Figure 1B). Animals still show a general arousing effect of cues on instrumental performance. In contrast, lesions of the NAc core make animals insensitive to the general arousing effect of CS presentations during PIT, without affecting outcome specific transfer (Corbit et al., 2001; Hall et al., 2001).
4. The emerging functional architecture of the basal ganglia
Although striatal sub-regions mediate specific functions, striatal function has to be considered within the context of cortico-basal ganglia circuitry. The vast majority of striatal neurons are medium spiny neurons (MSN’s), which are GABAergic projection neurons. MSN’s form the striatal output, which consists of striatonigral and striatopallidal pathways (see Figure 1A for review). Striatonigral neurons project to basal ganglia output structures, namely the substantia nigra pars reticulata (SNr) and the internal segment of the globus pallidus (GPi), and constitute the direct pathway by which information flows from striatum to basal ganglia output. Striatopallidal neurons project to the external segment of the globus pallidus (GPe), which itself projects to the SNr and GPi via the subthalamic nucleus. This relay constitutes the indirect pathway of information flow through the basal ganglia. Although the direct and indirect pathways form the main intrinsic connections between basal ganglia nuclei, they are not the only pathways; recent research has described several important connections including direct path collateral projections to the external globus pallidus (GPe) (Nadjar et al, 2006) which, in turn, not only projects to the STN but also the GPi and SNr and to the substantia nigra pas compacta (SNc) (Smith et al, 1998) (Figure 1A). Furthermore, although the basal ganglia form a predominantly feed forward network, there are substantial feedback pathways throughout the network including bidirectional connections between GPe and STN and between the GPe and dorsal striatum (Smith et al, 1998; Kita, 2007).
The topographical organization of basal ganglia connectivity has given rise to a functional anatomy based not only on its feed-forward connections but on a series of functionally segregated loops that feedback to the striatum via the thalamus directly and indirectly via the cortex (Alexander et al., 1986; Leblois et al., 2006; McHaffie et al., 2005; Smith et al., 2004). Although unique functions of direct and indirect thalamic feedback to the striatum have not yet been described, as elaborated above it is clear that, topographically speaking, distinct cortico-striatal thalamic loops mediate quite specific aspects of action control based on distinct learning and plasticity processes. These distinct functions in many ways reflect differences in regional inputs from cortical and thalamic structures. Thus, whereas regions associated with cognitive function (e.g., medial parafasicular nucleus and prefrontal cortical areas; (Smith et al., 2004)) tend to converge on the dorsomedial striatum, motor regions of the thalamus and cortex (i.e., ventrolateral and ventral anterior nuclei and sensorimotor cortices; (McFarland and Haber, 2000)) tend to converge on the dorsolateral striatum and more motivational/emotional structures on the ventral striatum (including mediodorsal thalamus, amygdala, and insular cortex). Although these inputs are not exclusively segregated in this way, and the functions of the loops within which these structures are positioned have not yet been fully characterized, it seems likely that their functional significance is predominantly derived from the nature of these extrinsic inputs to the basal ganglia.
Summary
Based on the evidence described above, it is clear that the striatum contains functional sub-regions that are necessary for specific forms of learning and action control – refer Figure 1B. Within the dorsal striatum, the pDMS and DLS mediate the acquisition and performance of goal-directed and habitual behavior, respectively. Evidence is also emerging to suggest that medial agranular cortex and projections from that region to the dorsocentral region of dorsal striatum may be necessary for the acquisition of heterogeneous chains of actions (Ostlund et al, 2009; Wu et al, 2009). In the NAc, although the core is not necessary for instrumental learning, evidence suggests it mediates the effects of changes in the experienced reward value of the outcome on instrumental performance (Corbit et al, 2001). Furthermore, core and shell are necessary for the general excitatory and outcome-specific effects of CS’s on instrumental performance, respectively. Although the striatum has functionally defined sub-regions, this functional specialization may be determined, in part, by extrinsic inputs to striatum. In addition, the basal ganglia contain intrinsic feed-forward and feedback circuits that may be crucial for striatal function. In particular, striato-midbrain-striatal connections have been found that connect neighboring striatal regions through the VTA and SNc. This spiraling architecture links ventral striatum to the DMS, and the DMS to the DLS (Haber et al., 2000). Such an arrangement may enable striatal sub-regions to work cooperatively to support the transition from goal-directed to habitual behavior, as well as enable Pavlovian incentive information (from NAc) to influence action control guided by outcome information mediated by dorsal striatum (Corbit, Janak & Balleine, 2007; Corbit and Janak, 2007; Yin et al., 2008).
5. The molecular basis of instrumental reward-based learning and action control in the striatum
The striatum receives dopaminergic and glutamatergic inputs that form synapses in close proximity on striatal neurons (Sesack et al., 2003). This convergence of inputs in the striatum enables DA and glutamate signals to interact, and the integration of these signals is crucial for normal striatal function. At the intracellular level, signaling molecules that are sensitive to the combined activity of DA and glutamate receptors have been shown to regulate cellular excitability, corticostriatal synaptic plasticity, and behavior.
5.1 Molecular mechanisms of excitability and synaptic plasticity in striatum
One of the most important striatal inputs is that from the midbrain dopamine neurons in the ventral tegmental area (VTA) and SNc. These projections are essentially topographical with the more medial VTA projecting to NAc shell and the more lateral to NAc core. Likewise, more medial SNc projects to DMS whereas a more lateral region projects to the DLS. These projections terminate on MSN’s and on tonically active, cholinergeric interneurons. The projection neurons that receive this dopaminergic input comprise the striatopallidal and striatonigral pathways, and differ in their expression of dopamine receptor subtypes. Striatonigral MSN’s primarily express the D1 receptor; whereas striatopallidal neurons primarily express the D2 receptor sub-types (Gerfen et al., 1990). The dopamine D1 and D2 receptors have opposing effects on cellular physiology, which is due, in part, to their opposing effects on the enzyme adenylyl cyclase (AC). Activation of the D1 receptor increases AC activity, which increases cAMP formation and stimulates cAMP-dependent protein kinase (PKA) (see Greengard, 2001; Greengard et al., 1999 for review). PKA phosphorylates multiple cellular targets, including ion channels, neurotransmitter receptors, transcription factors, and other phosphoproteins. D1 receptor activation generally serves to increase MSN excitability and increase the potential for corticostriatal synaptic plasticity (Surmeier et al., 2007). Unlike D1 receptor activation, D2 receptor activation inhibits AC activity and reduces cAMP formation. This and other indirect effects of D2 receptor activation have a net effect of reducing striatal cell excitability (Surmeier et al., 2007).
It should be no surprise, therefore, that theories of basal ganglia function based on the distribution of DA receptor subtypes in the striatum and the effects of the direct and indirect pathways on basal ganglia output activity, have generally hypothesized that the striatonigral and striatopallidal circuits provide a functional architecture for the promotion or inhibition of action initiation (Albin et al., 1989; Eyny and Horvitz, 2003; Gerfen, 2000; Smith et al., 1998; Surmeier et al., 2007). However, this view may not fully capture basal ganglia function. First, as described above and elsewhere (e.g., (Redgrave et al., 2010)), the direct and indirect pathway architecture offers an incomplete description of the complex intrinsic basal ganglia circuitry. Second, although there are clear opposing effects of direct and indirect pathway stimulation on motor output (e.g., (Kravitz et al., 2010), there is less clear evidence dissociating these pathways on cognitive and motivational processes (e.g., (Di Ciano et al., 2001; Floresco, 2007; Salamone et al., 2002)). We suggest that the functional role of DA receptor subtypes may be characterized more accurately within the context of basal ganglia functional loops described above. A further description of the role dopamine signaling in the striatum plays in learning and action control can be found in the sections below.
In addition to DA receptors, striatal MSNs contain glutamate receptors, including AMPA, NMDA and group I metabotropic glutamate receptors (mgluR’s). Activation of ionotropic glutamate receptors in MSN’s directly or indirectly causes an influx of ca++ and activation of ca++ dependent intracellular signaling molecules, such as calcium-and-calmodulin-dependent protein kinase II (CamKII) and the protein phosphatase PP-2B (calcineurin) – refer to Figure 2. Activation of mgluR’s in striatum activates phospholipase C (PLC), which increases formation of inositol trisphosphate (IP3), and the release of intracellular calcium, which initiates a number of cellular processes, including an increase in the activity of the multi-functional kinase cyclin dependent kinase 5 (cdk5) (Liu et al., 2001). These signaling pathways influence a variety of cellular operations, and, as described below, provide a regulatory effect on dopamine signaling.
Figure 2.
Intracellular signaling pathways integrate dopamine and glutamate signals. D1 receptor activation increases adenylyl cyclase (AC) activation, cAMP formation, and PKA activation. PKA phosphorylates DARPP-32 at Thr-34 residue, making it a potent inhibitor of protein phosphatase-1 (PP-1). Inhibition of PP-1 indirectly increases extracellular signal regulated kinase (ERK) activation, by preventing PP-1 from dephosphorylating striatal-enriched phosphatase (STEP), which, when dephosphorylated, inhibits ERK. NMDA and AMPA receptor activation increases ca++ influx, whereas mGluR1 activation causes release of intracellular calcium through the phospholipase C (PLC), inositol trisphosphate (IP3) pathway. Ca++ activates a number of signaling pathways, including calcium and calmodulin-dependent kinase II (CamKII), and protein phosphatase 2B (PP-2B). PP-2B activation results in Thr-75 DARPP-32 phosphorylation, through casein kinase (ck) and cyclin-dependent kinase 5 (cdk5). When phospohrylated at Thr-75, DARPP-32 becomes an inhibitor of PKA. PP-2B also dephosphoryltes DARPP-32 at Thr-34. These effects negatively regulate ERK activation. CamKII activates ERK via the ras-raf-mek cascade. Thus ERK activation depends on the phosphorylation state of DARPP-32, which itself is sensitive to the relative activation of glutamate and DA receptors. ERK regulates gene expression through its phosphorylation of transcription factors, such as cAMP response element binding protein (CREB), and regulates neural excitability through its phosphorylation of the Kv4.2 channel. Red arrows indicate negative regulation of ERK, green arrows indicate positive regulation of ERK.
5.2 Dopamine-glutamate interactions and plasticity
DA and glutamate signaling interact in the striatum to influence neural excitability and synaptic plasticity. How DA and glutamate interact depends on the activity state of the MSN. At rest, striatal MSNs display a hyperpolarized membrane potential known as the down state (−80 mV) (Kreitzer, 2009; Nisenbaum and Wilson, 1995). During the down state, dopamine signaling acts to suppress MSN activity in response to glutamatergic inputs; however, during coordinated glutamatergic input, MSN’s shift to a less hyperpolarized up state (−60 mV), during which spiking is more likely to occur (Carr et al., 2003; Surmeier et al., 1992). During the up state, dopamine signaling facilitates changes in membrane potential caused by glutamate receptor activation, thereby increasing cell excitability (Carter and Sabatini, 2004; Flores-Hernandez et al., 2002; Hallett et al., 2006; Surmeier et al., 1995). Based on these properties, DA signaling has been hypothesized to act as a filter to reduce the effects of uncoordinated glutamatergic inputs on MSN activity and amplify the effects on MSN activity of coordinated glutamatergic inputs.
In addition to modulating the efficacy of glutamatergic inputs, DA is critical for synaptic plasticity at glutamatergic corticostriatal synapses. Long-term potentiation (LTP) and long-term depression (LTD) can be induced at corticostriatal synapses, and both forms of plasticity depend on dopamine (Di Filippo et al., 2009). LTP induction depends on coordinated activation of D1 and NMDA receptors (Calabresi et al., 1992; Kerr and Wickens, 2001); whereas LTD induction depends on D2 and mGluR1 receptor activation at indirect pathway neurons (Kreitzer and Malenka, 2005; Shen et al., 2008). LTD induction in striatum depends on the generation of a retrograde endocannabinoid signal that activates pre-synaptic CB1 receptors located on glutamatergic terminals, and causes a reduction in glutamate release (Adermark and Lovinger, 2007; Gerdeman et al., 2002; Kreitzer and Malenka, 2005). Thus, although the cellular mechanisms differ, coordinated DA and glutamate signaling enable corticostriatal LTP and LTD induction.
The ability of DA to influence glutamate-dependent signaling in MSN’s relies on a complex interaction of intracellular signaling pathways (Nakano et al., 2010). Much of this interaction centers on Dopamine- and cAMP-regulated phosphoprotein, 32 Kda (DARPP-32). DARPP-32 contains multiple phosphorylation sites, which enables it to serve as a dual-function protein (see Greengard et al., 1999; Svenningsson et al., 2004 and Figure 2 for review). During D1 receptor stimulation, PKA phosphorylates DARPP-32 at its Thr-34 residue. When phosphorylated at Thr-34, DARPP-32 becomes a potent inhibitor of the protein phosphatase PP-1. PP-1 dephosphorylates a number of cellular proteins, including ion channels, transcription factors and signaling kinases. Thr-34 DARPP-32 phosphorylation therefore affects a number of cellular processes through its inhibition of PP-1. In contrast, during glutamate receptor stimulation, calcium-mediated signaling activates cdk5, which phosphorylates DARPP-32 at its Thr-75 residue. When phosphorylated at Thr-75, DARPP-32 becomes an inhibitor of PKA. In addition, glutamate-mediated calcium influx activates PP-2B, which dephosphorylates DARPP-32 at Thr-34. Thus, DARPP-32 phosphorylation serves to integrate DA and glutamate signaling, and through its inhibition of PP-1 or PKA regulates a variety of cellular responses to extracellular signals.
One of the signaling pathways regulated by DARPP-32 in the striatum is extracellular signal regulated kinase (ERK), a member of the mitogen activated protein kinase (MAPK) signaling family. D1 receptor activation increases ERK phosphorylation in a manner that depends on DARPP-32’s inhibition of PP-1 (Valjent et al., 2005). PP-1 ordinarily dephosphorylates another phosphatase, the striatal-enriched phosphatase (STEP). PP-1’s dephosphorylation of STEP allows STEP to dephosphorylate ERK (Paul et al., 2003). PP-1 inhibition by Thr-34 DARPP-32 increases phosphorylated STEP and, as a consequence, reduces ERK dephosphorylation by STEP (Valjent et al., 2005) (Figure 2). DARPP-32’s inhibition of PP-1 also prevents PP-1 from dephosphorylating a kinase upstream of ERK. Thus, D1 receptor activation increases ERK phosphorylation by preventing its dephosphorylation by STEP and by preventing the dephosphorylation of kinases upstream of ERK. In addition, PKA activates ERK by phosphorylating rap1, a kinase upstream of ERK (Grewal et al., 2000).
ERK activation is also sensitive to glutamate receptor signaling. NMDA and AMPA receptor activation increases ERK phosphorylation through activation of calcium-dependent signaling pathways (Fasano et al., 2009; Mao et al., 2004; Perkinton et al., 1999; Wang et al., 2007) (Figure 2). However, glutamate negatively regulates ERK signaling through calcium-mediated phosphorylation of DARPP-32 at Thr-75 and dephosphorylation of DARPP-32 at Thr-34. This prevents DARPP-32 from inhibiting PP-1, which allows STEP to dephosphorylate ERK (Paul et al., 2003). Activation of mgluR’s in striatum appears to activate ERK through a calcium-dependent process and a calcium-independent process that depends on the scaffolding protein homer 1b/c (Mao et al., 2005).
Based on its complex regulation by phosphatases and kinases, ERK appears to be maximally activated during combined D1 and glutamate receptor signaling. This has lead to the suggestion that ERK acts to detect coincident activity of glutamate and DA signals (Girault et al., 2007). This may make ERK particularly important for mediating neural and behavioral processes that depend on coincident activation of these signaling pathways. In striatum, ERK has been shown to have an important role in regulating MSN excitability and corticostriatal plasticity. One target for ERK phosphorylation is the inwardly rectifying A-type K+ channel Kv4.2. Kv4.2 channels are fast-inactivating K+ channels located in distal dendrites that reduce spike initiation and limit action potential back propagation (Song et al., 1998; Tkatch et al., 2000). ERK phosphorylation of the Kv4.2 channel increases its open probability and therefore increases MSN excitability (Birnbaum et al., 2004). In this way, ERK activation may have a role in regulating striatal activity and ongoing behavior. In addition to targets in the cellular membrane, ERK regulates immediate-early gene expression through its phosphorylation of regulators of transcription, such as mitogen- and stress-activated protein kinase 1 (MSK1), ribosomal s6 kinase (rsk) and elk-1 (Sgambato et al., 1998). Rsk, in turn, phosphorylates the transcription factor cAMP response element binding protein (CREB) (Xing et al., 1996). CREB has a critical role in learning and synaptic plasticity (Bailey and Kandel, 1993). In MSN’s, DARPP-32, ERK and CREB are required for long-lasting forms of LTP and LTD (Calabresi et al., 2000; Mazzucchelli et al., 2002; Pittenger et al., 2006). This signaling pathway, activated by coordinated DA and glutamate receptor signaling, and resulting in DARPP-32, ERK and CREB phosphorylation, may have a critical role in corticostriatal plasticity and learning.
5.3 Molecular processes involved in goal-directed and habitual behavior
The learning processes that mediate the acquisition of goal-directed and habitual actions are likely the result of synaptic plasticity in the striatum, specifically corticostriatal synaptic plasticity. Numerous pieces of evidence support this hypothesis. First, lesions of cortical regions that project to the striatum have the same effect as striatal lesions on instrumental learning (Ostlund and Balleine, 2005). Second, treatments that disrupt corticostriatal plasticity prevent learning. Infusions into the pDMS of the NMDA antagonist AP-5, which is known to block LTP at corticostriatal synapses, prevents the acquisition of action-outcome associations during instrumental learning (Yin et al., 2005a). Likewise, disruption of cannabinoid signaling, which is critical for corticostriatal LTD, interferes with both instrumental learning and habit formation (Crombag et al.,; Hilario et al., 2007). Similarly, disruption of AC5, an isoform of AC enriched in striatum, disrupts corticostriatal LTD and prevents stimulus-response learning (Kheirbek et al., 2009). Third, the degree of potentiation at corticostriatal synapses is correlated with acquisition of an instrumental response for intracranial self-stimulation (Reynolds et al., 2001), suggesting that corticostriatal plasticity is directly related to acquisition of an instrumental response.
Further indirect evidence supports the notion that glutamate-dependent signaling is necessary for goal-directed learning. Disruption of AMPA signaling impairs instrumental performance following outcome devaluation. Mice lacking the gluR1 subunit of the AMPA receptor show no preference for the valued compared to the devalued action following specific satiety (Johnson et al., 2005). Because this mouse lacks gluR1 subunits throughout the brain, it may be that other regions that project to striatum and are important for outcome encoding (e.g., basolateral amygdala) may mediate this effect. Furthermore, it is not known whether this deficit reflects impaired learning or performance. In subsequent studies, however, GluR1 deficient mice showed no effect of sensory-specific satiety on instrumental responding in a heterogeneous instrumental chain. Normally, devaluation of the instrumental outcome reduces responding on the distal action in an instrumental chain (Corbit and Balleine, 2003). GluR1 deficient mice showed no effect of devaluation on distal responses, but showed an effect of general satiety on proximal responses. Thus, these mice appear to have a specific deficit in using sensory-specific outcome representations to guide instrumental behavior (Johnson et al., 2007).
DA also has an important role in goal-directed and habitual behavior. DA release in the striatum during reward learning tasks is well documented, and is thought to reflect a reward prediction error as defined in reinforcement learning (Schultz et al., 1997; Schultz and Dickinson, 2000). However, the effects of DA antagonists during goal-directed learning have been difficult to interpret. One difficulty with assessing the effects of DA antagonists in the striatum is that they also impair motor performance; hence a supposed learning deficit may in fact be due to the animal’s inability to carry out the motor actions necessary for learning, yet learning, if it were possible, may be intact (e.g., (Darvas and Palmiter, 2010; Robinson et al., 2007; Robinson et al., 2006). Despite these challenges, it has been shown that 6-hydroxydopamine (OHDA) infusions that deplete DA in the pDMS prior to instrumental learning result in impaired performance on a contingency degradation paradigm, but leave outcome devaluation intact (Lex and Hauber, 2010). In the dorsolateral striatum, 6-OHDA infusions in the DLS prevent the development of habitual, i.e. outcome insensitive, instrumental responding (Faure et al., 2005). In subsequent studies, it was shown that DA agonists infused into this area did not restore habitual responding in DA-depleted animals, suggesting that, although acquisition of habitual behavior is DA dependent, performance of habitual actions is not (Faure et al., 2010). Data from studies of conditioned approach behavior, which is stimulus bound and ostensibly mediated by a similar mechanism, also shows a transition from DA dependent to DA-independent responding as a result of training experience (Choi et al., 2005). Although focused on different learning and performance processes, these data are consistent in suggesting an initial dependence on DA in the striatum during learning of habitual and goal-directed actions, whereas performance of at least habitual actions appears to become DA independent after acquisition.
Intracellular signaling activity sensitive to DA has also been implicated in the performance of goal-directed and habitual actions. Regulators of cAMP formation have been shown to alter instrumental performance. One such regulator of cAMP in the striatum is the lipid sphingosine-1-phosphate, which regulates cAMP formation through a G protein-coupled receptor (GPR6). GPR6 is found specifically in striatopallidal neurons, and mice with a Gpr6 mutation showed reduced cAMP formation (Lobo et al., 2007). Behaviorally, Gpr6 null mice showed increased response rates in a progressive ratio task and reduced sensitivity to contingency degradation (Lobo et al., 2007). Additional tests demonstrated that these mice had normal outcome valuation, suggesting that the enhanced responding in the progressive ratio was likely the result of an altered ability to initiate actions. A similar finding was observed in mice with a mutation of cdk5. Mice with a cdk5 mutation show increased striatal cell excitability and increased responses on a progressive ratio schedule (Benavides et al., 2007). Since cdk5 negatively regulates PKA, the behavioral effects of cdk5 mutation presumably are the result of increased PKA activity. Thus, inhibition of DA-dependent signaling through Gpr6 mutation in the indirect pathway, or disinhibition of DA-dependent signaling through cdk5 mutation in the direct pathway increases action initiation as measured by progressive ratio responding.
ERK signaling has an important role in both the acquisition and performance of goal-directed actions (Shiflett and Balleine, 2011; Shiflett et al., 2010). Instrumental training increases ERK phosphorylation in the pDMS. Furthermore, inhibition of ERK activation by infusion of the MEK/ERK inhibitor U0126 into the pDMS prior to an instrumental training episode prevents accurate performance in a subsequent devaluation task. These results demonstrate that the opportunity to acquire action-outcome associations activates ERK in the pDMS, and blocking ERK activation in this region prevents encoding of this relationship. In addition to a role in learning, ERK is necessary for the performance of goal-directed actions. Inhibition of ERK activity in the pDMS after training but prior to a devaluation choice extinction test was found to impair choice performance; during the test, rats showed no preference for the valued action compared to the devalued action (Shiflett et al, 2010). ERK inhibition did not affect the amount of food consumed during the specific-satiety procedure, suggesting that ERK inhibition does not interfere with the devaluation process itself, nor did ERK inhibition impair rats’ ability to make instrumental responses generally. Rather, it appears that ERK is specifically required in order to use outcome information, including the desirability of an outcome, to direct instrumental action. Interestingly, ERK inhibition in the DLS did not affect action-outcome learning; however, DLS ERK inhibition prior to the devaluation test impaired task performance, suggesting that the DLS has some role to play in the performance of goal-directed actions, and that this function requires ERK signaling. Given the role of the DLS in S-R learning, it is possible the DLS contributes to the ability of environmental stimuli to promote the selection (and, therefore, the performance) of actions related to salient cues. To the extent that ERK activity is engaged by a form of action-selection process, it should be expected that blocking ERK phosphorylation will tend to reduce action selection and performance, particularly of actions associated with a non-devalued outcome.
Regulation of gene expression in the striatum likely underlies the long-lasting memory associated with goal-directed and habitual behavior. Evidence in support of this comes from the finding that instrumental acquisition is associated with immediate-early gene induction in the striatum, and with more extensive training, expression becomes more prominent in lateral striatum (Hernandez et al., 2006). Additionally, interference with transcriptional regulation in the striatum during learning prevents development of habitual behavior. In particular, the transcription factor CREB is likely involved in the development of habitual behavior. Rats trained in a plus maze task that used a response strategy to encode the goal location show elevated phosphorylated CREB levels in the DLS after learning compared to rats that used a place strategy, which showed elevated CREB phosphorylation in the hippocampus (Colombo et al., 2003). Furthermore, when infused in DLS, a mutant form of CREB that impairs CREB-dependent gene expression also impairs acquisition of a response strategy (Brightwell et al., 2008). CREB may enable learning by mediating corticostriatal plasticity. A dominant-negative CREB mutation that is specific to the dorsal striatum impairs acquisition of a stimulus-response task, and impairs LTP and LTD induction at corticostriatal synapses (Pittenger et al., 2006). Although it is not known whether these tasks are governed in a goal-directed, habitual, or Pavlovian manner, the results suggest a critical involvement of CREB in the striatum for learning.
Although the striatum is clearly engaged during learning and performance of goal-directed and habitual actions, this does not necessarily entail that the striatum is the long-term store of associations that underlie these behaviors. Given that coordinated engagement of various components of cortico-basal ganglia circuitry is necessary for learning and task performance, it may not be plausible or even accurate to identify the location of the “engram” that encodes a particular action-outcome or stimulus-response association as lying within any one structure in this circuit. However, what differentiates various regions within the circuit is their recruitment during distinct task components (acquisition, retrieval, action selection and initiation) and, based on this interpretation, striatal sub-regions are clearly necessary, if not themselves sufficient, for both learning and performance of goal-directed and habitual actions.
5.4 Molecular processes involved in Pavlovian-instrumental interactions
The ability of conditioned stimuli to influence instrumental action depends critically on DA signaling in the NAc. DA release in the NAc has been shown to gradually shift from presentation of appetitive outcomes to the cues that predict those outcomes during Pavlovian conditioning (Day et al., 2007). During tests of Pavlovian-instrumental transfer (PIT), which measures the effect of cue presentation on instrumental responding, DA antagonist infusion into the NAc abolishes PIT (Lex and Hauber, 2008), as do lesions of the ventral tegmental area (VTA), which provides DA input to the NAc (Corbit et al., 2007; Murschall and Hauber, 2006). Intra-NAc infusion of the psychostimulant amphetamine, which enhances DA signaling, increases the general form of PIT (Wyvell and Berridge, 2000). DA-dependent signaling has also been shown to have a role in PIT. Mice with a knockout of AC5 signaling show impairments in PIT (Kheirbek et al., 2008). AC5 KO mice show normal sensitivity of instrumental performance to outcome devaluation, suggesting that AC5-dependent signaling has a specific role in expression of Pavlovian responses. AC5 KO’s show reduced ERK phosphorylation in the NAc following D1 receptor stimulation, suggesting that the impairment in behavior observed in these animals may be due to impaired ERK signaling (Kheirbek et al., 2008).
The role of glutamate during PIT is less well understood, in part because of the use of different PIT paradigms and whether the NAc core or shell was targeted. For example, infusions into the NAc of the AMPA antagonist CNQX or the NMDA antagonist dizocilpine have no effect on the general form of PIT (Murschall and Hauber, 2005). However, mice lacking the gluR1 subunit of the AMPA receptor show deficits in outcome-specific PIT (Johnson et al., 2007). It may be that outcome-specific PIT has more of a requirement for glutamate signaling than the general form of PIT (Crombag et al., 2008; Mead and Stephens, 2003). However, because the GluR1 knockout was not restricted to the striatum, it is not entirely clear whether the deficit in PIT in GluR1-deficient mice can be ascribed to impaired AMPA signaling in the striatum. More direct evidence implicating glutamate signaling in outcome-specific PIT comes from transgenic mice that express a constitutively active form of CamKII that is restricted to the striatum (Wiltgen et al., 2007). Changes in CaMKII phosphorylation regulate cellular excitability through a variety of mechanisms, most notably through the phosphorylation of the Kv4.2 channel (Varga et al., 2004) and of AMPA receptor channels (Carvalho et al, 2000). The transgene that mimics the constituitively active state is expressed only in the striatum and is under the control of a tetracycline promoter. As such, its expression can be regionally and temporally localized. Whether the transgene was turned off or on, both Pavlovian conditioning and instrumental conditioning were normal in these mice; performance during Pavlovian conditioning, during instrumental acquisition and in the outcome devaluation and contingency degradation tests did not differ from wildtype controls. However, with the transgene turned on, mice show impaired performance on an outcome-specific PIT task, and performance on the PIT task was immediately restored when the transgene was turned off (see Wiltgen et al, 2007). This implicates CamKII and striatal excitability in the ability of Pavlovian cues to bias choice performance in instrumental conditioning.
ERK signaling in the NAc plays an important role in PIT. ERK phosphorylation in the NAc increases during presentation of a reward-associated CS (Shiflett et al., 2008). Furthermore, infusion of the ERK inhibitor U0126 into the NAc prior to testing prevents PIT. Intra-NAc U0126 infusion has no effect on instrumental responding generally, which indicates that ERK inhibition causes a specific impairment in the ability of CS’s to motivate instrumental behavior. Like CamKII, ERK may enable PIT through its effects on cell excitability. ERK is known to modulate excitability through Kv4.2 channel phosphorylation, which gives rise to increased MSN excitability. ERK could also influence behavioral responses to cues through its effects on CREB. CREB phosphorylation in the NAc occurs during presentation of emotionally salient stimuli (Barrot et al., 2002; Barrot et al., 2005; Shaw-Lutchman et al., 2002). In addition, CS’s associated with rewards increase CREB phosphorylation in the NAc (Shiflett et al., 2009). Treatments that block CREB phosphorylation in the NAc, such as viral-mediated expression of an inactive form of CREB, or over-expression of an inhibitor of CREB function, such as inducible cAMP early repressor (ICER) increases behavioral responses to emotionally salient stimuli, whereas increasing CREB activation reduces behavioral responses (Barrot et al., 2002; Barrot et al., 2005; Green et al., 2006a; Green et al., 2006b). This process seems to depend on an alteration in MSN excitability. Constitutive expression of active CREB increases MSN excitability and NMDA-mediated up-state transitions, suggesting that increased MSN excitability reduces behavioral responses to stimuli including behavioral responsese to cocaine (Dong et al., 2006; Huang et al., 2008). Furthermore, artificial reduction in NAc activity by overexpressing K+ channels reduces behavioral responses to cocaine (Dong et al., 2006.) It may be that “phasic” increases in CREB in the NAc caused by presentation of an emotionally-salient stimulus mediate learning, whereas long-term alterations in CREB caused by repeated drug exposure results in constitutive changes in MSN excitability that may alter responsiveness to salient environmental stimuli.
6. Conclusions
Although our understanding of the molecular mechanisms responsible for striatal function is far from complete, some general conclusions can be drawn. It is clear that learning, whether in the form of action-outcome or stimulus-response associations, depends upon the integration of glutamate and dopamine signals (Horvitz, 2009). Although this review focused on intracellular mechanisms that integrate DA and glutamate signals, interactions between these signaling systems occurs through multiple cellular mechanisms. The general theme is that under conditions of coordinated glutamatergic input to the striatum, DA facilitates the ability of glutamate signals to drive MSN activity and initiate corticostriatal plasticity. Not coincidentally, the conditions that give rise to coordinated glutamate and DA release in the striatum are situations in which learning and decision-making take place. Furthermore, it is likely that goal-directed learning is based on corticostriatal LTP in the pDMS, and that acquisition of habits is based on LTD in the DLS. This is based, in part, on the observation that LTP is more easily induced in the DMS, whereas LTD is more easily induced in the DLS (Partridge et al., 2000).
DA and glutamate signals interact in a complex fashion at the level of intracellular signaling molecules. Highlighted in this review is the special importance of the ERK signaling cascade, which, because of its sensitivity to combined glutamate and DA receptor activation, may serve to detect coincident activity in these signaling pathways (Girault et al., 2007). The complex regulation of ERK activity by DARPP-32 and other signaling pathways seems to favor conditions of combined D1 and NMDA receptor activation for maximal ERK activation. ERK activation may enable synaptic plasticity in the striatum through transcriptional regulation. In particular, ERK phosphorylates the transcription factor CREB, which is known to be necessary for lasting forms of corticostriatal LTP and LTD. It is not entirely clear how ERK and CREB are involved in corticostriatal LTD if the translational modifications necessary for corticostriatal LTD are pre-synaptic (Yin et al., 2006). Nor is it known whether ERK is necessary for the acquisition of habitual behavior. It may be that a post-synaptic translational component of LTD exists in corticostriatal LTD, and much like LTD in other regions, requires ERK activation (Thiels et al., 2002). A question for future research is whether the plasticity induced in these learning situations primarily affects the striatopallidal indirect or striatonigral direct pathway neurons. Such information will be useful in understanding how learning alters striatal activity to enable performance of goal-directed or habitual actions (Stalnaker et al., 2010; Thorn et al., 2010; Yin et al., 2009).
In addition to its role in learning, ERK activation in the pDMS has been shown to be necessary for flexible responding following outcome devaluation. ERK may enable this behavioral flexibility by conferring MSN’s with plasticity of cellular excitability. During the outcome devaluation task, outcome value information (perhaps provided by DA release in striatum) is combined with action-outcome information (perhaps provided by glutamate release in striatum). Selection and initiation of actions may depend on the ability of these inputs to generate MSN activity, a function that may be mediated by ERK. Similar to its role in the dorsal striatum, ERK activation in the NAc is required for environmental stimuli to modulate instrumental performance. Like outcome devaluation, outcome-selective PIT requires the integration of outcome information, in this case stimulus-outcome information with action-outcome information to direct responding. ERK alters excitability through its phosphorylation of Kv4.2 channels, which results in enhanced activity in response to glutamatergic input. It may be that the loss of ERK signaling prevents accurate performance following outcome devaluation or during the PIT task because DA and glutamate signals are unable to be integrated properly to modulate MSN activity and thereby allow for initiation of the appropriate action. If this is the case, then ERK should have its behavioral effects by phosphorylating cytosolic targets. Treatments that block the nuclear transport of ERK but preserve its cytosolic signaling may be instructive in this regard (Paul et al., 2007). Further tests of this hypothesis would include an examination of Kv4.2 phosphorylation during choice performance following outcome devaluation and whether disruption of Kv4.2 phosphorylation prevents flexible responding.
Many of the signaling molecules described in this review have been implicated in drug addiction (Berke and Hyman, 2000). This may be expected when one considers drug-seeking behavior and the response to drug-associated cues are governed by the same neural processes that underlie natural rewards. What makes drugs of abuse unique from natural rewards, and may give rise to features of addictive behavior, is the overactivity of these signaling mechanisms that results from drug exposure. Indeed, exposure to abused substances may involve abnormal corticostriatal plasticity and changes in MSN excitability (Bamford et al., 2008). Many of the signaling molecules discussed here have been implicated in various aspects of addiction-related behavior (Ortiz et al., 1995; Tropea et al., 2008; Valjent et al., 2000; Walters and Blendy, 2001); e.g., in similar fashion to its role in processing natural rewards, inhibition of ERK activation in the NAc prevents expression of a drug conditioned place preference (Miller and Marshall, 2004). Placing the molecular alterations as a result of repeated drug exposure in the context of striatal function may help to understand to what extent addictive behavior is a disorder of mechanisms regulating goal-directed action, habit, or control of behavior by Pavlovian cues.
Understanding the relationship between molecular processes and adaptive behavior is an important but daunting goal of neuroscience research. The role of a particular signaling molecule, such as ERK, in behavior has not only to be understood within its functional context; e.g., how ERK regulates gene expression and cellular activity but also, in turn, has to be understood within a wider context; i.e. how changes in gene expression and cellular activity regulate basal ganglia network activity and ultimately specific aspects of adaptive behavior. At the same time, an effective framework for understanding specific psychological functions is necessary for interpreting the effects of neural manipulations on behavior. This is particularly relevant for translational research, whose goal is to enable research findings to inform therapeutic strategy. The research described in this review attempts to make these links between molecular changes in striatum, adaptive behavior and, more broadly, adaptive psychological capacities that control those behavioral effects. It is our belief that the sort of research described in this review will facilitate the development of effective treatments for disorders of the striatum, such as neurodegenerative conditions and addiction.
Acknowledgements
The preparation of this article was supported by a Laureate Fellowship from the Australian Research Council, grant #MH56446 from the NIMH and #HD59257 from NICHD to BWB and grant # 29544 from NIDA to MWS.
Abbreviation list
- AC
adenylyl cyclase
- Cdk5
cyclin dependent kinase 5
- CREB
cAMP response element binding protein
- CS
conditioned stimulus
- DA
dopamine
- D1
dopamine D1 receptor
- D2
dopamine D2 receptor
- DARPP-32
Dopamine- and cAMP-regulated phosphoprotein
- DMS
dorsomedial striatum
- DLS
dorsolateral striatum
- ERK
extracellular signal regulated kinase
- GPe
globus pallidus external
- GPi
globus pallidus internal
- IMD
intermediodoral nucleus of the thalamus
- LTD
long term depression
- LTP
long term potentiation
- MD
mediodorsal thalamus
- mGLUr
metabotropic glutamate receptors
- MSK1
mitogen- and stress-activated protein kinase 1
- MSN
medium spiny neuron
- NAc
nucleus accumbens
- pDMS
posterior dorsomedial striatum
- PIT
Pavlovian-instrumental transfer
- PKA
protein kinase A
- PL
prelimbic region of the medial prefrontal cortex
- PO
posterior nucleus of the thalamus
- PP
protein phosphatase
- R-O
response-outcome
- Rsk
ribosomal s6 kinase
- S-R
stimulus-response
- SNc
substantia nigra pars compacta
- SNr
substantia nigra pars reticulata
- STEP
striatal-enriched phosphatase
- STN
subthalamic nucleus
- VA/VL
ventroanterior/ventrolateral nucleus of the thalamus
- VP
ventral pallidum
- VS
ventral striatum
- VTA
ventral tegmental area
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- Adams CD, Dickinson A. Instrumental responding following reinforcer devaluation. Quarterly Journal of Experimental Psychology Section B-Comparative and Physiological Psychology. 1981;33:109–121. [Google Scholar]
- Adermark L, Lovinger DM. Combined activation of L-type Ca2+ channels and synaptic transmission is sufficient to induce striatal long-term depression. J Neurosci. 2007;27:6781–6787. doi: 10.1523/JNEUROSCI.0280-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Albin RL, Young AB, Penney JB. The functional anatomy of basal ganglia disorders. Trends Neurosci. 1989;12:366–375. doi: 10.1016/0166-2236(89)90074-x. [DOI] [PubMed] [Google Scholar]
- Alexander GE, Crutcher MD, DeLong MR. Basal ganglia-thalamocortical circuits: parallel substrates for motor, oculomotor, "prefrontal" and "limbic" functions. Prog Brain Res. 1990;85:119–146. [PubMed] [Google Scholar]
- Alexander GE, Delong MR, Strick PL. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu Rev Neurosci. 1986;9:357–381. doi: 10.1146/annurev.ne.09.030186.002041. [DOI] [PubMed] [Google Scholar]
- Bailey CH, Kandel ER. Structural changes accompanying memory storage. Annu Rev Physiol. 1993;55:397–426. doi: 10.1146/annurev.ph.55.030193.002145. [DOI] [PubMed] [Google Scholar]
- Balleine BW, Dickinson A. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology. 1998;37:407–419. doi: 10.1016/s0028-3908(98)00033-1. [DOI] [PubMed] [Google Scholar]
- Balleine BW, Doya K, O'Doherty J, Sakagami M. Current trends in decision making. Ann N Y Acad Sci. 2007;1104:xi–xv. doi: 10.1196/annals.1390.2226. [DOI] [PubMed] [Google Scholar]
- Balleine BW, Liljeholm M, Ostlund SB. The integrative function of the basal ganglia in instrumental conditioning. Behav Brain Res. 2008 doi: 10.1016/j.bbr.2008.10.034. [DOI] [PubMed] [Google Scholar]
- Balleine BW, Liljeholm M, Ostlund SB. The integrative function of the basal ganglia in instrumental conditioning. Behav Brain Res. 2009;199:43–52. doi: 10.1016/j.bbr.2008.10.034. [DOI] [PubMed] [Google Scholar]
- Balleine BW, O'Doherty JP. Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology. 2010;35:48–69. doi: 10.1038/npp.2009.131. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Balleine BW, Ostlund SB. Still at the choice-point: action selection and initiation in instrumental conditioning. Ann N Y Acad Sci. 2007;1104:147–171. doi: 10.1196/annals.1390.006. [DOI] [PubMed] [Google Scholar]
- Bamford NS, Zhang H, Joyce JA, Scarlis CA, Hanan W, Wu N-P, André VM, Cohen R, Cepeda C, Levine MS, Harleton E, Sulzer D. Repeated Exposure to Methamphetamine Causes Long-Lasting Presynaptic Corticostriatal Depression that Is Renormalized with Drug Readministration. Neuron. 2008;58:89–103. doi: 10.1016/j.neuron.2008.01.033. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Barrot M, Olivier JDA, Perrotti LI, DiLeone RJ, Berton O, Eisch AJ, Impey S, Storm DR, Neve RL, Yin JC, Zachariou V, Nestler EJ. CREB activity in the nucleus accumbens shell controls gating of behavioral responses to emotional stimuli. PNAS. 2002;99:11435–11440. doi: 10.1073/pnas.172091899. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Barrot M, Wallace DL, Bolanos CA, Graham DL, Perrotti LI, Neve RL, Chambliss H, Yin JC, Nestler EJ. Regulation of anxiety and initiation of sexual behavior by CREB in the nucleus accumbens. Proc Natl Acad Sci U S A. 2005;102:8357–8362. doi: 10.1073/pnas.0500587102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bechara A, Dolan S, Hindes A. Decision-making and addiction (part II): myopia for the future or hypersensitivity to reward? Neuropsychologia. 2002;40:1690–1705. doi: 10.1016/s0028-3932(02)00016-7. [DOI] [PubMed] [Google Scholar]
- Belin D, Jonkman S, Dickinson A, Robbins TW, Everitt BJ. Parallel and interactive learning processes within the basal ganglia: relevance for the understanding of addiction. Behav Brain Res. 2009;199:89–102. doi: 10.1016/j.bbr.2008.09.027. [DOI] [PubMed] [Google Scholar]
- Benavides DR, Quinn JJ, Zhong P, Hawasli AH, DiLeone RJ, Kansy JW, Olausson P, Yan Z, Taylor JR, Bibb JA. Cdk5 modulates cocaine reward, motivation, and striatal neuron excitability. J Neurosci. 2007;27:12967–12976. doi: 10.1523/JNEUROSCI.4061-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Berendse HW, Galis-de Graaf Y, Groenewegen HJ. Topographical organization and relationship with ventral striatal compartments of prefrontal corticostriatal projections in the rat. J Comp Neurol. 1992;316:314–347. doi: 10.1002/cne.903160305. [DOI] [PubMed] [Google Scholar]
- Berke J, Hyman S. Addiction, dopamine and the molecular mechanisms of memory. Neuron. 2000;25:515–532. doi: 10.1016/s0896-6273(00)81056-9. [DOI] [PubMed] [Google Scholar]
- Berridge KC. 'Liking' and 'wanting' food rewards: brain substrates and roles in eating disorders. Physiol Behav. 2009;97:537–550. doi: 10.1016/j.physbeh.2009.02.044. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Birnbaum SG, Varga AW, Yuan L-L, Anderson AE, Sweatt JD, Schrader LA. Structure and Function of Kv4-Family Transient Potassium Channels. Physiol. Rev. 2004;84:803–833. doi: 10.1152/physrev.00039.2003. [DOI] [PubMed] [Google Scholar]
- Brightwell JJ, Smith CA, Neve RL, Colombo PJ. Transfection of mutant CREB in the striatum, but not the hippocampus, impairs long-term memory for response learning. Neurobiol Learn Mem. 2008;89:27–35. doi: 10.1016/j.nlm.2007.09.004. [DOI] [PubMed] [Google Scholar]
- Brog JS, Salyapongse A, Deutch AY, Zahm DS. The patterns of afferent innervation of the core and shell in the "accumbens" part of the rat ventral striatum: immunohistochemical detection of retrogradely transported fluoro-gold. J Comp Neurol. 1993;338:255–278. doi: 10.1002/cne.903380209. [DOI] [PubMed] [Google Scholar]
- Burns LH, Everitt BJ, Kelley AE, Robbins TW. Glutamate-dopamine interactions in the ventral striatum: role in locomotor activity and responding with conditioned reinforcement. Psychopharmacology (Berl) 1994;115:516–528. doi: 10.1007/BF02245576. [DOI] [PubMed] [Google Scholar]
- Calabresi P, Gubellini P, Centonze D, Picconi B, Bernardi G, Chergui K, Svenningsson P, Fienberg AA, Greengard P. Dopamine and cAMP-Regulated Phosphoprotein 32 kDa Controls Both Striatal Long-Term Depression and Long-Term Potentiation, Opposing Forms of Synaptic Plasticity. J. Neurosci. 2000;20:8443–8451. doi: 10.1523/JNEUROSCI.20-22-08443.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Calabresi P, Pisani A, Mercuri NB, Bernardi G. Long-term Potentiation in the Striatum is Unmasked by Removing the Voltage-dependent Magnesium Block of NMDA Receptor Channels. Eur J Neurosci. 1992;4:929–935. doi: 10.1111/j.1460-9568.1992.tb00119.x. [DOI] [PubMed] [Google Scholar]
- Cardinal RN, Everitt BJ. Neural and psychological mechanisms underlying appetitive learning: links to drug addiction. Curr Opin Neurobiol. 2004;14:156–162. doi: 10.1016/j.conb.2004.03.004. [DOI] [PubMed] [Google Scholar]
- Cardinal RN, Parkinson JA, Hall J, Everitt BJ. Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev. 2002;26:321–352. doi: 10.1016/s0149-7634(02)00007-6. [DOI] [PubMed] [Google Scholar]
- Carr DB, Day M, Cantrell AR, Held J, Scheuer T, Catterall WA, Surmeier DJ. Transmitter modulation of slow, activity-dependent alterations in sodium channel availability endows neurons with a novel form of cellular plasticity. Neuron. 2003;39:793–806. doi: 10.1016/s0896-6273(03)00531-2. [DOI] [PubMed] [Google Scholar]
- Carter AG, Sabatini BL. State-dependent calcium signaling in dendritic spines of striatal medium spiny neurons. Neuron. 2004;44:483–493. doi: 10.1016/j.neuron.2004.10.013. [DOI] [PubMed] [Google Scholar]
- Choi WY, Balsam PD, Horvitz JC. Extended habit training reduces dopamine mediation of appetitive response expression. J Neurosci. 2005;25:6729–6733. doi: 10.1523/JNEUROSCI.1498-05.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cohen MX, Frank MJ. Neurocomputational models of basal ganglia function in learning, memory and choice. Behav Brain Res. 2009;199:141–156. doi: 10.1016/j.bbr.2008.09.029. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Colombo PJ, Brightwell JJ, Countryman RA. Cognitive Strategy-Specific Increases in Phosphorylated cAMP Response Element-Binding Protein and c-Fos in the Hippocampus and Dorsal Striatum. J. Neurosci. 2003;23:3547–3554. doi: 10.1523/JNEUROSCI.23-08-03547.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Colwill RM, Rescorla RA. Postconditioning devaluation of a reinforcer affects instrumental responding. Journal of Experimental Psychology-Animal Behavior Processes. 1985;11:120–132. [PubMed] [Google Scholar]
- Corbit LH, Balleine BW. Instrumental and Pavlovian incentive processes have dissociable effects on components of a heterogeneous instrumental chain. J Exp Psychol Anim Behav Process. 2003;29:99–106. doi: 10.1037/0097-7403.29.2.99. [DOI] [PubMed] [Google Scholar]
- Corbit LH, Balleine BW. Double Dissociation of Basolateral and Central Amygdala Lesions on the General and Outcome-Specific Forms of Pavlovian-Instrumental Transfer. J. Neurosci. 2005;25:962–970. doi: 10.1523/JNEUROSCI.4507-04.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Corbit LH, Janak PH. Inactivation of the lateral but not medial dorsal striatum eliminates the excitatory impact of Pavlovian stimuli on instrumental responding. J Neurosci. 2007;27:13977–13981. doi: 10.1523/JNEUROSCI.4097-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Corbit LH, Janak PH, Balleine BW. General and outcome-specific forms of Pavlovian-instrumental transfer: the effect of shifts in motivational state and inactivation of the ventral tegmental area. European Journal of Neuroscience. 2007;26:3141–3149. doi: 10.1111/j.1460-9568.2007.05934.x. [DOI] [PubMed] [Google Scholar]
- Corbit LH, Muir JL, Balleine BW. The role of the nucleus accumbens in instrumental conditioning: Evidence of a functional dissociation between accumbens core and shell. J Neurosci. 2001;21:3251–3260. doi: 10.1523/JNEUROSCI.21-09-03251.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cory-Slechta DA, O'Mara DJ, Brockel BJ. Learning versus performance impairments following regional administration of MK-801 into nucleus accumbens and dorsomedial striatum. Behav Brain Res. 1999;102:181–194. doi: 10.1016/s0166-4328(99)00015-7. [DOI] [PubMed] [Google Scholar]
- Costa RM. Plastic Corticostriatal Circuits for Action Learning. Annals of the New York Academy of Sciences. 2007;1104:172–191. doi: 10.1196/annals.1390.015. [DOI] [PubMed] [Google Scholar]
- Crombag HS, Johnson AW, Zimmer AM, Zimmer A, Holland PC. Deficits in sensory-specific devaluation task performance following genetic deletions of cannabinoid (CB1) receptor. Learn Mem. 17:18–22. doi: 10.1101/lm.1610510. [DOI] [PubMed] [Google Scholar]
- Crombag HS, Sutton JM, Takamiya K, Holland PC, Gallagher M, Huganir RL. A role for alpha-amino-3-hydroxy-5-methylisoxazole-4-propionic acid GluR1 phosphorylation in the modulatory effects of appetitive reward cues on goal-directed behavior. Eur J Neurosci. 2008;27:3284–3291. doi: 10.1111/j.1460-9568.2008.06299.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Darvas M, Palmiter RD. Restricting dopaminergic signaling to either dorsolateral or medial striatum facilitates cognition. J Neurosci. 2010;30:1158–1165. doi: 10.1523/JNEUROSCI.4576-09.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Day JJ, Carelli RM. The nucleus accumbens and Pavlovian reward learning. Neuroscientist. 2007;13:148–159. doi: 10.1177/1073858406295854. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Day JJ, Roitman MF, Wightman RM, Carelli RM. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat Neurosci. 2007;10:1020–1028. doi: 10.1038/nn1923. [DOI] [PubMed] [Google Scholar]
- Dayan P, Balleine B. Reward, motivation, and reinforcement learning. Neuron. 2002;36:285–298. doi: 10.1016/s0896-6273(02)00963-7. [DOI] [PubMed] [Google Scholar]
- Delgado MR. Reward-related responses in the human striatum. Ann N Y Acad Sci. 2007;1104:70–88. doi: 10.1196/annals.1390.002. [DOI] [PubMed] [Google Scholar]
- Di Ciano P, Cardinal RN, Cowell RA, Little SJ, Everitt BJ. Differential Involvement of NMDA, AMPA/Kainate, and Dopamine Receptors in the Nucleus Accumbens Core in the Acquisition and Performance of Pavlovian Approach Behavior. J. Neurosci. 2001;21:9471–9477. doi: 10.1523/JNEUROSCI.21-23-09471.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Di Filippo M, Picconi B, Tantucci M, Ghiglieri V, Bagetta V, Sgobio C, Tozzi A, Parnetti L, Calabresi P. Short-term and long-term plasticity at corticostriatal synapses: Implications for learning and memory. Behavioural Brain Research. 2009;199:108–118. doi: 10.1016/j.bbr.2008.09.025. [DOI] [PubMed] [Google Scholar]
- Dickinson A, Balleine B. Motivational control of goal-directed action. Anim Learn Behav. 1994;22:1–18. [Google Scholar]
- Dickinson A, Balleine B. The role of learning in the operation of motivational systems. In: Pashler H, editor. Steven's Handbook of Experimental Psychology. New York: Wiley; 2002. pp. 497–533. [Google Scholar]
- Dickinson A, Wood N, Smith JW. Alcohol seeking by rats: action or habit? Q J Exp Psychol B. 2002;55:331–348. doi: 10.1080/0272499024400016. [DOI] [PubMed] [Google Scholar]
- Dong Y, Green T, Saal D, Marie H, Neve R, Nestler EJ, Malenka RC. CREB modulates excitability of nucleus accumbens neurons. Nat Neurosci. 2006;9:475–477. doi: 10.1038/nn1661. [DOI] [PubMed] [Google Scholar]
- Eyny YS, Horvitz JC. Opposing roles of D1 and D2 receptors in appetitive conditioning. J Neurosci. 2003;23:1584–1587. doi: 10.1523/JNEUROSCI.23-05-01584.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fasano S, D'Antoni A, Orban PC, Valjent E, Putignano E, Vara H, Pizzorusso T, Giustetto M, Yoon B, Soloway P, Maldonado R, Caboche J, Brambilla R. Ras-guanine nucleotide-releasing factor 1 (Ras-GRF1) controls activation of extracellular signal-regulated kinase (ERK) signaling in the striatum and long-term behavioral responses to cocaine. Biol Psychiatry. 2009;66:758–768. doi: 10.1016/j.biopsych.2009.03.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Faure A, Haberland U, Conde F, Massioui NE. Lesion to the Nigrostriatal Dopamine System Disrupts Stimulus-Response Habit Formation. J. Neurosci. 2005;25:2771–2780. doi: 10.1523/JNEUROSCI.3894-04.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Faure A, Leblanc-Veyrac P, El Massioui N. Dopamine agonists increase perseverative instrumental responses but do not restore habit formation in a rat model of Parkinsonism. Neuroscience. 2010;168:477–486. doi: 10.1016/j.neuroscience.2010.03.047. [DOI] [PubMed] [Google Scholar]
- Flores-Hernandez J, Cepeda C, Hernandez-Echeagaray E, Calvert CR, Jokel ES, Fienberg AA, Greengard P, Levine MS. Dopamine Enhancement of NMDA Currents in Dissociated Medium-Sized Striatal Neurons: Role of D1 Receptors and DARPP-32. J Neurophysiol. 2002;88:3010–3020. doi: 10.1152/jn.00361.2002. [DOI] [PubMed] [Google Scholar]
- Floresco SB. Dopaminergic regulation of limbic-striatal interplay. J Psychiatry Neurosci. 2007;32:400–411. [PMC free article] [PubMed] [Google Scholar]
- Gerdeman GL, Ronesi J, Lovinger DM. Postsynaptic endocannabinoid release is critical to long-term depression in the striatum. Nat Neurosci. 2002;5:446–451. doi: 10.1038/nn832. [DOI] [PubMed] [Google Scholar]
- Gerfen CR. Molecular effects of dopamine on striatal-projection pathways. Trends Neurosci. 2000;23:S64–S70. doi: 10.1016/s1471-1931(00)00019-7. [DOI] [PubMed] [Google Scholar]
- Gerfen CR, Engber TM, Mahan LC, Susel Z, Chase TN, Monsma FJ, Jr, Sibley DR. D1 and D2 dopamine receptor-regulated gene expression of striatonigral and striatopallidal neurons. Science. 1990;250:1429–1432. doi: 10.1126/science.2147780. [DOI] [PubMed] [Google Scholar]
- Girault JA, Valjent E, Caboche J, Herve D. ERK2: a logical AND gate critical for drug-induced plasticity? Curr Opin Pharmacol. 2007;7:77–85. doi: 10.1016/j.coph.2006.08.012. [DOI] [PubMed] [Google Scholar]
- Graybiel AM. Habits, Rituals, and the Evaluative Brain. Annual Review of Neuroscience. 2008;31:359–387. doi: 10.1146/annurev.neuro.29.051605.112851. [DOI] [PubMed] [Google Scholar]
- Green TA, Alibhai IN, Hommel JD, DiLeone RJ, Kumar A, Theobald DE, Neve RL, Nestler EJ. Induction of inducible cAMP early repressor expression in nucleus accumbens by stress or amphetamine increases behavioral responses to emotional stimuli. J Neurosci. 2006a;26:8235–8242. doi: 10.1523/JNEUROSCI.0880-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Green TA, Alibhai IN, Hommel JD, DiLeone RJ, Kumar A, Theobald DE, Neve RL, Nestler EJ. Induction of Inducible cAMP Early Repressor Expression in Nucleus Accumbens by Stress or Amphetamine Increases Behavioral Responses to Emotional Stimuli. J. Neurosci. 2006b;26:8235–8242. doi: 10.1523/JNEUROSCI.0880-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Greengard P. The neurobiology of slow synaptic transmission. Science. 2001;294:1024–1030. doi: 10.1126/science.294.5544.1024. [DOI] [PubMed] [Google Scholar]
- Greengard P, Allen PB, Nairn AC. Beyond the dopamine receptor: the DARPP-32/protein phosphatase-1 cascade. Neuron. 1999;23:435–447. doi: 10.1016/s0896-6273(00)80798-9. [DOI] [PubMed] [Google Scholar]
- Grewal SS, Horgan AM, York RD, Withers GS, Banker GA, Stork PJ. Neuronal calcium activates a Rap1 and B-Raf signaling pathway via the cyclic adenosine monophosphate-dependent protein kinase. J Biol Chem. 2000;275:3722–3728. doi: 10.1074/jbc.275.5.3722. [DOI] [PubMed] [Google Scholar]
- Groenewegen HJ, Berendse HW, Wolters JG, Lohman AH. The anatomical relationship of the prefrontal cortex with the striatopallidal system, the thalamus and the amygdala: evidence for a parallel organization. Prog Brain Res. 1990;85:95–116. doi: 10.1016/s0079-6123(08)62677-1. discussion 116–118. [DOI] [PubMed] [Google Scholar]
- Groenewegen HJ, Wright CI, Beijer AV, Voorn P. Convergence and segregation of ventral striatal inputs and outputs. Ann N Y Acad Sci. 1999;877:49–63. doi: 10.1111/j.1749-6632.1999.tb09260.x. [DOI] [PubMed] [Google Scholar]
- Haber SN, Fudge JL, McFarland NR. Striatonigrostriatal Pathways in Primates Form an Ascending Spiral from the Shell to the Dorsolateral Striatum. J. Neurosci. 2000;20:2369–2382. doi: 10.1523/JNEUROSCI.20-06-02369.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hall J, Parkinson JA, Connor TM, Dickinson A, Everitt BJ. Involvement of the central nucleus of the amygdala and nucleus accumbens core in mediating Pavlovian influences on instrumental behaviour. Eur J Neurosci. 2001;13:1984–1992. doi: 10.1046/j.0953-816x.2001.01577.x. [DOI] [PubMed] [Google Scholar]
- Hallett PJ, Spoelgen R, Hyman BT, Standaert DG, Dunah AW. Dopamine D1 Activation Potentiates Striatal NMDA Receptors by Tyrosine Phosphorylation-Dependent Subunit Trafficking. J. Neurosci. 2006;26:4690–4700. doi: 10.1523/JNEUROSCI.0792-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hammond LJ. The effect of contingency upon the appetitive conditioning of free-operant behavior. J Exp Anal Behav. 1980;34:297–304. doi: 10.1901/jeab.1980.34-297. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Heimer L, Alheid GF, de Olmos JS, Groenewegen HJ, Haber SN, Harlan RE, Zahm DS. The accumbens: beyond the core-shell dichotomy. J Neuropsychiatry Clin Neurosci. 1997;9:354–381. doi: 10.1176/jnp.9.3.354. [DOI] [PubMed] [Google Scholar]
- Hernandez PJ, Schiltz CA, Kelley AE. Dynamic shifts in corticostriatal expression patterns of the immediate early genes Homer 1a and Zif268 during early and late phases of instrumental training. Learn Mem. 2006;13:599–608. doi: 10.1101/lm.335006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hikosaka O. Basal ganglia mechanisms of reward-oriented eye movement. Ann N Y Acad Sci. 2007;1104:229–249. doi: 10.1196/annals.1390.012. [DOI] [PubMed] [Google Scholar]
- Hilario MR, Clouse E, Yin HH, Costa RM. Endocannabinoid signaling is critical for habit formation. Front Integr Neurosci. 2007;1:6. doi: 10.3389/neuro.07.006.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Holman EW. Some conditions for the dissociation of consummatory and instrumental behavior in rats. Learning & Motivation. 1975;6:358–366. [Google Scholar]
- Horvitz JC. Dopamine gating of glutamatergic sensorimotor and incentive motivational input signals to the striatum. Behavioural Brain Research. 2002;137:65–74. doi: 10.1016/s0166-4328(02)00285-1. [DOI] [PubMed] [Google Scholar]
- Horvitz JC. Stimulus-response and response-outcome learning mechanisms in the striatum. Behav Brain Res. 2009;199:129–140. doi: 10.1016/j.bbr.2008.12.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Huang YH, Lin Y, Brown TE, Han MH, Saal DB, Neve RL, Zukin RS, Sorg BA, Nestler EJ, Malenka RC, Dong Y. CREB Modulates the Functional Output of Nucleus Accumbens Neurons: A CRITICAL ROLE OF N-METHYL-D-ASPARTATE GLUTAMATE RECEPTOR (NMDAR) RECEPTORS. J Biol Chem. 2008;283:2751–2760. doi: 10.1074/jbc.M706578200. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hull CL. Principles of Behavior. New York: Appleton; 1943. [Google Scholar]
- Joel D, Weiner I. The organization of the basal ganglia-thalamocortical circuits: Open interconnected rather than closed segregated. Neuroscience. 1994;63:363–379. doi: 10.1016/0306-4522(94)90536-3. [DOI] [PubMed] [Google Scholar]
- Johnson AW, Bannerman D, Rawlins N, Sprengel R, Good MA. Targeted deletion of the GluR-1 AMPA receptor in mice dissociates general and outcome-specific influences of appetitive rewards on learning. Behav Neurosci. 2007;121:1192–1202. doi: 10.1037/0735-7044.121.6.1192. [DOI] [PubMed] [Google Scholar]
- Johnson AW, Bannerman DM, Rawlins NP, Sprengel R, Good MA. Impaired outcome-specific devaluation of instrumental responding in mice with a targeted deletion of the AMPA receptor glutamate receptor 1 subunit. J Neurosci. 2005;25:2359–2365. doi: 10.1523/JNEUROSCI.4146-04.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kerr JN, Wickens JR. Dopamine D-1/D-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro. J Neurophysiol. 2001;85:117–124. doi: 10.1152/jn.2001.85.1.117. [DOI] [PubMed] [Google Scholar]
- Kheirbek MA, Beeler JA, Ishikawa Y, Zhuang X. A cAMP Pathway Underlying Reward Prediction in Associative Learning. J. Neurosci. 2008;28:11401–11408. doi: 10.1523/JNEUROSCI.4115-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kheirbek MA, Britt JP, Beeler JA, Ishikawa Y, McGehee DS, Zhuang X. Adenylyl Cyclase Type 5 Contributes to Corticostriatal Plasticity and Striatum-Dependent Learning. J. Neurosci. 2009;29:12115–12124. doi: 10.1523/JNEUROSCI.3343-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Konorski . Integrative Activity of the Brain. Chicago: University of Chicago Press; 1967. [Google Scholar]
- Kravitz AV, Freeze BS, Parker PRL, Kay K, Thwin MT, Deisseroth K, Kreitzer AC. Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry. Nature. 2010;466:622–626. doi: 10.1038/nature09159. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kreitzer AC. Physiology and Pharmacology of Striatal Neurons. Annual Review of Neuroscience. 2009;32:127–147. doi: 10.1146/annurev.neuro.051508.135422. [DOI] [PubMed] [Google Scholar]
- Kreitzer AC, Malenka RC. Dopamine modulation of state-dependent endocannabinoid release and long-term depression in the striatum. J Neurosci. 2005;25:10537–10545. doi: 10.1523/JNEUROSCI.2959-05.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Leblois A, Boraud T, Meissner W, Bergman H, Hansel D. Competition between Feedback Loops Underlies Normal and Pathological Dynamics in the Basal Ganglia. J. Neurosci. 2006;26:3567–3583. doi: 10.1523/JNEUROSCI.5050-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lex A, Hauber W. Dopamine D1 and D2 receptors in the nucleus accumbens core and shell mediate Pavlovian-instrumental transfer. Learn. Mem. 2008;15:483–491. doi: 10.1101/lm.978708. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lex B, Hauber W. The role of dopamine in the prelimbic cortex and the dorsomedial striatum in instrumental conditioning. Cereb Cortex. 2010;20:873–883. doi: 10.1093/cercor/bhp151. [DOI] [PubMed] [Google Scholar]
- Liu F, Ma XH, Ule J, Bibb JA, Nishi A, DeMaggio AJ, Yan Z, Nairn AC, Greengard P. Regulation of cyclin-dependent kinase 5 and casein kinase 1 by metabotropic glutamate receptors. Proc Natl Acad Sci U S A. 2001;98:11062–11068. doi: 10.1073/pnas.191353898. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lobo MK, Cui Y, Ostlund SB, Balleine BW, William Yang X. Genetic control of instrumental conditioning by striatopallidal neuron-specific S1P receptor Gpr6. Nat Neurosci. 2007;10:1395–1397. doi: 10.1038/nn1987. [DOI] [PubMed] [Google Scholar]
- Lovinger DM. Neurotransmitter roles in synaptic modulation, plasticity and learning in the dorsal striatum. Neuropharmacology. 2010;58:951–961. doi: 10.1016/j.neuropharm.2010.01.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mao L, Tang Q, Samdani S, Liu Z, Wang JQ. Regulation of MAPK/ERK phosphorylation via ionotropic glutamate receptors in cultured rat striatal neurons. Eur J Neurosci. 2004;19:1207–1216. doi: 10.1111/j.1460-9568.2004.03223.x. [DOI] [PubMed] [Google Scholar]
- Mao L, Yang L, Tang Q, Samdani S, Zhang G, Wang JQ. The scaffold protein Homer1b/c links metabotropic glutamate receptor 5 to extracellular signal-regulated protein kinase cascades in neurons. J Neurosci. 2005;25:2741–2752. doi: 10.1523/JNEUROSCI.4360-04.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mazzucchelli C, Vantaggiato C, Ciamei A, Fasano S, Pakhotin P, Krezel W, Welzl H, Wolfer DP, Pages G, Valverde O, Marowsky A, Porrazzo A, Orban PC, Maldonado R, Ehrengruber MU, Cestari V, Lipp HP, Chapman PF, Pouyssegur J, Brambilla R. Knockout of ERK1 MAP kinase enhances synaptic plasticity in the striatum and facilitates striatal-mediated learning and memory. Neuron. 2002;34:807–820. doi: 10.1016/s0896-6273(02)00716-x. [DOI] [PubMed] [Google Scholar]
- McFarland NR, Haber SN. Convergent Inputs from Thalamic Motor Nuclei and Frontal Cortical Areas to the Dorsal Striatum in the Primate. J. Neurosci. 2000;20:3798–3813. doi: 10.1523/JNEUROSCI.20-10-03798.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
- McHaffie JG, Stanford TR, Stein BE, Coizet V, Redgrave P. Subcortical loops through the basal ganglia. Trends in Neurosciences. 2005;28:401–407. doi: 10.1016/j.tins.2005.06.006. [DOI] [PubMed] [Google Scholar]
- Mead AN, Stephens DN. Selective disruption of stimulus-reward learning in glutamate receptor gria1 knock-out mice. J Neurosci. 2003;23:1041–1048. doi: 10.1523/JNEUROSCI.23-03-01041.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Miles FJ, Everitt BJ, Dickinson A. Oral cocaine seeking by rats: action or habit? Behav Neurosci. 2003;117:927–938. doi: 10.1037/0735-7044.117.5.927. [DOI] [PubMed] [Google Scholar]
- Miller C, Marshall J. Involvement of the ERK 1/2 signaling pathway during cue-elicited drug-seeking. Society for Neuroscience Abstracts. 2004 Program No 576. [Google Scholar]
- Murschall A, Hauber W. Effects of a systemic AMPA/KA and NMDA receptor blockade on pavlovian-instrumental transfer. Psychopharmacology (Berl) 2005;182:290–296. doi: 10.1007/s00213-005-0073-9. [DOI] [PubMed] [Google Scholar]
- Murschall A, Hauber W. Inactivation of the ventral tegmental area abolished the general excitatory influence of Pavlovian cues on instrumental performance. Learn Mem. 2006;13:123–126. doi: 10.1101/lm.127106. [DOI] [PubMed] [Google Scholar]
- Nakano T, Doi T, Yoshimoto J, Doya K. A kinetic model of dopamine- and calcium-dependent striatal synaptic plasticity. PLoS Comput Biol. 2010;6:e1000670. doi: 10.1371/journal.pcbi.1000670. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nisenbaum ES, Wilson CJ. Potassium currents responsible for inward and outward rectification in rat neostriatal spiny projection neurons. J. Neurosci. 1995;15:4449–4463. doi: 10.1523/JNEUROSCI.15-06-04449.1995. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nishi A, Bibb JA, Matsuyama S, Hamada M, Higashi H, Nairn AC, Greengard P. Regulation of DARPP-32 dephosphorylation at PKA- and Cdk5-sites by NMDA and AMPA receptors: distinct roles of calcineurin and protein phosphatase-2A. J Neurochem. 2002;81:832–841. doi: 10.1046/j.1471-4159.2002.00876.x. [DOI] [PubMed] [Google Scholar]
- Ortiz J, Harris HW, Guitart X, Terwilliger RZ, Haycock JW, Nestler EJ. Extracellular signal-regulated protein kinases (ERKs) and ERK kinase (MEK) in brain: regional distribution and regulation by chronic morphine. J Neurosci. 1995;15:1285–1297. doi: 10.1523/JNEUROSCI.15-02-01285.1995. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ostlund SB, Balleine BW. Lesions of medial prefrontal cortex disrupt the acquisition but not the expression of goal-directed learning. J Neurosci. 2005;25:7763–7770. doi: 10.1523/JNEUROSCI.1921-05.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ostlund SB, Winterbauer NE, Balleine BW. Evidence of action sequence chunking in goal-directed instrumental conditioning and its dependence on the dorsomedial prefrontal cortex. Journal of Neuroscience. 2009;29:8280–8287. doi: 10.1523/JNEUROSCI.1176-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Packard MG. Dissociation of multiple memory systems by posttraining intracerebral injections of glutamate. Psychobiology. 1999;27:40–50. [Google Scholar]
- Packard MG, Knowlton BJ. Learning and memory functions of the basal ganglia. Annual Review of Neuroscience. 2002;25:563–593. doi: 10.1146/annurev.neuro.25.112701.142937. [DOI] [PubMed] [Google Scholar]
- Packard MG, Teather LA. Posttraining injections of MK-801 produce a time-dependent impairment of memory in two water maze tasks. Neurobiology of Learning and Memory. 1997;68:42–50. doi: 10.1006/nlme.1996.3762. [DOI] [PubMed] [Google Scholar]
- Packard MG, White NM. DISSOCIATION OF HIPPOCAMPUS AND CAUDATE-NUCLEUS MEMORY-SYSTEMS BY POSTTRAINING INTRACEREBRAL INJECTION OF DOPAMINE AGONISTS. Behavioral Neuroscience. 1991;105:295–306. doi: 10.1037//0735-7044.105.2.295. [DOI] [PubMed] [Google Scholar]
- Partridge JG, Tang K-C, Lovinger DM. Regional and Postnatal Heterogeneity of Activity-Dependent Long-Term Changes in Synaptic Efficacy in the Dorsal Striatum. J Neurophysiol. 2000;84:1422–1429. doi: 10.1152/jn.2000.84.3.1422. [DOI] [PubMed] [Google Scholar]
- Paul S, Nairn AC, Wang P, Lombroso PJ. NMDA-mediated activation of the tyrosine phosphatase STEP regulates the duration of ERK signaling. Nat Neurosci. 2003;6:34–42. doi: 10.1038/nn989. [DOI] [PubMed] [Google Scholar]
- Paul S, Olausson P, Venkitaramani DV, Ruchkina I, Moran TD, Tronson N, Mills E, Hakim S, Salter MW, Taylor JR, Lombroso PJ. The Striatal-Enriched Protein Tyrosine Phosphatase Gates Long-Term Potentiation and Fear Memory in the Lateral Amygdala. Biological Psychiatry. 2007;61:1049–1061. doi: 10.1016/j.biopsych.2006.08.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pennartz CMA, Berke JD, Graybiel AM, Ito R, Lansink CS, van der Meer M, Redish AD, Smith KS, Voorn P. Corticostriatal Interactions during Learning, Memory Processing, and Decision Making. J. Neurosci. 2009;29:12831–12838. doi: 10.1523/JNEUROSCI.3177-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Perkinton MS, Sihra TS, Williams RJ. Ca2+-Permeable AMPA Receptors Induce Phosphorylation of cAMP Response Element-Binding Protein through a Phosphatidylinositol 3-Kinase-Dependent Stimulation of the Mitogen-Activated Protein Kinase Signaling Cascade in Neurons. J. Neurosci. 1999;19:5861–5874. doi: 10.1523/JNEUROSCI.19-14-05861.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pittenger C, Fasano S, Mazzocchi-Jones D, Dunnett SB, Kandel ER, Brambilla R. Impaired Bidirectional Synaptic Plasticity and Procedural Memory Formation in Striatum-Specific cAMP Response Element-Binding Protein-Deficient Mice. J. Neurosci. 2006;26:2808–2813. doi: 10.1523/JNEUROSCI.5406-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Redgrave P, Gurney K, Reynolds J. What is reinforced by phasic dopamine signals? Brain Res Rev. 2008;58:322–339. doi: 10.1016/j.brainresrev.2007.10.007. [DOI] [PubMed] [Google Scholar]
- Redgrave P, Rodriguez M, Smith Y, Rodriguez-Oroz MC, Lehericy S, Bergman H, Agid Y, DeLong MR, Obeso JA. Goal-directed and habitual control in the basal ganglia: implications for Parkinson's disease. Nat Rev Neurosci. 2010;11:760–772. doi: 10.1038/nrn2915. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Reynolds JNJ, Hyland BI, Wickens JR. A cellular mechanism of reward-related learning. Nature. 2001;413:67–70. doi: 10.1038/35092560. [DOI] [PubMed] [Google Scholar]
- Robbins TW, Everitt BJ. Limbic-striatal memory systems and drug addiction. Neurobiol Learn Mem. 2002;78:625–636. doi: 10.1006/nlme.2002.4103. [DOI] [PubMed] [Google Scholar]
- Robinson S, Rainwater AJ, Hnasko TS, Palmiter RD. Viral restoration of dopamine signaling to the dorsal striatum restores instrumental conditioning to dopamine-deficient mice. Psychopharmacology (Berl) 2007;191:567–578. doi: 10.1007/s00213-006-0579-9. [DOI] [PubMed] [Google Scholar]
- Robinson S, Sotak BN, During MJ, Palmiter RD. Local dopamine production in the dorsal striatum restores goal-directed behavior in dopamine-deficient mice. Behav Neurosci. 2006;120:196–200. doi: 10.1037/0735-7044.120.1.000. [DOI] [PubMed] [Google Scholar]
- Salamone JD, Arizzi MN, Sandoval MD, Cervone KM, Aberman J. E Dopamine antagonists alter response allocation but do not suppress appetite for food in rats: contrast between the effects of SKF 83566, raclopride, and fenfluramine on a concurrent choice task. Psychopharmacology (Berl) 2002;160:371–380. doi: 10.1007/s00213-001-0994-x. [DOI] [PubMed] [Google Scholar]
- Salamone JD, Correa M. Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine. Behavioural Brain Research. 2002;137:3–25. doi: 10.1016/s0166-4328(02)00282-6. [DOI] [PubMed] [Google Scholar]
- Schultz W. Multiple dopamine functions at different time courses. Annual Review of Neuroscience. 2007;30:259–288. doi: 10.1146/annurev.neuro.28.061604.135722. [DOI] [PubMed] [Google Scholar]
- Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275:1593–1599. doi: 10.1126/science.275.5306.1593. [DOI] [PubMed] [Google Scholar]
- Schultz W, Dickinson A. Neuronal Coding of Prediction Errors. Annual Review of Neuroscience. 2000;23:473–500. doi: 10.1146/annurev.neuro.23.1.473. [DOI] [PubMed] [Google Scholar]
- Sesack SR, Carr DB, Omelchenko N, Pinto A. Anatomical substrates for glutamate-dopamine interactions: evidence for specificity of connections and extrasynaptic actions. Ann N Y Acad Sci. 2003;1003:36–52. doi: 10.1196/annals.1300.066. [DOI] [PubMed] [Google Scholar]
- Sgambato V, Pages C, Rogard M, Besson M-J, Caboche J. Extracellular Signal-Regulated Kinase (ERK) Controls Immediate Early Gene Induction on Corticostriatal Stimulation. J. Neurosci. 1998;18:8814–8825. doi: 10.1523/JNEUROSCI.18-21-08814.1998. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shaw-Lutchman TZ, Barrot M, Wallace T, Gilden L, Zachariou V, Impey S, Duman RS, Storm D, Nestler EJ. Regional and Cellular Mapping of cAMP Response Element-Mediated Transcription during Naltrexone-Precipitated Morphine Withdrawal. J. Neurosci. 2002;22:3663–3672. doi: 10.1523/JNEUROSCI.22-09-03663.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shen W, Flajolet M, Greengard P, Surmeier DJ. Dichotomous dopaminergic control of striatal synaptic plasticity. Science. 2008;321:848–851. doi: 10.1126/science.1160575. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shiflett MW, Balleine BW. Contributions of ERK signaling in the striatum to instrumental learning and performance. Behavioural Brain Research. 2011;218:240–247. doi: 10.1016/j.bbr.2010.12.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shiflett MW, Brown RA, Balleine BW. Acquisition and performance of goal-directed instrumental actions depends on ERK signaling in distinct regions of dorsal striatum in rats. J Neurosci. 2010;30:2951–2959. doi: 10.1523/JNEUROSCI.1778-09.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shiflett MW, Martini RP, Mauna JC, Foster RL, Peet E, Thiels E. Cue-elicited reward-seeking requires extracellular signal-regulated kinase activation in the nucleus accumbens. Journal of Neuroscience. 2008;28:1434–1443. doi: 10.1523/JNEUROSCI.2383-07.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shiflett MW, Mauna JC, Chipman AM, Peet E, Thiels E. Appetitive Pavlovian conditioned stimuli increase CREB phosphorylation in the nucleus accumbens. Neurobiol Learn Mem. 2009;92:451–454. doi: 10.1016/j.nlm.2009.02.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Smith-Roe SL, Kelley AE. Coincident Activation of NMDA and Dopamine D1 Receptors within the Nucleus Accumbens Core Is Required for Appetitive Instrumental Learning. J. Neurosci. 2000;20:7737–7742. doi: 10.1523/JNEUROSCI.20-20-07737.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Smith Y, Bevan MD, Shink E, Bolam JP. Microcircuitry of the direct and indirect pathways of the basal ganglia. Neuroscience. 1998;86:353–387. doi: 10.1016/s0306-4522(98)00004-9. [DOI] [PubMed] [Google Scholar]
- Smith Y, Raju DV, Pare J-F, Sidibe M. The thalamostriatal system: a highly specific network of the basal ganglia circuitry. Trends in Neurosciences. 2004;27:520–527. doi: 10.1016/j.tins.2004.07.004. [DOI] [PubMed] [Google Scholar]
- Song WJ, Tkatch T, Baranauskas G, Ichinohe N, Kitai ST, Surmeier DJ. Somatodendritic depolarization-activated potassium currents in rat neostriatal cholinergic interneurons are predominantly of the A type and attributable to coexpression of Kv4.2 and Kv4.1 subunits. J Neurosci. 1998;18:3124–3137. doi: 10.1523/JNEUROSCI.18-09-03124.1998. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stalnaker TA, Calhoon GG, Ogawa M, Roesch MR, Schoenbaum G. Neural correlates of stimulus-response and response-outcome associations in dorsolateral versus dorsomedial striatum. Frontiers in Integrative Neuroscience. 2010;4 doi: 10.3389/fnint.2010.00012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Surmeier DJ, Bargas J, Hemmings HC, Jr, Nairn AC, Greengard P. Modulation of calcium currents by a D1 dopaminergic protein kinase/phosphatase cascade in rat neostriatal neurons. Neuron. 1995;14:385–397. doi: 10.1016/0896-6273(95)90294-5. [DOI] [PubMed] [Google Scholar]
- Surmeier DJ, Ding J, Day M, Wang Z, Shen W. D1 and D2 dopamine-receptor modulation of striatal glutamatergic signaling in striatal medium spiny neurons. Trends in Neurosciences. 2007;30:228–235. doi: 10.1016/j.tins.2007.03.008. [DOI] [PubMed] [Google Scholar]
- Surmeier DJ, Eberwine J, Wilson CJ, Cao Y, Stefani A, Kitai ST. Dopamine receptor subtypes colocalize in rat striatonigral neurons. Proc Natl Acad Sci U S A. 1992;89:10178–10182. doi: 10.1073/pnas.89.21.10178. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Svenningsson P, Nishi A, Fisone G, Girault JA, Nairn AC, Greengard P. DARPP-32: an integrator of neurotransmission. Annu Rev Pharmacol Toxicol. 2004;44:269–296. doi: 10.1146/annurev.pharmtox.44.101802.121415. [DOI] [PubMed] [Google Scholar]
- Thiels E, Kanterewicz BI, Norman ED, Trzaskos JM, Klann E. Long-term depression in the adult hippocampus in vivo involves activation of extracellular signal-regulated kinase and phosphorylation of Elk-1. J Neurosci. 2002;22:2054–2062. doi: 10.1523/JNEUROSCI.22-06-02054.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Thorn CA, Atallah H, Howe M, Graybiel AM. Differential Dynamics of Activity Changes in Dorsolateral and Dorsomedial Striatal Loops during Learning. Neuron. 2010;66:781–795. doi: 10.1016/j.neuron.2010.04.036. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tkatch T, Baranauskas G, Surmeier DJ. Kv4.2 mRNA abundance and A-type K(+) current amplitude are linearly related in basal ganglia and basal forebrain neurons. J Neurosci. 2000;20:579–588. doi: 10.1523/JNEUROSCI.20-02-00579.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tropea TF, Kosofsky BE, Rajadhyaksha AM. Enhanced CREB and DARPP-32 phosphorylation in the nucleus accumbens and CREB, ERK, and GluR1 phosphorylation in the dorsal hippocampus is associated with cocaine-conditioned place preference behavior. J Neurochem. 2008;106:1780–1790. doi: 10.1111/j.1471-4159.2008.05518.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Valjent E, Corvol J-C, Pages C, Besson M-J, Maldonado R, Caboche J. Involvement of the Extracellular Signal-Regulated Kinase Cascade for Cocaine-Rewarding Properties. J. Neurosci. 2000;20:8701–8709. doi: 10.1523/JNEUROSCI.20-23-08701.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Valjent E, Pascoli V, Svenningsson P, Paul S, Enslen H, Corvol JC, Stipanovich A, Caboche J, Lombroso PJ, Nairn AC, Greengard P, Herve D, Girault JA. Regulation of a protein phosphatase cascade allows convergent dopamine and glutamate signals to activate ERK in the striatum. Proc Natl Acad Sci U S A. 2005;102:491–496. doi: 10.1073/pnas.0408305102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Varga AW, Yuan LL, Anderson AE, Schrader LA, Wu GY, Gatchel JR, Johnston D, Sweatt JD. Calcium-calmodulin-dependent kinase II modulates Kv4.2 channel expression and upregulates neuronal A-type potassium currents. J Neurosci. 2004;24:3643–3654. doi: 10.1523/JNEUROSCI.0154-04.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Voorn P, Vanderschuren LJMJ, Groenewegen HJ, Robbins TW, Pennartz CMA. Putting a spin on the dorsal-ventral divide of the striatum. Trends in Neurosciences. 2004;27:468–474. doi: 10.1016/j.tins.2004.06.006. [DOI] [PubMed] [Google Scholar]
- Walters CL, Blendy JA. Different requirements for cAMP response element binding protein in positive and negative reinforcing properties of drugs of abuse. Journal of Neuroscience. 2001;21:9438–9444. doi: 10.1523/JNEUROSCI.21-23-09438.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wang JQ, Fibuch EE, Mao L. Regulation of mitogen-activated protein kinases by glutamate receptors. J Neurochem. 2007;100:1–11. doi: 10.1111/j.1471-4159.2006.04208.x. [DOI] [PubMed] [Google Scholar]
- White NM. Some highlights of research on the effects of caudate nucleus lesions over the past 200 years. Behav Brain Res. 2009;199:3–23. doi: 10.1016/j.bbr.2008.12.003. [DOI] [PubMed] [Google Scholar]
- Wickens JR, Budd CS, Hyland BI, Arbuthnott GW. Striatal contributions to reward and decision making - Making sense of regional variations in a reiterated processing matrix. Reward and Decision Making in Corticobasal Ganglia Networks. 2007;1104:192–212. doi: 10.1196/annals.1390.016. [DOI] [PubMed] [Google Scholar]
- Wiltgen BJ, Law M, Ostlund S, Mayford M, Balleine BW. The influence of Pavlovian cues on instrumental performance is mediated by CaMKII activity in the striatum. Eur J Neurosci. 2007;25:2491–2497. doi: 10.1111/j.1460-9568.2007.05487.x. [DOI] [PubMed] [Google Scholar]
- Wu JH, Corwin JV, Reep RL. Organization of the corticostriatal projection from rat medial agranular cortex to far dorsolateral striatum. Brain research. 2009;1280:69–76. doi: 10.1016/j.brainres.2009.05.044. [DOI] [PubMed] [Google Scholar]
- Wyvell CL, Berridge KC. Intra-Accumbens Amphetamine Increases the Conditioned Incentive Salience of Sucrose Reward: Enhancement of Reward "Wanting" without Enhanced "Liking" or Response Reinforcement. J. Neurosci. 2000;20:8122–8130. doi: 10.1523/JNEUROSCI.20-21-08122.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Xing J, Ginty DD, Greenberg ME. Coupling of the RAS-MAPK pathway to gene activation by RSK2, a growth factor-regulated CREB kinase. Science. 1996;273:959–963. doi: 10.1126/science.273.5277.959. [DOI] [PubMed] [Google Scholar]
- Yin HH, Davis MI, Ronesi JA, Lovinger DM. The Role of Protein Synthesis in Striatal Long-Term Depression. J. Neurosci. 2006;26:11811–11820. doi: 10.1523/JNEUROSCI.3196-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yin HH, Knowlton BJ. The role of the basal ganglia in habit formation. Nat Rev Neurosci. 2006;7:464–476. doi: 10.1038/nrn1919. [DOI] [PubMed] [Google Scholar]
- Yin HH, Knowlton BJ, Balleine BW. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur J Neurosci. 2004;19:181–189. doi: 10.1111/j.1460-9568.2004.03095.x. [DOI] [PubMed] [Google Scholar]
- Yin HH, Knowlton BJ, Balleine BW. Blockade of NMDA receptors in the dorsomedial striatum prevents action-outcome learning in instrumental conditioning. Eur J Neurosci. 2005a;22:505–512. doi: 10.1111/j.1460-9568.2005.04219.x. [DOI] [PubMed] [Google Scholar]
- Yin HH, Mulcare SP, Hilario MRF, Clouse E, Holloway T, Davis MI, Hansson AC, Lovinger DM, Costa RM. Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill. Nat Neurosci. 2009;12:333–341. doi: 10.1038/nn.2261. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yin HH, Ostlund SB, Balleine BW. Reward-guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico-basal ganglia networks. Eur J Neurosci. 2008;28:1437–1448. doi: 10.1111/j.1460-9568.2008.06422.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yin HH, Ostlund SB, Knowlton BJ, Balleine BW. The role of the dorsomedial striatum in instrumental conditioning. Eur J Neurosci. 2005b;22:513–523. doi: 10.1111/j.1460-9568.2005.04218.x. [DOI] [PubMed] [Google Scholar]
- Zahm DS. An integrative neuroanatomical perspective on some subcortical substrates of adaptive responding with emphasis on the nucleus accumbens. Neurosci Biobehav Rev. 2000;24:85–105. doi: 10.1016/s0149-7634(99)00065-2. [DOI] [PubMed] [Google Scholar]
- Zahm DS, Brog JS. On the significance of subterritories in the "accumbens" part of the rat ventral striatum. Neuroscience. 1992;50:751–767. doi: 10.1016/0306-4522(92)90202-d. [DOI] [PubMed] [Google Scholar]