Extracellular Signal-Related Kinase Activation During Natural Reward Learning: A Physiological Role for Phasic Nucleus Accumbens Dopamine?

The ability to learn from and act on reward-associated cues is a critical component of foraging behavior and, ultimately, survival. In addition to serving as a means by which future rewards can be predicted, conditioned stimuli can convey information about the size, delay, location, desirability, and risks associated with a reward and thereby facilitate its acquisition. Moreover, these stimuli often develop their own motivational properties, as can be demonstrated by a phenomenon known as pavlovian-to-instrumental transfer (PIT). Here, an animal that has learned in separate behavioral sessions to associate a conditioned stimulus (CS) with a reward (a pavlovian relationship) and to emit a specified response which leads to reward delivery (an instrumental or operant relationship) is re-exposed to the pavlovian CS during the operant context. Although there was no explicit connection between the CS and the operant response during training, presentation of the CS at this stage reliably enhances reward-seeking operant behavior, indicating that it has acquired a motivational component as a result of conditioning (Everitt and Robbins, 2005).

A wide variety of reward-related behaviors have now been shown to be mediated by a complex neural circuit involving the nucleus accumbens (NAc) and its dopamine innervation from the ventral tegmental area. Microinjection of a dopamine antagonist into the NAc disrupts both the acquisition and expression of learned associations between rewards and predictive conditioned stimuli, and lesions to the NAc core block PIT (Di Ciano et al., 2001; Hall et al., 2001). Moreover, antagonism of NMDA receptors in the NAc core subregion produces deficits during the initial stages of reward-based pavlovian learning (Di Ciano et al., 2001), suggesting that glutamate signaling at this receptor is critical for the acquisition of cue–reward associations.

Although the plastic circuit-level changes underlying reward learning have received extensive attention, the intracellular pathways that mediate such adaptations remain unclear. One candidate molecule is extracellular signal-related kinase (ERK), a mitogen-activated protein kinase that is expressed in the NAc and activated during exposure to drugs such as cocaine and amphetamine (Valjent et al., 2005). Importantly, ERK phosphorylation in the NAc requires coactivation of NMDA glutamate receptors and D₁ dopamine receptors (Valjent et al., 2005), suggesting that ERK operates as a functional coincidence detector that can integrate highly specific glutamatergic input with the more general reward prediction information provided by dopamine neurons (Schultz et al., 1997; Girault et al., 2007). However, despite this hypothesis, the ability of natural reward-related stimuli to promote ERK activation in the NAc has not been examined.

In their recent manuscript published in The Journal of Neuroscience, Shiflett et al. (2008) conclusively demonstrate for the first time that ERK is activated in the NAc during natural reward learning. Animals were first trained in a conditioning paradigm in which sucrose rewards were repeatedly paired with the presentation of a 90-s audio CS, and phosphorylated ERK levels were measured after the first and fourth conditioning sessions. Increased ERK phosphorylation was present after the fourth (but not first) conditioning session, and corresponded to the development of a learned conditioned response which signaled that animals had acquired the cue–reward association [Shiflett et al. (2008), their Fig. 1A, http://www.jneurosci.org/cgi/content/full/28/6/1434/F1)]. Importantly, although this conditioning-induced enhancement of ERK activity was transient (lasting <3 h), it was specific to sessions in which the CS was presented. ERK phosphorylation levels in trained animals that were exposed only to the experimental context (yet received no cues or rewards) was no different from control levels [Shiflett et al. (2008), their Fig. 2B (http://www.jneurosci.org/cgi/content/full/28/6/1434/F2)]. Furthermore, ERK was activated in both the core and shell subregions of the NAc but not in the dorsal striatum [Shiflett et al. (2008), their Figs. 2C (http://www.jneurosci.org/cgi/content/full/28/6/1434/F2), 3 (http://www.jneurosci.org/cgi/content/full/28/6/1434/F3)]. Thus, the ability of a reward-paired cue to evoke increased ERK phosphorylation was both temporally and regionally specific.

Figure 1.

Simplified representation of key nucleus accumbens inputs and putative molecular pathways leading to ERK activation. A, Precise information about a CS or context and its associated outcome is encoded within brain regions such as the basolateral amygdala (BLA), prefrontal cortex (PFC), and hippocampus (HPC), all of which could communicate this information to the NAc via increased or altered glutamatergic activity. B, Dopamine neurons within the ventral tegmental area (VTA) carry a reward prediction error signal to the NAc, producing a transient increase in NAc dopamine concentration when reward-predictive cues are presented. C, These signals arrive at dopamine D₁ and glutamate NMDA receptors located on medium spiny neurons within the NAc. At D1 G-protein-coupled receptors (D1DR), dopamine begins an intracellular cascade that involves adenylyl cyclase (AC)-dependent cAMP and PKA activation, DARPP-32 phosphorylation, and a protein phosphatase-1 (PP-1) dependent increase in ERK phosphorylation. Likewise, calcium influx produced by NMDAR activation promotes ERK phosphorylation via MEK. Together, these pathways may mediate the increases in ERK activity produced during appetitive conditioning and pavlovian-to-instrumental transfer. The molecular substrates and transcription factors that ERK acts on to produce short- and long-term cellular changes are presently unknown. Data in A and B are theoretical but are based on known patterns of cellular activity or neurotransmitter release.

Shiflett et al. (2008) also report that ERK was strongly activated after a session in which the presentation of reward-paired conditioned stimuli were found to increase operant reward seeking (i.e., positive pavlovian-to-instrumental transfer occurred) [Shiflett et al. (2008), their Fig. 5 (http://www.jneurosci.org/cgi/content/full/28/6/1434/F5)]. To assess the functional role of ERK phosphorylation in the NAc during PIT, a separate group of animals received both pavlovian and operant training sessions. After training, animals were tested for PIT during bilateral intra-NAc infusion of either vehicle or the mitogen-activated protein kinase/ERK kinase (MEK) inhibitor U0126 (which was independently shown to reduce ERK phosphorylation in the NAc). In the control group, animals made significantly more operant responses for a food reward when conditioned stimuli were presented than in an identical time block before CS presentation. Conversely, animals that received microinjections of U0126 in the NAc exhibited no change in response frequency during exposure to the CS, indicating that PIT did not occur [Shiflett et al. (2008), their Fig. 4A,B (http://www.jneurosci.org/cgi/content/full/28/6/1434/F4)]. However, U0126 did not alter the overall number of operant responses or conditioned food-cup approaches. Thus, although ERK inhibition did not disrupt the expression of previously acquired stimulus–reward or action–outcome associations, it did prevent the ability of conditioned stimuli to promote reward-seeking responses.

As mentioned above, one hypothesis is that ERK functions as a dopamine–glutamate coincidence detector at NAc synapses (Girault et al., 2007). Within this framework, the observation that ERK becomes activated in the NAc as a result of conditioning may help to elucidate the functional significance of phasic NAc dopamine release evoked by CS presentation (Day et al., 2007). Under basal conditions, the extracellular concentration of dopamine in the NAc is likely insufficient to activate low-affinity D₁ dopamine receptors located on medium spiny neurons (Watson et al., 2006). However, the robust phasic elevation in NAc dopamine concentration produced by reward-paired conditioned stimuli may be adequate to activate a large proportion of D₁ receptors, triggering a signaling cascade that includes dopamine and cAMP-regulated phosphoprotein, 32 kDa (DARPP-32)-mediated inhibition of ERK dephosphorylation (Fig. 1) (Girault et al., 2007). Simultaneously, glutamatergic inputs from the basolateral amygdala, hippocampus, and prefrontal cortex, which carry precise environmental and contextual information to the NAc, could activate NMDA receptors and produce MEK-induced phosphorylation of ERK (Fig. 1). Although either of these inputs alone may be ineffective at producing robust ERK phosphorylation, their combination could result in significant (and behaviorally meaningful) ERK activity. This mechanism would allow convergent streams of information to initiate both short-term and long-term plasticity at corticostriatal synapses, which could conceivably underlie changes in the motivational significance (and response-eliciting properties) of reward-paired conditioned stimuli.

Although the results presented by Shiflett et al. (2008) advance our understanding of the neural mechanisms of goal-directed behavior, several key issues remain unresolved. First, whereas numerous studies have suggested that the core and shell subregions of the NAc mediate separate aspects of reward-related learning (Di Chiara, 2002), appetitive conditioning increased ERK activation equally in these areas (Shiflett et al., 2008). Precisely how ERK operates differently within core- and shell-associated neural circuits, and what role this plays in the dissociable behavioral function of these regions, is still unclear. In the study by Shiflett et al. (2008), ERK phosphorylation was pharmacologically inhibited only in the NAc core [Shiflett et al. (2008), their Fig. 4E, (http://www.jneurosci.org/cgi/content/full/28/6/1434/F4)], making it difficult to assess potential subregion differences. Therefore, future research is needed to investigate whether ERK phosphorylation within these subregions promotes or controls different aspects of learning and/or motivation with respect to natural rewards. Second, although ERK disruption specifically alters the motivational transfer of pavlovian cues and not the performance of conditioned appetitive responses or instrumental behavior, it remains possible that ERK activation is critical for the acquisition of these learned behaviors. Indeed, given that NMDAR or D₁R receptor blockade early in appetitive conditioning both disrupt learning (Di Ciano et al., 2001), and that ERK activation requires stimulation at both of these receptors (Valjent et al., 2005), it seems likely that ERK phosphorylation plays a role in the acquisition of conditioned responses to predicted rewards. Finally, the precise molecular substrates that ERK acts through to mediate behaviorally relevant short-term and long-term cellular changes also remain unknown. In brain nuclei other than the NAc, ERK activation has been associated with changes in a Shal-type voltage-dependent potassium channel (Kv4.2) and enhanced neuronal excitability (Adams et al., 2000). However, ERK phosphorylation has also been linked to a number of gene transcription factors, including Zif268 and cAMP-responsive element-binding protein (Girault et al., 2007). Determining which of these molecular substrates are important for the creation and long-term maintenance of pavlovian or instrumental memories will not only dramatically enhance our knowledge of natural reward learning, but will also provide a valuable point of comparison for models of neural plasticity associated with drug addiction.