How Does Dopamine Release in the Nucleus Accumbens Core Relate to Encoding of a Pavlovian Incentive Stimulus?

Sex, drugs, and other pleasurable stimuli activate reward circuits in the midbrain. The ventral tegmental area (VTA) and nucleus accumbens (NAc) are key components of these circuits. Reward processing in these brain areas is studied using simple conditioning paradigms. During the early stages of training in which a stimulus (e.g., a tone) signals delivery of a reward (e.g., a food pellet to a hungry animal), VTA neurons initially respond to unexpected presentations of reward. However, as training continues, these neurons cease responding to reward presentations and instead respond to presentations of the conditioned stimulus (CS). This migration of the phasic neuronal response from the reward to the CS is correlated with the emergence of anticipatory conditioned behaviors (e.g., approaching the site of reward delivery). However, precisely what is represented by release of dopamine in the midbrain continues to be debated. Some argue that phasic activation of VTA neurons encode a teaching signal that supports learning about relationships between environmental stimuli (prediction error hypothesis; Schultz, 1998). In contrast, others suggest that activation of these neurons, and consequently, phasic dopamine release in the NAc, is critical for assigning incentive value to reward-predictive stimuli (incentive value hypothesis; Berridge, 2007). These two hypotheses differ with respect to the aspects of a CS–reward relationship encoded by NAc dopamine: the prediction error hypothesis emphasizes dopamine encoding of informational rather than motivational value; whereas the incentive value hypothesis emphasizes dopamine encoding of motivational rather than informational value.

Because conditioning of incentive value is intimately related to predictive learning of a CS–reward relationship, it is difficult to distinguish between dopamine encoding of one process or the other. In a recent paper, Clark et al. (2013) attempt to draw this distinction. Using fast-scan cyclic voltammetry in rats, the authors examined changes in NAc core (NAcc) dopamine release during acquisition and maintenance of Pavlovian conditioned behavior. In their task, rats were exposed to 8 s presentations of both a lever and a light (the CS); after 8 s, the lever was retracted and a single food pellet delivered to a food cup [the unconditioned stimulus (US)], regardless of whether the rat pressed the lever. The authors measured responses directed toward the lever (i.e., lever presses) as an index of CS incentive value. They show that the number of lever presses (as well as the proportion of trials on which a lever press was recorded) reached an asymptote by the fifth training session; that CS-evoked dopamine release in the NAcc also increased over this time; and further, that the behavioral and pharmacological changes were correlated. However, across 10 additional sessions of postasymptotic training, lever pressing ceased to be related to NAcc dopamine release: the CS continued to elicit lever pressing but CS-evoked dopamine release in the NAcc declined.

Based on this apparent dissociation between lever pressing and NAcc dopamine release, Clark et al. (2013) inferred a decrease in dopamine encoding of Pavlovian incentive stimuli across extended training. In keeping with this view, systemic administration of a dopamine D1 receptor antagonist reduced lever pressing after 15 sessions of training instead of after five sessions. It was also shown that dopamine release in the NAcc at the later time point was related to the structure of the task itself, with CSs presented after longer intertrial intervals being more predictable and evoking less NAcc dopamine: modulation of NAcc dopamine release by trial predictability was not evident after just five sessions. The authors take these findings to imply a shift in what is encoded by dopamine across postasymptotic training: dopamine encoding of incentive value diminishes and concomitant with this change, acquisition of task knowledge causes a decline in NAcc dopamine release.

Although the findings by Clark et al. (2013) suggest that dopamine encoding of incentive value diminishes across extended training, they are also consistent with alternative explanations. For example, across training in which reward is signaled by a fixed-duration CS, animals redistribute their behavior such that responding is suppressed in the period immediately following CS onset but steadily increases up to the time of US delivery. The relative suppression of responding following CS onset is referred to as inhibition of delay and has a rich experimental history (Rescorla, 1967). Given that the CS used by Clark et al. (2013) was in fact of fixed duration, increased inhibition of delay across postasymptotic training (i.e., improved timing of conditioned behavior) may have contributed to the decline in NAcc dopamine release. This point is particularly important given that the data presented from fast-scan cyclic voltammetry are the peak CS- and US-evoked dopamine release in the 2 s period after stimulus presentation. Unfortunately, Clark et al. (2013) do not present the distributions of behavior or NAcc dopamine release across the 8 s CS presentations, leaving open the possibility that dopamine release in the NAcc was not in fact diminished across extended training, but rather redistributed across the later periods of the CS when rats were actively engaged in lever pressing. At first glance, this suggestion seems at odds with changes in phasic activation of VTA dopamine neurons across CS–US training: as noted above, firing of these neurons migrates from the time of US delivery to the time of CS onset. However, it should be noted that the relationship between activation of VTA neurons and NAcc dopamine release does not follow a simple one-to-one correspondence, as dopamine release in the latter region is strongly influenced by activity in other brain areas, including the prefrontal cortex and hippocampus (Goto and Grace, 2005).

A second alternative explanation for the apparent decline in NAcc dopamine release across extended training is a change in the way that the CS itself is processed. According to one influential theory (Pearce and Hall, 1980), the attention commanded by a CS is proportional to the certainty with which it predicts reward, so that a CS that signals a variable (or unexpected) reward is better attended than a CS that signals a consistent (or expected) reward. Critically, variance in reward delivery is also reflected in activation of midbrain dopamine neurons, specifically, in their levels of tonic (as opposed to phasic) activation, which increases with reward uncertainty (Fiorillo et al., 2003). Dopaminergic coding of uncertainty means that, across extended CS–US training of the sort used by Clark et al. (2013), increasing certainty about the established CS–US contingency may have resulted in decreased attention to the CS, as well as a change in tonic output of midbrain dopamine neurons. Clark et al. (2013) could have tested these possibilities by including a test session in which CS probe trials were not reinforced. Moreover, these probe trials could have been presented using the same range of intertrial intervals as used during regular training, thus preserving the temporal structure of the task while the predictive status of the CS is rendered uncertain. One could then assess the consequences of this uncertainty for NAcc dopamine release on the next (regular) CS–US trial.

Two other issues in the paper make it difficult to evaluate the claim that dopamine encoding of incentive value diminishes with extended training. First, it is not clear whether performance in the task is subject to Pavlovian as opposed to instrumental response control. The cardinal assay of this control is sensitivity of performance to an omission schedule (Peden et al., 1977). In the study by Clark et al. (2013), it remains possible that rats had in fact (mistakenly) detected a contingency between lever pressing and the delivery of the food pellet reward, and thus treated the task as an instrumental one instead of a Pavlovian one. Overtraining of an instrumental response has been shown to favor automaticity of responding (habits), and this automaticity is correlated with a shift from ventral to dorsal striatal dopamine dependence (Faure et al., 2005), as well as an increase in dependence on dopaminergic mechanisms in the prefrontal cortex (Naneix et al., 2009). So, far from dopamine encoding of Pavlovian incentive value diminishing with extended training, the procedure used by Clark et al. (2013) leaves open the possibility that the pathways through which dopamine encodes overtrained responses changes across the course of training.

The second issue relates to the finding that blockade of D1 receptors had a greater depressive effect on lever pressing after five sessions compared with 15 sessions of training. The authors infer from this that after prolonged training, responding to a Pavlovian incentive stimulus becomes less dopamine-dependent in general. However, we would argue that the only inference permitted by these data is that lever pressing has become less dependent on activation of D1 receptors; leaving open the possibility that the substrate supporting lever pressing has shifted to activation of other dopamine receptor subtypes (e.g., D2).

The main finding in the paper is that, after conditioned performance had reached an asymptote, dopamine release in the NAcc declined across the first 2 s of CS presentations in continued training. We are left to speculate about changes in behavior and NAcc dopamine release across the remaining 6 s of the CS. Even if one were to accept these data as evidence that presentations of the CS across extended training caused less dopamine to be released in the NAcc, we do not know whether changes in dopamine release occur elsewhere in the brain. Without knowing this information, it is difficult to evaluate the hypothesis that dopamine encoding of incentive value diminishes with extended training. In addition, we have highlighted alternative explanations of Clark et al.'s (2013) findings that do not require or appeal to a change in the substrate of incentive value with extended training. In our opinion, this remains an open question.