The VTA to accumbens GABA projection: promoting prediction or engineering extinction?

The nucleus accumbens (NAc) is critical for setting the vigor of reward-seeking behavior and for learning to respond to reward-predictive cues. Aberrant NAc neuronal activity also contributes to neuropsychiatric diseases such as drug and gambling addiction. Each of these processes depends critically on the projections to the NAc from the midbrain ventral tegmental area (VTA). Most studies of the VTA have focused on its dopamine neurons, which project mainly to the NAc and prefrontal cortex. Dopamine neurons both facilitate reward seeking and play a critical role in learning. They do so by signaling value prediction errors, which are an integral component of temporal difference learning – the form of plasticity that allows animals to associate stimuli with rewards. However, the VTA is a heterogenous brain area that contains not only dopamine neurons, but also a subpopulation of neurons that release the inhibitory neurotransmitter ƴ-aminobutyric acid (GABA). This neuronal population both provides local inhibition to dopaminergic projection neurons and projects to several of the same regions as VTA dopamine neurons, including the NAc (1). NAc-projecting VTA GABA neurons (NGNs) have not been as well studied as their dopaminergic counterparts, and thus the functional role of this connection is not understood.

Modern methods for targeted modulation of defined neuronal populations now allow this functional role to be tested. In a previous study (2), Wakabayashi et al used Cre-recombinase to drive the specific expression of hM3D only in VTA GABA neurons. hM3D, a designer receptor exclusively activated by designer drugs (DREADD), makes neurons more excitable by activating G-protein signaling when bound to an exogenous ligand, such as clozapine-N-oxide (CNO) (3). Rats expressing this construct in VTA GABA neurons were trained to approach and press a lever in response to a cue predicting sucrose reward – a reward-seeking task that is highly sensitive to disruption by manipulations of the VTA and NAc (4). Consistent with direct inhibition of VTA dopamine neurons by VTA GABA neurons, activation of VTA GABA neurons by systemic CNO injection reduced rats’ willingness to respond to the cue. Intriguingly, however, local injection of CNO directly into the NAc – which should have specifically impaired NGNs – had no effect (2), suggesting that this projection serves a different functional role than that of the population of VTA GABA neurons as a whole.

In this edition of Biological Psychiatry, Wakabayashi et al propose that NGNs have a more specific role: they facilitate the adaptation in reward-seeking behavior that occurs after the reward magnitude is changed (5). This hypothesis stems from previous studies indicating that VTA GABA neurons (although not necessarily the actions of these neurons in the NAc) contribute to the prediction error signaling of dopamine neurons (6). Such prediction error signaling is likely important for adjusting cued reward-seeking behavioral responses when the reward magnitude changes unexpectedly. Wakabayashi et al first trained rats to respond to audiovisual cues to receive a 64 μl liquid sucrose reward, and then tested them on a shifting value task in which reward volumes varied in random order among 16, 48, and 128 μl across blocks of trials. The authors observed that local infusion of CNO into the NAc of animals expressing hM3D in VTA dopamine neurons caused a gradual decline in the proportion of cues the animals responded to, as well as a gradual increase in latency to respond. Curiously, these effects were specific to the small reward block; responding during the medium and large reward blocks was largely unaffected. By contrast, when all GABA VTA neurons were excited via systemic CNO administration, animals exhibited more widespread attenuation in cue responding regardless of reward block.

Wakabayashi et al interpret their findings to mean that NGNs activate processes that update response-outcome contingencies when the value of the reward obtained differs from the value expected based on prior experience. Their results support this possibility in the negative reward prediction error case, when the reward delivered (16 μl) was less than the reward animals had been trained to expect (64 μl). Interestingly, the authors did not observe a complementary increase in cue responding during blocks when the reward was larger (128 μl) than the trained reward. However, because their animals had likely reached a performance ceiling that prevented detection of further increases, the authors do not rule out a role for NGNs in adjusting behavior to larger-than-expected rewards. Future experiments could circumvent this technical difficulty by using a task in which the trained reward value is small enough that performance is not near a ceiling (e.g., by using a lower concentration of sucrose), or by using an inhibitory DREADD in the same task to determine whether inhibition of NGNs attenuates the enhancement of responding that should occur after unexpectedly large rewards.

However, there is reason to be skeptical that activation of NGNs causes its inhibitory effects on reward seeking via interference with negative prediction error signaling. Because reward magnitudes were shifted across blocks of ~35 trials, animals should have developed an expectation of reward magnitude during each block that would then be violated at block transitions (for example, from the 128 μl block to the 48 μl block). However, the authors do not report that their CNO injections influenced the behavioral response to these transitions, perhaps because this behavioral response itself was not robust. Thus, the possibility remains open that rather than play a specific role in adjusting reward value estimates in the face of new experience, activation of NGNs may instead act to reduce response ratio and response vigor in all conditions, but only exert a significant effect when the reward volume is small and thus less intrinsically motivating. In this view, activation of NGNs alone has the same effect as global activation of all VTA GABA neurons, but the effects are less pronounced simply because the magnitude of activation is less and/or the location of activation is more limited.

Another consideration is that hM3D activation produces continuous excitation rather than excitation time-locked to specific events such as reward delivery. It is unclear how such continuous excitation would impact the learning that results from dopamine prediction error signaling. NGNs are thought to synapse primarily onto cholinergic “tonically active” interneurons (CINs) (7), which in turn may gate plasticity when their tonic discharge is paused by rewards and other stimuli (8). In theory, continuous inhibition of CINs should silence them, but the behavioral effects would depend on whether the important factor for gating plasticity is simply that cholinergic tone is reduced in a way that need not be temporally specific, or instead a brief, temporally specific transition from high to low cholinergic activity must occur during rewards or cues. In the former case, one would expect plasticity to be enhanced by activating an excitatory DREADD in NGNs; but in the latter, one would expect it to be reduced. These possibilities could be distinguished by optogenetically manipulating NGNs, as well as CINs, just when reward is delivered and/or when cues are presented. A previous study by Brown et al found that optogenetic activation of NGNs during fear conditioning attenuates freezing to a non-shock-predictive cue after stimulus generalization (7), a result that may be consistent with a role for this projection in modulating plasticity.

One potential explanation of both the Wakabayashi et al and Brown et al findings is that NGNs are specifically involved in extinction. In both studies, evidence that activation of this projection influences learning is based on observations of reduced responding to cues that predict outcomes of low value (small reward in Wakabayashi et al, and nothing in Brown et al). Enhancement of extinction would be expected to produce exactly the gradual decline in responding to the small reward observed by Wakabayashi et al. If the enhancement were strong enough, it would also be expected to reduce responding to cues predicting more valuable outcomes, perhaps explaining why global activation of VTA GABA neurons had greater behavioral effects than localized excitation of their NAc-projecting subdivision. A role for VTA GABA neurons in extinction finds support in observations that they fire in response to presentation of non-reward-predictive cues (5), and that selective elimination of CINs in the dorsomedial striatum enhances extinction of reward place learning (9, but see 10).

Participation of NGNs in extinction is, of course, not inconsistent with a role in signaling negative prediction errors, but neurons can contribute to extinction without altering prediction error signals. Therefore, future studies of NGNs should utilize not only the more temporally specific manipulations afforded by optogenetics, but also more elaborate behavioral paradigms that allow prediction error-related effects to be distinguished from other effects, particularly extinction. Physiological investigations to measure the activity of NGNs during such tasks would also yield a great deal of insight, especially in conjunction with similar studies of VTA dopamine neurons and NAc CINs. Thus, although the precise role played by NGNs remains elusive, the Wakabayashi et al results provide an intriguing starting point.