Mesoaccumbal Dopamine Heterogeneity: What Do Dopamine Firing and Release Have to Do with It?

Abstract

Ventral tegmental area (VTA) dopamine (DA) neurons are often thought to uniformly encode reward prediction errors. Conversely, DA release in the nucleus accumbens (NAc), the prominent projection target of these neurons, has been implicated in reinforcement learning, motivation, aversion, and incentive salience. This contrast between heterogeneous functions of DA release versus a homogeneous role for DA neuron activity raises numerous questions regarding how VTA DA activity translates into NAc DA release. Further complicating this issue is increasing evidence that distinct VTA DA projections into defined NAc subregions mediate diverse behavioral functions. Here, we evaluate evidence for heterogeneity within the mesoaccumbal DA system and argue that frameworks of DA function must incorporate the precise topographic organization of VTA DA neurons to clarify their contribution to health and disease.

1. INTRODUCTION

Midbrain dopamine (DA) neurons have received great attention because of their important roles in a broad range of motivated behaviors and their dysfunction in several neuropsychiatric disorders, including Parkinson’s disease, major depressive disorder, substance use disorder, and schizophrenia (Bjorklund & Dunnett 2007, Bromberg-Martin et al. 2010, Nestler & Carlezon 2006, Robinson & Berridge 2008, Schultz 2007, Wise 2004, Wise & Robble 2020). The striatum is the major projection target of midbrain DA neurons (Albanese & Minciacchi 1983, Ikemoto 2007, Lammel et al. 2008, Poulin et al. 2018, Roeper 2013). It is often subdivided into the nucleus accumbens (NAc), which is located in the ventral striatum, and the dorsal striatum based upon projections arising from the ventral tegmental area (VTA) and substantia nigra pars compacta (SNc) DA neurons, respectively.

Rapid technological advances in system neuroscience have provided researchers with unprecedented opportunities to study the midbrain DA system. The incorporation of optogenetic approaches and use of transgenic animals have allowed for reliable access to DA neurons, with laser-light delivery during electrophysiological recordings eliciting rapid neural responses and enabling identification of genetically defined DA neurons (i.e., opto-tagging) (Cohen et al. 2012). Two-photon calcium imaging through an implanted lens has been used to record the activity of hundreds of DA neurons in the VTA during complex behavioral tasks (Engelhard et al. 2019, Hamid et al. 2021). Additionally, genetically encoded G protein–coupled DA sensors have been developed to successfully monitor the dynamics of DA release with high spatiotemporal resolution in freely behaving animals (Patriarchi et al. 2018, Sun et al. 2018). Together, these approaches allowed the field to overcome some of its previous challenges, for example, the reliable identification of VTA DA neurons in vivo (Margolis et al. 2006, Ungless & Grace 2012), and advances have been made in long-standing debates about the role of DA neurons in aversion (Schultz 2019, Verharen et al. 2020). While previous notions about the function and connectivity of VTA DA neurons are slowly beginning to change, the field faces a number of new challenges. While the nature of some of these challenges is methodological, such as the development of strategies to relate data from calcium imaging and electrophysiological experiments in DA neurons (Fleming et al. 2021, Sabatini & Tian 2020), associating VTA DA firing patterns and accumbal DA release with a precise behavioral function has been more difficult than previously thought. A recent study suggested that the slow, gradual release of DA (ramping) in the NAc is controlled locally, independent from the firing of VTA DA cell bodies, and encodes information about the value of the anticipated reward (Mohebi et al. 2019), suggesting that DA dynamics mediated by different mechanisms in different locations (i.e., at the level of cell bodies versus terminals) may underly different behavioral functions. In contrast, another recent study developed an experimental paradigm using virtual reality to disambiguate reward prediction error (RPE) encoding from value, suggesting that ramping DA signals are consistent with RPE rather than value and can be observed both at the level of individual VTA DA neurons and in axon terminals (Kim et al. 2020). Thus, an important question that currently hampers our understanding of DA function is how neural activity of VTA DA neurons translates into DA release in the NAc during behavior.

Despite an increasing appreciation of DA heterogeneity, in both the SNc and the VTA, the use of such advanced approaches has surprisingly led to renewed propositions of a uniform or single function common to all DA neurons. In this review, we explore and compare studies that have recorded activity of VTA DA neurons at the level of individual cells versus DA release at the axon terminal level in the context of behaviors associated with reward learning, motivation, and aversion to better understand whether DA activity and release share common functions. Collectively, we argue that many exciting developments have emphasized numerous unique features within the DA system, and we hope to encourage investigators to incorporate this heterogeneity into computational and behavioral frameworks for DA function in health and disease.

2. TOPOGRAPHIC ORGANIZATION OF A HETEROGENEOUS MIDBRAIN DOPAMINE SYSTEM

Like every tissue in the body, VTA and striatum are composed of multiple cell types that vary in their gene and protein expression, morphology, and connectivity. In addition to DA neurons, the VTA contains GABAergic and glutamatergic neurons, and some DA neurons may also co-release GABA or glutamate (Morales & Margolis 2017). The striatum is composed of GABAergic medium spiny neurons, which can be largely subdivided into cells that either express DA D1 receptors or D2 receptors, as well as cholinergic interneurons (Kreitzer 2009). The pronounced cellular heterogeneity, particularly in the VTA, and the fact that some DA neurons potentially co-release glutamate and GABA, require methodological approaches that precisely identify the cellular identity of DA neurons when studying their functional role in diverse behavioral settings. In the past, DA neurons were often identified based on indirect measures, resulting in controversies about the reliability of electrophysiological criteria used to identify DA neurons in vivo (Margolis et al. 2006, Ungless & Grace 2012). Current opto-tagging approaches allow researchers to reliably identify DA neurons through the use of transgenic animals, restricting expression of optogenetic proteins, such as channelrhodopsin, to cells expressing genes that are associated with the DA phenotype (e.g., TH or DAT), but in vivo electrophysiological experiments are typically conducted from DA neurons without knowing their specific projection target.

This makes sense to some degree, given that reliable recordings from projection-defined DA neurons are not yet feasible. The primary retrograde-traveling AAV that is commercially available (Tervo et al. 2016) transfects only a small number of projection-defined VTA DA cells, and other common retrograde viruses such as CAV2-Cre would require combinatorial genetic strategies to limit expression in a projection-specific manner (e.g., use of Cre and Flp systems). Fractionating VTA neurons on the basis of unique cell types from classical tyrosine hydroxylase and DAT markers may reveal circuit- and projection-specific enrichment and a path to exploit intersectional genetic strategies to limit manipulations and cell type–specific recordings (Poulin et al. 2020). However, discrepancies of exact projection targets in the striatum labeled by medial shell–exclusive cholecystokinin-positive VTA neurons versus core-exclusive corticotropin releasing factor receptor 1–positive VTA neurons have been demonstrated, which may limit the utility of identifying projection-defined cells on the basis of the presence of these genes (Heymann et al. 2020, Poulin et al. 2018). A lasting issue remains in that all genetic tools are developed primarily in mice, with a lack of similar tool kits for rats [e.g., DAT-Flp mice (Kramer et al. 2021)], which may ultimately produce bias in the exact behavioral approaches investigators use and, in turn, more disparities in conclusions.

A major limitation of most previous in vivo electrophysiological studies is that recordings from DA neurons were often biased toward the lateral VTA without considering DA cells in more medial VTA subregions (e.g., Cohen et al. 2012; Eshel et al. 2015, 2016) (Figure 1). Because it has become clear that the topographic organization and connectivity of DA neurons is an important feature for their diverse functions (Collins & Saunders 2020, Poulin et al. 2020, Verharen et al. 2020), recording DA cells in only one VTA subregion alone cannot be used to annotate the full spectrum of DA diversity. Consistent with this, recent studies using in vivo calcium imaging or electrophysiological approaches have observed pronounced functional heterogeneity of DA neurons located in more medial VTA subregions (Engelhard et al. 2019, Kim et al. 2020), whereas DA neurons in the lateral VTA seem to be more homogeneous (Eshel et al. 2016). From an anatomical perspective, it is important to mention that medial and lateral VTA DA neurons have nonoverlapping projections to distinct subregions in the NAc (Beier et al. 2015, Lammel et al. 2008). These distinct projections suggest unique functions between VTA DA neurons preventing generalization, which we discuss further below.

Figure 1.

The mesoaccumbal dopamine system. (a) Schematic showing anatomical organization of the mesoaccumbal DA system. DA neurons projecting to the NAc originate in the VTA. These neurons exhibit a topographical gradient of projections from medioventral and dorsolateral VTA to medial and lateral NAc, respectively. MSNs in the NAcMed directly inhibit medial VTA DA neurons projecting to the NAcMed, whereas MSNs in the NAcLat target GABAergic neurons in the lateral VTA that ultimately disinhibit NAcLat-projecting lateral VTA DA neurons. (b) Approaches for monitoring DA neuron activity and DA release are often biased toward specific VTA and NAc subregions, respectively. In the NAc (top), recordings of DA release are often biased toward the NAcMed and NAc core. In the VTA (bottom), recordings of DA neural activity are often biased toward the lateral VTA. Note that the preferred recording locations in the VTA and NAc are inconsistent with the topography of the mesoaccumbal DA system. Thus, we argue that studies of DA function should carefully consider the precise topographic organization of the mesoaccumbal DA system when investigating DA cell activity and release in the context of behavior. Abbreviations: DA, dopamine; FSCV, fast-scan cyclic voltammetry; IPN, interpeduncular nucleus; MSN, medium spiny neuron; NAc, nucleus accumbens; NAcLat, nucleus accumbens lateral shell; NAcMed, nucleus accumbens medial shell; SN, substantia nigra; VTA, ventral tegmental area.

Traditionally, the NAc has been subdivided into shell and core substructures. The NAc shell [also often called NAc medial shell (NAcMed)] is shaped like a crescent and located medial and ventral to the core. The differences between shell and core have been defined by various histochemical (e.g., calbindin, calretinin, substance P), electrophysiological, connectivity, and functional criteria (Castro & Bruchas 2019, Ikemoto 2007, Zahm & Brog 1992). An additional ventral striatal subregion, termed nucleus accumbens lateral shell (NAcLat) (see the sidebar titled Nucleus Accumbens Lateral Shell), which is located laterally to the NAc core, has been described in several previous studies (Beier et al. 2015, Breton et al. 2019, de Jong et al. 2019, Farassat et al. 2019, Ikemoto 2007, Lammel et al. 2008, Liu et al. 2020, Yuan et al. 2019) but generally is less studied compared to the NAc core or NAc medial shell. Nonetheless, the findings that the NAcLat is part of a major disinhibitory feedback loop (Yang et al. 2018) and the most prominent projection target of DA neurons that are located in the lateral VTA (Breton et al. 2019, Lammel et al. 2008) suggest that this often overlooked structure is an integral part of the mesolimbic DA system that needs to be taken into consideration when studying the functional role of DA release in the context of DA neuron activity.

3. THE PHYSIOLOGY OF DOPAMINE CELL FIRING AND DOPAMINE RELEASE

A foundational study using in vivo electrophysiology recordings delineated two different firing modes of putative DA neurons: tonic (0.2–10 Hz) and burst (>10 Hz) firing patterns (Grace & Bunney 1984). In acute brain slice recordings, VTA DA neurons exhibit pacemaker or irregular firing patterns (Krabbe et al. 2015, Paladini & Roeper 2014). Conversely, burst firing, which depends on NMDA receptor activation and network activity (Zweifel et al. 2009), is absent in ex vivo slice preparation but can be induced in a subset of neurons by inhibition of T-type calcium channels (Wolfart & Roeper 2002). Both excitatory and inhibitory afferents to midbrain DA neurons have been extensively surveyed using pseudotyped rabies tracing (Beier et al. 2015, Faget et al. 2016, Lerner et al. 2015, Menegas et al. 2015, Watabe-Uchida et al. 2012). These studies have highlighted that DA neurons with distinct projection targets receive inputs from a variety of overlapping brain regions.

Given the surprisingly relatively uniform distribution of inputs onto VTA DA neurons with distinct projection targets, it is important to emphasize that rabies tracing does not quantify synaptic strength and the resultant functional connectivity of these inputs. Moreover, it can be difficult to predict the computational or functional role of a defined direct input onto VTA DA neurons. For instance, a landmark study leveraged rabies tracing to record from opto-tagged monosynaptic inputs onto VTA DA neurons found that the information required to encode RPEs is distributed with high redundancy among several VTA DA afferents, and no single aspect of the RPE signal was localized in any particular afferent (Tian et al. 2016). This does not preclude that there exist afferents to VTA DA neurons that exert specific functions via biased connectivity in a projection-defined manner (Cardozo Pinto & Lammel 2018, de Jong et al. 2019, Juarez & Han 2016, Lammel et al. 2012, Morales & Margolis 2017, Nieh et al. 2016, Qi et al. 2014, Steidl et al. 2017). For example, glutamatergic afferents from the laterodorsal tegmentum and dorsal raphe nucleus are biased toward NAcLat-projecting VTA DA neurons, and these subcircuits are predominately involved in reward-related behaviors (de Jong et al. 2019, Lammel et al. 2012, Qi et al. 2014, Steidl et al. 2017). Conversely, glutamatergic afferents from the lateral hypothalamus primarily innervate medial VTA DA neurons, encode aversive stimuli, and promote DA release in the ventral NAcMed while indirectly inhibiting other VTA DA neurons (de Jong et al. 2019, Nieh et al. 2016). Thus, the questions of how DA neurons integrate information from different afferents and how it affects DA cell firing remain open.

Another important matter that remains unaddressed is to elucidate the neurobiological mechanisms that underlie tonic and phasic DA firing as well as the switch between these two firing states. However, there is a paucity of studies that have examined projection-defined DA neurons or DA neurons in different VTA subregions in this context. Nonetheless, two recent studies have made important progress. First, Farassat et al. (2019) combined juxtacellular recordings with retrograde tracing and found that NAcMed- and NAcLat-projecting VTA DA neurons have distinct baseline firing rate and burst characteristics. Second, Otomo et al. (2020) established in vivo patch clamp recordings from DA neurons and performed the first in vivo intracellular recordings from DA neurons since the seminal work of Grace & Bunney (1984) in the 1980s. This work delineated two qualitatively distinct types of burst firing characterized by distinct membrane potential dynamics and opposing shifts in action potential threshold. However, medial VTA DA neurons were underrepresented in the Otomo et al. (2020) study, and the projection target of all recorded neurons was unknown. Furthermore, although these studies provide important insights into the nature of tonic and phasic firing patterns, future research will need to resolve the mechanisms that mediate the switch between these firing states.

Finally, the relationship between somatic action potential firing and DA release at the axon terminal requires further investigations. Striatal DA release is evoked by action potentials that originate in the soma, but DA release sites are rapidly depleted, causing depression of release following repeated firing (Liu et al. 2018, 2021). This requires DA reserves to be replenished by reuptake mediated by the dopamine transporter (DAT). Importantly, DAT transporter expression varies between DA neurons with different projection targets (Lammel et al. 2008, Sesack et al. 1998), suggesting a similar pattern of neuronal firing might result in different extracellular DA concentrations at distinct DA release sites. The rapid depletion of DA release sites makes it unlikely that DA release precisely tracks cell body firing, and DA axon terminals do not form traditional synapses that would support such a 1:1 relationship of spiking to release (Liu et al. 2018). Increased synchronization that occurs during phasic DA firing has been proposed to cause widespread diffusion of DA and activation of postsynaptic DA receptors by saturating DAT reuptake (Liu et al. 2021). Putting the emphasis on synchronized firing poses the possibility that ramping responses, which are predominantly observed when examining striatal DA release (Howe et al. 2013, Mohebi et al. 2019, Roitman et al. 2004, Wassum et al. 2012), are not exclusively the result of ramping increases in somatic firing but instead are caused by increased synchronization of tonic firing neurons. Related to this, recent work in opto-tagged VTA DA neurons provided evidence for multiplexing in that a subset of DA neurons encoded environmental cues through phasic bursts and goal-directed actions through modulation of tonic firing (Kremer et al. 2020). Further highlighting the nonlinear relationship between soma activity and axonal DA release is the observation that while individual phasic bursts cause rapid desensitization of the DA response, a slow (over multiple seconds) facilitation of striatal DA release occurs following repeated bursts of high-frequency (60 Hz) electrical stimulation of DA neurons, which is presumed to be mediated by increased refilling of the vesicular pool (Montague et al. 2004). Additionally, in vitro experiments have shown that local control by cholinergic interneurons via activation of nicotinic acetylcholine receptors on DA axon terminals can evoke action potential–independent DA release (Threlfell et al. 2012). We summarize these potential mechanisms that may contribute to ramping of DA release in the striatum in Figure 2, and discuss this topic further in the following section.

Figure 2.

Possible mechanisms for ramping of DA release in the striatum. Schematic of dopaminergic release sites (blue) in the striatum releasing DA in close proximity to DA D1R and D2R expressing MSN. Ramping of DA release may be generated through (①) action potential firing at the level of DA cell bodies, (②) modulation of DA release by cholinergic interneurons, (③) diffusion from numerous sparse release sites, (④) synchronization of VTA DA cell firing, and (⑤) saturation of DAT-mediated reuptake mechanisms. Abbreviations: D1R, D1 receptor; D2R, D2 receptor; DA, dopamine; DAT, dopamine transporter; MSN, medium spiny neuron; NAc, nucleus accumbens; VMAT2, vesicular monoamine transporter type 2; VTA, ventral tegmental area.

Taken together, the possibility of reliably identifying DA neurons during in vivo electrophysiological recordings and the availability of newly developed tools to measure DA release with high temporal and spatial precision have opened new avenues to study the relationship between somatic spiking and axonal DA release. However, because of the distinct input-output relationships and anatomical, molecular, and physiological properties of DA neurons, future studies need to take the topographic organization and connectivity of the DA system into consideration. Moreover, the complex dynamics of DA release and reuptake (Liu et al. 2021) and methodological limitations (Sabatini 2019) will have to be taken into account, though advances have been made to correlate DA activity observed through calcium influx to spiking activity (Fleming et al. 2021). Together, these efforts will no doubt shed light on the precise relationship between DA firing and release in different DA pathways.

4. DOPAMINE CELL FIRING AND RELEASE IN REWARD LEARNING AND MOTIVATION

Given that DA neurons are highly heterogeneous, it is somewhat surprising that leading perspectives on DA cell activity and DA release tend to propose a homogeneous role for these neurons in encoding RPEs (Cohen et al. 2012; Dabney et al. 2020; Eshel et al. 2015, 2016; Gardner et al. 2018; Gershman & Uchida 2019; Kim et al. 2020; Schultz 2019). This idea stems from the foundational studies of Schultz and colleagues (Hollerman & Schultz 1998; Schultz et al. 1993, 1997) performing single-unit recordings in the primate midbrain that have since been replicated with optogenetically assisted tagging of DA neurons in rodents (Bayer & Glimcher 2005; Cohen et al. 2012; Eshel et al. 2015, 2016). These studies have been further corroborated by the ability to record calcium activity in DA neuron axon terminals and DA release with fluorescent biosensors and have reinforced a long-standing debate regarding DA’s actions in the striatum as mediating reward learning via RPE encoding as opposed to a motivational account (Kim et al. 2020, Mohebi et al. 2019, Patriarchi et al. 2018, Sun et al. 2018, Tsutsui-Kimura et al. 2020). While we cannot cover all aspects of this debate (for recent reviews, see Berke 2018, Coddington & Dudman 2019, Collins & Saunders 2020, Dayan & Berridge 2014, Gardner et al. 2018, Gershman & Uchida 2019, Keiflin & Janak 2015, Morales & Margolis 2017, Watabe-Uchida et al. 2017), we wish to offer here the perspective that the circuit architecture of the heterogeneous DA system refutes such a universal role for DA in exclusively mediating reinforcement learning.

DA neurons of the midbrain are embedded in a spiraling network, with inputs from their primary targets in the striatum such that a mediolateral and dorsoventral gradient exists in the projections of DA neurons and, in turn, the projections back onto these neurons from the striatum. This architecture appears to be homologous in rodents and primates (Everitt & Robbins 2013, Haber & Knutson 2010, Haber et al. 2000, Joel & Weiner 2000, Yang et al. 2018), although we recently found that in mice, projections from the NAcMed directly inhibit VTA DA neurons, while projections from the NAcLat provide canonical disinhibition of lateral VTA DA neurons via a GABA neuron intermediary (Yang et al. 2018). This surprising difference in the architecture of the striato-midbrain loop has functional consequences as DA release in the NAcMed and NAcLat has distinct response functions, with NAcMed DA release more closely mirroring behavioral output (e.g., licking for reward) and NAcLat DA release having a phasic cue and reward response (de Jong et al. 2019, Yuan et al. 2019). DA release in this manner should be heterogeneous within a region, for example, a distinction between slow fluctuations in basal DA tone following behavioral output and the fast and rapid rises in DA in response to environmental stimuli (Floresco et al. 2003, Grace & Bunney 1984, Hamid et al. 2016, Hyland et al. 2002, Niv et al. 2007, Schultz 2007). That even on a rapid timescale DA release in distinct NAc subregions appears more tonic or more phasic is in opposition to proposals for unified or universal theories of DA function that have arisen from studies of DA release primarily in the NAc core (Berke 2018, Kim et al. 2020, Lowet et al. 2020, Mohebi et al. 2019). We considered whether this heterogeneity has been overlooked by analyzing the sites of recording for studies making use of optogenetic tagging of DA neurons and in papers recording DA release in the ventral striatum. Overall, it is apparent that there has been a bias to record DA neuron activity in the lateral VTA and DA release in the NAc core in recent investigations of DA circuit function (Figure 1; Supplemental Table 1). But the innervation of the NAc core and NAcMed predominantly by VTA DA neurons originating from the medial VTA has created an apparent mismatch between cell body activity recordings in the lateral VTA and measurements of DA release in the NAc medial shell or NAc core. This bias may arise in part due to the difficulties in recording electrophysiological activity along the midline of rodents, given pronounced blood vessels that impede surgical access. Difficulties in surgical access to medial VTA regions due to blood vessels close to the midline may also exist in nonhuman primates. Additionally, medial VTA DA neurons have unique cellular and electrophysiological properties that may make them resistant to identification with existing optogenetic tagging methods (Farassat et al. 2019; Lammel et al. 2008, 2011; Yang et al. 2018).

Despite an ever increasing appreciation of functional diversity within VTA DA neurons, generally it is conceivable that, in some tasks, DA release in certain striatal regions and DA neuron activity in certain VTA subregions might be reduced to a RPE signal. Given the spiraling architecture of midbrain DA neurons, it appears fair to infer a shared computation among these neurons, but it remains less clear why such an extensive system in the brain would be devoted to uniformly processing and projecting the same computation, RPEs, to all of its downstream targets. Moreover, these correlational metrics of DA activity may look like RPEs but are typically assessed absent functional considerations—that is, whether RPE signals are phenomenological or actually serve to update behavior on a trial-to-trial basis (Dayan & Berridge 2014). The original demonstrations of DA neurons as RPEs arose from the Rescorla-Wagner model of Pavlovian conditioning, which would suggest that RPEs, and in turn DA function, are causal for Pavlovian learning. However, DA function is not required for learning in many Pavlovian situations that have been used to demonstrate the existence of RPE signaling in DA neurons (Day et al. 2007, Flagel et al. 2011). In non-human primates, selective inhibition of DA release in the NAc impaired motivation to earn reward but had no impact on reward learning, suggesting a limited role for DA release in the NAc—or in a yet unspecified subregion of the NAc—to exclusively mediate learning (Vancraeyenest et al. 2020).

In one recent compelling report, despite pairing cues directly with DA activation producing learning, individual DA projections to the striatum resulted in drastically different behavioral outcomes in Pavlovian conditioning. Only projections to the NAc core supported classical conditioning as evidenced by a cue-directed approach; projections to the dorsal striatum resulted in cue-generated rotations; and NAcMed projections, arising from the medial VTA, did not establish any cue-conditioned behaviors. Despite these fractionated behavioral functions in Pavlovian conditioning, all of these projections supported equivalent levels of optogenetic intracranial self-stimulation (Saunders et al. 2018). This is surprising because it suggests that even though DA release may uniformly support operant reinforcement, the contribution of the same exact DA signal has drastically different effects in Pavlovian conditioning in differing striatal subregions. Why distinct and disparate functions emerge from a proposed universal RPE signal in even the simplest of Pavlovian tasks remains unclear, as does how these functions could be transferred according to the spiraling architecture of the striato-midbrain system (Russek et al. 2017). Theories of DA function assume RPE has equivalent functions in Pavlovian and operant conditioning, yet it appears that this may not be the case. Although DA release and activity may conform to RPE encoding in these separate striatal regions, predicting the functional consequence of circuit-specific DA release appears nontrivial.

These disparities can potentially be addressed by taking into consideration local control and modulation of DA release within specific subregions of the striatum. Along with the ventromedial to dorsolateral spiraling of midbrain and striatal connectivity, glutamatergic projections from cortex, thalamus, and amygdala exhibit specialized projections throughout the striatum (Berendse et al. 1992, Groenewegen 1988, Heilbronner et al. 2016, Kelley 2004, Mogenson et al. 1980, Voorn et al. 2004, Wright et al. 1996). These projections have been shown to modulate DA release either via direct excitation of DA terminals or through cholinergic interneuron intermediaries (Adrover et al. 2020; Jones et al. 1989, 2010; Kosillo et al. 2016; Parsons et al. 2007; Pinto et al. 2003; Threlfell et al. 2012). However, there appear to be discrepancies on how different thalamic inputs impact DA release within the striatum, and differences may exist in how such local control arises throughout the striatum (Liu et al. 2021, Parsons et al. 2007, Sulzer et al. 2016, Threlfell et al. 2012). This local control of DA release has largely been proposed as an answer to the generation of ramping DA release (Mohebi et al. 2019), which has been observed in awake behaving animals during motivated behavior, yet a mechanism for ramp generation in vivo remains to be demonstrated (Berke 2018, Collins & Saunders 2020, Hamid et al. 2016, Howe et al. 2013, Kim et al. 2020, Lloyd & Dayan 2015, Roitman et al. 2004, Wassum et al. 2012). A critical and outstanding question is how the topographically distinct composition of amygdala, cortical, and thalamic inputs to different striatal subcompartments affects local control of DA release in vivo, if they contribute at all, especially as many of the demonstrations for such cholinergic interneuron–mediated forms of release have been conducted ex vivo in the dorsal striatum (Adrover et al. 2020, Kosillo et al. 2016, Threlfell et al. 2012).

Exploiting technical advances to fluorescently record the activity of defined striatal inputs or cholinergic interneurons alongside DA release will be essential to resolving the mechanisms underlying afferent control of DA release within the striatum in awake and behaving animals. These approaches can also resolve whether ramping release is an exclusive property of DA release manifested at the axon terminal or whether there is a potential for ramping activity in DA cell bodies, as has been recently observed in a select few DA neurons (Kim et al. 2020). Despite the observation of ramping in a small population of VTA DA neurons, their ramping could be described as RPE signaling (Kim et al. 2020), whereas other researchers have argued that ramping is constrained to striatal DA release where its function is instead to convey value and drive motivated behaviors (Mohebi et al. 2019). It is interesting to note that the authors also identified a small population of ramping VTA DA neurons whose activity could not be captured by RPE models and that the activity of these neurons is most reminiscent of observations of ramping obtained with fast-scan cyclic voltammetry (FSCV) (Howe et al. 2013, Kim et al. 2020). Given the extensive axonal branching of DA neuron axons in the striatum, directly identifying DA release in the striatum as arising from a simultaneously recorded VTA DA neuron is challenging (Aransay et al. 2015, Liu et al. 2021, Matsuda et al. 2009). We are not aware of antidromic approaches being successfully applied to record VTA DA neurons in rodents in such a circuit-specific manner during behavior, which could provide a means to optically record striatal DA release coincident with the spiking activity of these neurons within the VTA (Gadagkar et al. 2016, Guyenet & Aghajanian 1978). These technically challenging approaches may be necessary to resolve the long-standing discrepancy between DA neuron activity and release, and even provide insight into striatal compartment–specific specialization of these mechanisms.

One emerging topic of interest is what DA activity and release convey in the content of what is learned regarding relationships between cues, actions, and outcomes (Gardner et al. 2018, Langdon et al. 2018). DA neurons are proposed to conform to conveying model-free signals, updating the values of actions and states absent any representation but an abstract scalar value term (Bayer & Glimcher 2005, Hollerman & Schultz 1998, Schultz et al. 1997, Sutton & Barto 2018, Waelti et al. 2001). In contrast, in many instances, VTA DA neurons exhibit activity that suggests these neurons have access to richer representations that are evoked by cues such as the delay to reward, links between previously neutral sounds or lights, and precise sensory qualities of the upcoming reward (Sadacca et al. 2016; Stalnaker et al. 2019; Takahashi & Schoenbaum 2016; Takahashi et al. 2011, 2016, 2017). The ability of DA neurons to respond in this fashion suggests that they are more actively involved in generating very precise representations in contrast to most computational models of the function of these neurons (Dayan & Berridge 2014). Indeed, manipulations of VTA DA neurons alter behavior in tasks designed specifically to require the use of these model-based representations (Keiflin et al. 2019; Sharpe et al. 2017, 2020). For instance, learning induced by optogenetic activation of VTA DA neurons is sensitive to manipulations of the value of the predicted reward (Keiflin et al. 2019). Moreover, it appears that it is VTA, but not SNc, DA neurons that are critical for these model-based signals (Keiflin et al. 2019). This specialization in the function of distinct DA projections in model-based versus model-free learning makes difficult the assumption that DA signals are merely relayed from mesolimbic to striatonigral DA systems with time (Belin-Rauscent et al. 2012; Everitt & Robbins 2013; Willuhn et al. 2012, 2014). Indeed, increasing attention is being given to the diversity of computational metrics encoded by individual VTA DA neurons (Engelhard et al. 2019, Kremer et al. 2020). Despite this, we have little information about how rich, model-based information is transmitted throughout the striatum, as the primary evidence for such sensitivity has largely been demonstrated in the NAc core when the physiological state of the animal has been shifted (Aitken et al. 2016, Cone et al. 2016, Hsu et al. 2020). Moreover, it is unclear how model-based information itself is conveyed to VTA DA neurons to even allow such a complex computation (Langdon et al. 2018). It will be critical to disentangle the circuit-specific routing of model-based and model-free signals throughout the striatum to better refine our understanding of these neurons in motivated behavior. Generating and testing these hypotheses will require increasingly sophisticated behavioral approaches in tandem with circuit-specific recording of VTA DA neurons and DA release in defined striatal subregions.

5. DOPAMINE CELL FIRING AND RELEASE IN AVERSION

The processing of aversive stimuli by DA neurons has historically been a source of controversy. Foundational studies described a homogeneous population of DA neurons that were uniformly inhibited by aversive events or the omission of predicted reward (Hollerman & Schultz 1998; Kim et al. 2010, 2012; Ljungberg et al. 1991, 1992; Schultz et al. 1993; Ungless et al. 2004). This pattern of response supported the notion that these neurons participate in RPE encoding. However, recordings of putative DA neurons in the ventromedial VTA found that these neurons were excited by the presentation of a cue that predicted mild electric shock (Guarraci & Kapp 1999). Further, antidromic identification of prefrontal cortex (PFC)-projecting putative DA neurons, which are primarily found in the medial VTA, also found that most of these neurons are excited by a noxious tail pinch (Mantz et al. 1989). Despite these findings, heterogeneity in response to aversive stimuli was controversial, as these studies primarily made use of electrophysiological characteristics to identify DA neurons, which made certainty regarding their neurochemical identity difficult (Margolis et al. 2006, Ungless & Grace 2012) and left open the possibility that, even if a small number of electrophysiologically identified DA neurons responded to aversive stimuli (Bromberg-Martin et al. 2010, Matsumoto & Hikosaka 2009, Mirenowicz & Schultz 1996), these may in fact not be dopaminergic. To clarify this discrepancy, Brischoux et al. (2009) juxtacellularly labeled VTA neurons that were recorded while electric shocks were delivered to allow the confirmation of these neurons’ neurochemical identity with immunohistochemistry for tyrosine hydroxylase, the enzyme responsible for DA synthesis. Surprisingly, aversion-excited DA neurons were primarily observed in the medial VTA, whereas aversion-inhibited DA neurons were found in the lateral VTA (Brischoux et al. 2009). Given that medial VTA DA neurons primarily project to the PFC or NAcMed, this finding suggested circuit-specific specialization for the processing of aversion within the DA system (Brischoux et al. 2009, Lammel et al. 2008).

The suggestion that subsets of VTA DA neurons selectively process aversion was supported by the observation that recording DA release using microdialysis primarily detected increases in DA turnover in the NAc and PFC following restraint or electric shock (Abercrombie et al. 1989, Jackson & Moghaddam 2004, Young 2004). However, the DA response in the NAc to these stressors appears to depend on controllability, as it is diminished after repeated exposure unless the animal is given an opportunity to perform an action to terminate the aversive stimulus (Cabib & Puglisi-Allegra 1994, Imperato et al. 1993). However, the minute-long timescale of microdialysis made it difficult to link increases in DA release to the aversive experience itself or potential relief arising from its termination (Mirenowicz & Schultz 1996, Salinas-Hernández et al. 2018). Recording DA release on a seconds-long timescale with FSCV indicated that, while aversive foot shocks evoke DA release throughout the striatum, DA release in the NAcMed was selectively increased in response to shock-predictive cues (Badrinarayan et al. 2012, Budygin et al. 2012). To clarify this discrepancy between the proposed specialization of VTA DA neurons in encoding aversion versus an apparent general increase in DA in the striatum to aversive foot shock, we systematically mapped DA release throughout the NAc during aversive Pavlovian conditioning (de Jong et al. 2019) (Figure 3). We found that aversive foot shocks, as well as cues that predict them, selectively evoked DA release in the ventral NAcMed, while NAcLat DA release was decreased by aversive foot shock (de Jong et al. 2019). The selective increase in DA release in the ventral NAcMed suggests that this region can be divided into distinct dorsal and ventral compartments, as dorsal NAcMed DA release is decreased by noxious stimuli such as a bitter-tasting quinine solution (Roitman et al. 2008, Yuan et al. 2019) and increased by the termination or omission of an aversive stimulus (Budygin et al. 2012, de Jong et al. 2019). This dorsal reward and ventral aversion processing gradient in the NAcMed is also supported by the finding that direct activation of dorsal dynorphin-producing NAcMed neurons induces place preference, while ventral dynorphinergic NAcMed activation elicits place aversion (Al-Hasani et al. 2015).

Figure 3.

Aversion encoding in the mesoaccumbal DA system. Aversive stimuli (e.g., foot shocks) and their associated cues elicit decreases in DA release and DA terminal activity in the dorsal NAcMed, in the NAc core, and throughout the NAcLat. In contrast, these stimuli elicit increases in DA release and DA terminal activity within the ventral NAcMed. Abbreviations: DA, dopamine; NAc, nucleus accumbens; NAcLat, nucleus accumbens lateral shell; NAcMed, nucleus accumbens medial shell.

While advances have been made in defining DA subsystems that are excited or inhibited by aversive stimuli, further resolving the functions of VTA DA neuron activity and DA release in aversively motivated behaviors will require exploiting circuit-specific approaches to record and manipulate neural activity within these specialized projection-defined systems. Existing evidence suggests that VTA DA neurons are excited by and facilitate learning about aversive stimuli, which runs counter to RPE theories, but other recent studies suggest VTA DA neurons are inhibited to facilitate learning about aversive stimuli (Cai et al. 2020, Jo et al. 2018). Perhaps taking advantage of more ethologically relevant behavioral paradigms can clarify the apparent discrepant contribution of VTA DA neurons to aversively motivated behavior (Dennis et al. 2021). Nonetheless, it appears that taking into account the precise striatal projection target of VTA DA neurons refines their contributions to aversion, and it will be critical for future studies to exploit this projection-defined heterogeneity.

6. CONCLUSIONS

Within the past decade there have been numerous technological advances that have facilitated a greater appreciation of the heterogeneity of the DA system. Even within the mesolimbic DA system separate subpopulations of VTA DA neurons projecting to distinct NAc subregions are specialized for the production and transmittance of defined behaviorally relevant signals. However, this poses a challenge to theories of DA function that posit homogeneous roles for mesolimbic DA neurons in encoding RPEs. VTA DA neurons also appear to exhibit a wide array of computations that run counter to the original proposition of these neurons exclusively encoding model-free variables, and it remains to be demonstrated whether these neurons integrate inputs to generate rich model-based predictions or whether this information is merely inherited from upstream brain regions. Perhaps an overemphasis on investigating DA neurons in the lateral VTA and DA release in the NAc core has resulted in preliminary universal and sometimes controversial theories of mesolimbic DA function that still require proof for their functions throughout the striatum. Advancing methodologies to record from circuit- and cell type–defined VTA DA neurons in tandem with sophisticated and elegant behavioral approaches will be essential to move the field forward and resolve these outstanding issues. Recognition of this circuit-specific specialization of DA function will be necessary to advance our understanding of the role of these neurons in health and disease.

SUMMARY POINTS.

1. The midbrain dopamine (DA) system is highly heterogeneous from the molecular composition and anatomical organization of DA neurons to the function of projection-defined DA systems in behavior.

2. Whether DA release and the firing activity of DA neurons are dissociable is a topic of current debate.

3. Aversion-excited DA neurons have distinct projections and physiological characteristics, are located in the medial ventral tegmental area, and appear to belong to a unique class.

4. Defining DA neurons with respect to their striatal target refines their role in behavior.

FUTURE ISSUES.

1. How can we integrate reward, motivation, and aversion encoding within different DA subsystems into a coherent framework of DA function?

2. What is the functional consequence of DA signaling in defined striatal territories, and how are these altered by experience or in disease?

3. Why is reward prediction error signaling so prominent both within the activity of DA neurons and in DA release in their target regions, and what is the significance of such a signal?

4. Should we consider DA neurons as performing individualized computations or as populations or subpopulations that collectively encode metrics critical for behavior?

5. Do DA neurons generate model-based computations, and if so, what inputs onto DA neurons support model-free versus model-based behaviors?

6. Are the mechanisms that govern local control of DA release conserved throughout the striatum?

7. Is ramping of DA release generated at the axon terminal, or does ramping arise from ramping activity of dopamine cell bodies?

NUCLEUS ACCUMBENS LATERAL SHELL.

The terminology nucleus accumbens lateral shell (NAcLat) may be misleading since it may suggest that it is part of the nucleus accumbens (NAc) shell, though the most striking distinction appears to be that the NAcLat is the most prominent projection target from dopamine (DA) neurons in the lateral ventral tegmental area (VTA), whereas the NAc medial shell is innervated by DA neurons in the medial VTA (Breton et al. 2019, Farassat et al. 2019, Lammel et al. 2008). Further, NAc core-projecting DA neurons are predominantly located in the medial VTA, but a few studies in rats have also observed NAc core-projecting DA neurons in more lateral VTA regions (Breton et al. 2019, Saunders et al. 2018). Nevertheless, we think the terminology NAcLat is preferable as it refers to an anatomical region defined in The Mouse Brain in Stereotaxic Coordinates (Franklin & Paxinos 2001) and The Rat Brain in Stereotaxic Coordinates (Paxinos & Watson 2007). Whether this region is more appropriately labeled NAcLat, ventrolateral striatum, or lateral NAc remains a semantic question. Using nomenclature based on widely established anatomical atlases will facilitate the targeting of this brain region and in turn the reproducibility of results.

Glossary

DA

dopamine

NAc

nucleus accumbens

VTA

ventral tegmental area

SNc

substantia nigra pars compacta

Opto-tagging

expressing optogenetic proteins in a cell type–specific manner to confirm the identity of in vivo electrophysiologically recorded cells

Ramping

gradual increases in extracellular dopamine that increase toward a peak as reward is earned or actions are performed

Reward prediction error (RPE)

a computation that can be described as the mismatch between how much reward was expected and how much reward was received

TH

tyrosine hydroxylase

DAT

dopamine transporter

NAcMed

nucleus accumbens medial shell

NAcLat

nucleus accumbens lateral shell

Reinforcement learning

a process by which agents refine their behavior by iteratively acting upon the environment and receiving feedback via prediction errors

Pavlovian conditioning

also called classical conditioning, a procedure by which stimuli are associated with the delivery of outcomes absent any action requirement

FSCV

fast-scan cyclic voltammetry

Model-free

a computation or representation that is purely value based and abstract

Model-based

a computation or representation that is highly complex and relies on a set of relationships between stimuli, actions, and outcomes

PFC

prefrontal cortex