Heterogeneity in striatal dopamine circuits: Form and function in dynamic reward seeking

Abstract

The striatal dopamine system has long been studied in the context of reward learning, motivation, and movement. Given the prominent role dopamine plays in a variety of adaptive behavioral states, as well as diseases like addiction, it is essential to understand the full complexity of dopamine neurons and the striatal systems they target. A growing number of studies are uncovering details of the heterogeneity in dopamine neuron subpopulations. Here, we review that work to synthesize current understanding of dopamine system heterogeneity across three levels: anatomical organization, functions in behavior, and modes of action, wherein we focus on signaling profiles and local mechanisms for modulation of dopamine release. Together, these studies reveal new and emerging dimensions of the striatal dopamine system, informing its contribution to dynamic motivational and decision-making processes.

Striatal dopamine is central to adaptive reward processes, as well as diseases of dysfunctional learning and motivation, such as addiction (Berke, 2018; Berridge and Robinson, 1998; Cox and Witten, 2019; Dauer and Przedborski, 2003; Everitt and Robbins, 2005; Wise, 2004). Building on a large foundational literature, recent advances offer a number of new ways to directly target and probe precise components of these systems that offer insight into dopamine’s diverse functions. Despite this, many theories of dopamine function tend to ascribe a single, homogenous role to midbrain dopamine cells and the actions of dopamine release in downstream striatal regions. Here, we focus on three levels of heterogeneity within the rodent striatal dopamine system that complicate that picture: 1) anatomical organization, 2) regional and circuit-level behavioral functions, and 3) modes of action for dopamine neurons, including signaling mechanisms and local modulation of dopamine release in striatum. With an emerging appreciation of these characteristics, we can better understand how midbrain-striatal networks allow animals to act adaptively across a variety of dynamic reward-seeking and decision-making states.

1. Anatomical organization of striatal dopamine systems

Immunohistochemical approaches developed in the 1960s allowed for the identification and anatomical localization of dopamine-producing neurons. These methods established the canonical dopaminergic midbrain, comprised of the retrorubral area (A8 group), the substantia nigra pars compacta (SNc, A9 group), and the ventral tegmental area (VTA, A10 group). A number of dopamine neuron populations exist in other regions, including the hypothalamus, olfactory bulb, and retina (Björklund and Dunnett, 2007; Dahlstrom and Fuxe, 1964). For the purposes of this review, we will focus on dopamine neurons in the VTA and SNc populations, as they project prominently to the striatum, and have been the subject of the majority of recent study. VTA dopamine neurons also project to areas we will not discuss, including prefrontal cortex and amygdala, where they have unique properties and functions (Haber and Knutson, 2010; Lutas et al., 2019; Vander Weele et al., 2018). Across the VTA and SNc, most neurons are dopaminergic, with the VTA being more heterogeneous, containing ~30% GABAergic and ~10% glutamatergic neurons (Fields et al., 2007; Morales and Margolis, 2017; Swanson, 1982; Wise, 2004). In addition to their unique cell-type composition, the SNc and VTA receive diverse inputs and send dopaminergic outputs to distinct regions.

Dopamine system heterogeneity begins at the level of the midbrain, with unique inputs onto neurons in the SNc and VTA (Beier et al., 2015; Ikemoto, 2007; Lerner et al., 2015; Watabe-Uchida et al., 2012). The SNc preferentially receives excitatory afferents from the somatosensory and motor cortices (Comoli et al., 2003; Haber and Knutson, 2010; Watabe-Uchida et al., 2012). SNc dopamine neurons are also under the control of inhibitory inputs from the pars reticulata portion of the substantia nigra (Grace and Bunney, 1984; Mailly et al., 2003; Rizzi and Tan, 2019). In contrast, dopamine neurons in the VTA receive excitatory, inhibitory, and neuromodulatory inputs from hypothalamus, dorsal raphe, prefrontal cortex, and ventral pallidum (Beier et al., 2015; Comoli et al., 2003; Watabe-Uchida et al., 2012; Xiao et al., 2017). The lateral habenula sends glutamatergic projections to the posterior end of the VTA (sometimes called the rostromedial tegmental nucleus), which in turn sends GABAergic projections to the anterior VTA (Barrot et al., 2012; Herkenham and Nauta, 1979). Via these intermediary inhibitory projections, activity in the lateral habenula suppresses activity of dopamine neurons (Bromberg-Martin et al., 2010a; Christoph et al., 1986; Matsumoto and Hikosaka, 2009). Lateral habenula neurons can also activate VTA dopamine neurons via a direct glutamatergic projection (Lammel et al., 2012). Finally, the VTA and SNc receive strong, topographically organized input from the striatum. Thus, midbrain dopamine neurons are regulated by a brain-wide network of inputs that have uniquely innervate the VTA and SNc.

The striatum.

While VTA/SNc dopamine neurons project broadly throughout the forebrain (Breton et al., 2019; Fields et al., 2007; Fuxe, 1965; Haber and Knutson, 2010; Swanson, 1982), their primary target is the striatum. Dopamine neurons are somewhat unique in that they almost exclusively project unilaterally to one target region (although see Fox et al., 2016), which results in a structured, parallel innervation pattern into the striatum. The striatum is historically divided into ventral and dorsal sections, based on unique anatomical connectivity and cytoarchitecture (Groenewegen et al., 1999; Voorn et al., 2004). The ventral striatum, which receives dopaminergic input from the VTA, is divided into the nucleus accumbens core (NAC) and shell (NAS). The dorsal striatum, receiving dopaminergic input from the SNc, is typically divided into the dorsal lateral (DLS) and dorsal medial (DMS) striatum. The striatum also receives a number of topographically organized non-dopaminergic inputs from thalamus, cortex, and other regions (Groenewegen et al., 1999; Haber and Knutson, 2010; Heilbronner et al., 2016). Approximately 90% of all striatal neurons are GABAergic medium spiny neurons (MSNs), which are the primary output. MSNs exist in two categories, containing either D1 or D2 dopamine receptors, which constitute the “direct” and “indirect” pathways as conceptualized by canonical basal ganglia literature (Albin et al., 1989; Bolam et al., 2000; Gerfen, 1992a). Finally, a number of interneuron populations form local microcircuits within the striatum, which can modulate glutamatergic inputs, dopamine release, and MSN activity (Bolam et al., 1986; Burke et al., 2017; Wilson et al., 1990).

Ventral Striatum.

The ventral striatum is at the center of the classic brain reward network, receiving the densest dopaminergic input, as well as strong innervation from prefrontal cortical areas, midline thalamus, amygdala, and hippocampus (Cardinal et al., 2002; Haber and Knutson, 2010; Ikemoto, 2007; Mogenson et al., 1980). Two populations of MSNs, D1- or D2-containing, provide the main output source from the striatum (Le Moine and Bloch, 1995). Within the NAC, a selective D1-containing MSN population projects back to predominantly GABAergic neurons within the VTA (Heimer et al., 1991; Kalivas et al., 1993; Xia et al., 2011), while both D1- and D2-containing MSNs project to the ventral pallidum (Lu et al., 1998; Tripathi et al., 2010), which sends projections to the mediodorsal thalamus (Kupchik et al., 2015; Tripathi et al., 2013). NAS has similar MSN populations projecting to the VTA, lateral hypothalamus, ventral pallidum and brainstem regions (Gangarossa et al., 2013; Heimer et al., 1991; Zahm and Brog, 1992). The NAS is complicated by a higher proportion of coexpressing D1 and D2 receptors (Bertran-Gonzalez et al., 2010). The NAS also has regionally and functionally distinct dopaminergic sub-regions: the medial and lateral NAS compartments (Lammel et al., 2008, 2011). Within the medial NAS, the dorsal and ventral portions also exhibit unique connectivity and function (Castro and Bruchas, 2019).

Dorsal Striatum.

The dorsal striatum receives dense dopaminergic innervation from the SNc and lateral portion of the VTA (Haber et al., 2000; Ikemoto, 2007; Lerner et al., 2015), as well as strong projections from somato-motor cortical areas and lateral thalamic motor nuclei (Smith et al., 2004). Projections from the dorsal striatum include D1-containing MSNs projecting to the internal globus pallidus (GPi) and the substantia nigra pars reticulata (SNr) (Aubert et al., 2000; Gerfen et al., 1990; Matamales et al., 2009). D2-containing MSNs indirectly project to the GPi and SNr via the globus pallidus external (GPe) (Gerfen, 1992b). These populations generally delineate the ‘direct’ and ‘indirect’ pathways, but a small proportion of D1- and D2-coexpressing populations project both directly and indirectly to the GPi and SNr (Lévesque et al., 2003; Nicola et al., 2000; Surmeier et al., 1996).

Recent focus on the posterior “tail” portion of the dorsal striatum has shown its unique anatomical properties (Griggs et al., 2017; Watabe-Uchida and Uchida, 2018). The rodent striatal tail receives input from dopamine neurons located in the most lateral portion of the SNc, and unique non-dopaminergic inputs from sensory regions, like the visual, auditory, and gustatory cortices, the dorsal raphe, basolateral amygdala and subthalamic nucleus (Griggs et al., 2017; Jiang and Kim, 2018; Kim et al., 2014; Menegas et al., 2015). Interestingly, midbrain neurons that project to the tail do not cluster in an organized manner, in contrast to canonical projection populations to the ventral or dorsal regions (Menegas et al., 2015). In primates, the tail of the caudate also has unique connectivity, compared to the rest of the striatum, including inputs from the temporal and occipital cortex (Griggs et al., 2017; Kemp and Powell, 1970; Yamamoto et al., 2012).

Striatal microcircuitry.

A variety of anatomical mechanisms can shape MSN activity and afferent dopamine signals at axon terminals in the striatum. Glutamatergic input from the cortex, thalamus, and other limbic structures, for example, can modulate dopamine release (Floresco et al., 1998; Jones et al., 2010; Krebs et al., 1991). Several interneuron populations also exist in the striatum, creating a local microcircuitry that is critical for shaping dopamine signal independent of dopamine neuron action potentials. These include so-called “fast-spiking” interneurons, which contain parvalbumin, and a number of other cell types that have unique electrophysiological properties (Berke, 2011; Burke et al., 2017; Owen et al., 2018; Wilson et al., 1990). A prominent group, the cholinergic interneurons (CINs) comprise only ~2% of neurons in the striatum (Bolam et al., 1986; Oorschot, 1996; Phelps et al., 1985) but are well situated to affect dopamine signaling, as they can regulate plasticity at glutamatergic inputs and also synapse densely with dopamine terminals (Descarries and Mechawar, 2000; Zhou et al., 2003). Nicotinic and muscarinic receptors located directly on dopamine terminals (Jones et al., 2001) regulate dopamine release (Exley et al., 2008; Exley et al., 2011; Cachope et al., 2012; Threlfell et al., 2012). In the rat, CINs share similar morphological features between the dorsal and ventral regions on the striatum (Ma et al., 2014; Phelps and Vaughn, 1986; Phelps et al., 1985). Through a number of mechanisms, terminal modulation of dopamine release regulates dopamine actions on MSN populations to influence local signaling dynamics and function, reviewed in detail below.

Midbrain-striatal connectivity.

The VTA and SNc not only project to the striatum in a topographic manner, they also receive anatomically organized inputs from the striatum. Striatal MSNs project back to the VTA and SNc, creating a midbrain-striatum-midbrain network that is thought to be is organized as a series of loops (Haber et al., 2000). Medially located dopamine neurons in the VTA project to the most medial and ventral portion of the striatum. There, some MSNs they target project back to the VTA near dopamine neurons that innervate slightly more lateral and dorsal parts of the striatum. This continues, back and forth, until the most lateral portion of the SNc, which innervates the DLS and tail portion. This loop architecture has been demonstrated anatomically in both nonhuman primates and rodents (Aoki et al., 2019; Haber et al., 2000; Ikemoto, 2007).

Open questions and future directions

The ventral midbrain, including the VTA and SNc, and the striatum, are among the best characterized brain regions in rodents from classic neuroanatomical investigations. Modern anatomical methods offer the opportunity to build on this work to begin to define these systems along new domains.

• Species differences in dopamine system organization: We are focusing primarily on rodents, and while in broad terms the dopamine system is organized similarly across all mammals, there are important differences that require further investigation. For example, a much larger fraction of dopamine neurons project to the frontal cortex in primates, compared to rodents, implicating dopamine in a number of unique cognitive and decision-making functions (Haber and Knutson, 2010). As another example, mice have a significant number of VTA neurons that contain mRNA for tyrosine hydroxylase, the enzymatic precursor for dopamine, but do not actually make the protein, indicating they likely do not release dopamine, a discrepancy that does not exist for rats (Lammel et al., 2015; Saunders et al., 2015b; Yamaguchi et al., 2015). These results underscore how important species is for consideration of dopamine system heterogeneity. Modern comparative anatomy approaches offer promise for systematic cross-species circuit characterization (Balsters et al., 2019; Heilbronner et al., 2016).

• Re-conceptualizing classic dopamine circuit boundaries: New methods for anatomical and molecular profiling allow for a more expansive characterization of how different dopamine neurons and their terminal locations are organized (Poulin et al., 2018). It will be important to fully integrate these emerging perspectives into existing anatomical frameworks for the classic VTA and SNc, which may lead to new functional classifications (Heymann et al., 2019).

• 3-Dimensional characterization of the organization of the striatum: Most research has focused on distinguishing the striatum in 2-D space along the dorso-ventral or medio-lateral axis. Initial investigations into 3-D brain space for anatomical characterization of the striatum suggests the need for restructuring of classic striatal sub-domain framework, based on modern anatomical approaches for systematic input-output tracing, and clustering along molecular, circuit, and/or functional domains (Ekstrand et al., 2014; Fürth et al., 2018; Hintiryan et al., 2016; Hunnicutt et al., 2016; Nectow et al., 2017).

2. Circuit and sub-regional functional heterogeneity

Circuit inputs to dopamine neurons drive a variety of functions. The superior colliculus conveys short-latency sensory information about salient stimuli to both the VTA and SNc (Coizet et al., 2003; Comoli et al., 2003; Dommett et al., 2005). The SNc and lateral VTA receives additional information about stimulus saliency and valence from somatosensory and motor cortices that contribute to movement (Bromberg-Martin et al., 2010b; Haber and Knutson, 2010). Movement is refined via inhibitory inputs from the pars reticulata portion of the substantia nigra, which serves as a feedback loop from basal ganglia output (Grace and Bunney, 1984; Mailly et al., 2003; Rizzi and Tan, 2019). The lateral VTA receives stimulus salience and valence (Ono et al., 1986), as well as internal state information that modulates motivation (Burton et al., 1976; Morgane, 1961) from the lateral hypothalamus and ventral pallidum (Mahler et al., 2014; Nieh et al., 2015; Watabe-Uchida et al., 2012). Taken together, inputs to dopamine neurons from basal ganglia, brainstem, and local GABAergic interneurons signal both mixed and distributed variables related to associative learning, that are filtered and integrated in the midbrain to produce functionally divergent dopamine neuron activity patterns to affect striatal sub-region targets (Eshel et al., 2015; Keiflin and Janak, 2015; Tian et al., 2016).

Divergent projections into the striatum from VTA and SNc dopamine neurons, coupled with reciprocal connectivity to the midbrain, create unique feedback networks. As such, midbrain dopamine neurons are well positioned to produce diverse functional outputs, contributing to learning, motivation, and movement. Broadly, dopamine neurons projecting to the striatum guide reward seeking, signaling information that allows animals to make predictions about the world, select appropriate actions, and generate a motivational drive to spur appropriate movements in response to, and in pursuit or avoidance of, rewards and threats in the environment. We will now review work, much of it recently reported, delineating circuit and striatal sub-regional functional heterogeneity of dopamine neurons.

The Midbrain: SNc and VTA.

In the context of reward learning, midbrain dopamine neurons assign motivational value to stimuli and actions. From an extensive literature, we know that deficits in VTA and SNc dopamine signaling typically impair learning and reward-directed behaviors, or movement planning, execution and vigor, respectively. Exaggerated VTA and SNc dopamine signaling, conversely, underlies compulsive motivation and behavioral inflexibility, which occur in diseases that sensitize the dopamine system, such as addiction (Cardinal et al., 2002; Everitt and Robbins, 2005; Robinson and Berridge, 1993; Wise, 2004). Recent work taking advantage of modern neural targeting methods has begun to isolate specific functions to different populations of dopamine neurons, which we will review here.

Activation of dopamine neurons has long been implicated in reinforcement, as demonstrated by intracranial electrical self-stimulation studies (Corbett and Wise, 1980; Fibiger et al., 1987; Shizgal and Murray, 1989; Wise, 2004). In recent years, optogenetics allowed for confirmation of a number of hypotheses about basic dopamine functions generated from those studies. For example, activation of dopamine neurons reinforces instrumental actions (Ilango et al., 2014a; Kim et al., 2012; Steinberg et al., 2014; Witten et al., 2011). Phasic stimulation of dopamine neurons drives Pavlovian learning to create conditioned stimuli that can elicit behaviors on their own (Saunders et al., 2018; Sharpe et al., 2017), supports conditioned place preference (Tsai et al., 2009), and strengthens ongoing reward-seeking actions (Adamantidis et al., 2011). Activation of dopamine neurons as animals make choices can bias and accelerate behavioral output (Hamid et al., 2016; Howard et al., 2017; Soares et al., 2016), while activation as animals consume rewards can drive learning about predictive cues (Steinberg et al., 2013). Conversely, inhibition of dopamine neurons drives extinction learning, while excitation blunts extinction of cue-reward associations (Chang et al., 2017; Fischbach-Weiss et al., 2018; Steinberg et al., 2013). Together these studies clearly demonstrate the widely accepted view that, broadly, dopamine neurons contribute to learning and motivation. It is becoming clear, however, that there is considerable heterogeneity in dopamine neuron function depending on their anatomical location and projection target (Howe and Dombeck, 2016; Parker et al., 2016; Saunders et al., 2018).

A series of recent studies (Saunders et al., 2018) revealed a key distinction in the function of SNc and VTA dopamine neurons. Optogenetic activation of either VTA or SNc dopamine neurons, in the absence of delivery of an external reward, is sufficient to turn a neutral predictive cue into a conditioned stimulus, with important region-specific differences. Cues that predict activation of VTA dopamine neurons evoke cue-directed approach behavior and acquire conditioned value that reinforces instrumental actions in the absence of stimulation. SNc cues, conversely, evoke vigorous but undirected movement, and they do not reinforce actions on their own. This suggests a selective role for VTA dopamine neurons in Pavlovian learning that assigns incentive salience to environmental cues, which is critical for persistent and adaptive reward pursuit (Berridge, 2007; Flagel et al., 2011; Saunders and Robinson, 2013). Furthermore, these data show that SNc dopamine neurons can increase responding to a conditioned cue without increasing the value of that cue, suggesting dissociable dopamine contributions to learning versus motivation.

The ability of VTA dopamine neurons to assign value to reward cues may stem from their ability to signal information about reward identity (Stalnaker et al., 2019; Takahashi et al., 2017) and learned “hidden states” (Starkweather et al., 2017), in contrast to SNc dopamine neurons (Keiflin et al., 2019), whose role in learning specific cue-reward associations, independent of nonspecific movement invigoration, is less clear. Whether reward identity is fully represented in the firing patterns of VTA dopamine neurons, or in downstream activity, is also unclear.

VTA and SNc dopamine neurons also have different roles in spontaneous movement generation independent of learning (Coddington and Dudman, 2019). Optogenetic activation of SNc dopamine neurons evokes movement, and their activity encodes self-paced movement initiation but not specific movement sequences (Dodson et al., 2016; da Silva et al., 2018), suggesting they contribute to general movement invigoration more than goal-directed pursuit (Saunders et al., 2018). VTA dopamine neurons are less responsive to spontaneous movements (Howe and Dombeck, 2016), and are instead engaged when animals emit cue- or goal-directed movements. Thus, in isolation, dichotomous VTA and SNc roles in behavior are clear. However, outside of targeted optogenetic manipulations, in intact behaving animals, VTA and SNc dopamine neurons appear to send parallel signals to assign value and invigorate movement for dynamic navigation of the environment.

These results show dissociable fundamental functions for VTA and SNc dopamine neurons in reward prediction, value assignment, and action invigoration (summarized in Figure 1). It remains unclear exactly how VTA and SNc dopamine neurons signal heterogeneous functions, given that in vivo measurements of their activity often suggest relatively uniform encoding of behavior, at least for simple learning paradigms, in head-fixed recording preparations (Coddington and Dudman, 2018; Eshel et al., 2016; Schultz et al., 1997). Other evidence suggests, however, that dopamine’s multiple functions stem from a combination of diverse electrophysiological properties (Farassat et al., 2019; Lammel et al., 2011; Lerner et al., 2015; Margolis et al., 2008), firing patterns and release during behavior (Engelhard et al., 2019; Howe and Dombeck, 2016; Matsumoto and Hikosaka, 2009; da Silva et al., 2018), and sub-regional specialization within the striatum (Howe and Dombeck, 2016; Lerner et al., 2015; Parker et al., 2016; Saunders et al., 2018; Yang et al., 2018; Yuan et al., 2019).

Figure 1. Heterogeneous functions for VTA and SNc dopamine circuits.

Dopamine neurons in the VTA and SNc send topographically organized projections anterior, into the striatum. Dopamine neurons positioned medially, in the VTA, project to the ventral portion of the striatum, the NAS and NAC. Lateral dopamine neurons, located in the SNc, project to the dorsal portion of the striatum, the DMS and DLS. Dopamine neurons projecting throughout the striatum support instrumental reinforcement, but subpopulations exhibit distinct functions in Pavlovian learning. NAC (but not NAS) projecting dopamine neurons participate in reward prediction and assignment of incentive value to stimuli and actions. DLS projecting dopamine neurons, conversely, do not assign value, but can invigorate actions.

Ventral striatum.

The ventral striatum is conceptualized as a ‘limbic-motor interface’ (Mogenson et al., 1980), where a variety of affective and reward-related information streams converge to generate behavioral output. Broadly, dopamine in the ventral striatum contributes to the formation of stimulus-outcome associations to guide flexible, goal-directed reward-seeking behavior. Within the ventral striatum, considerable attention is given to functional distinctions between the NAC and NAS subregions, which have unique anatomical input-output connectivity.

The medial portion of the NAS receives dopaminergic input from the ventromedial VTA (Breton et al., 2019; Ikemoto, 2007; Saunders et al., 2018). Diverse functions are ascribed to NAS dopamine, but a general consensus is that it is critical for signaling reinforcement associated with the receipt of rewards, and representing ongoing motivational drive in reward seeking, rather than associative learning per se (Bassareo and Di Chiara, 1999; Kelley and Delfs, 1991; Saddoris et al., 2015; Saunders et al., 2018; Wyvell and Berridge, 2000). Overall, the NAS is heterogeneous compared to the rest of the ventral striatum. A larger portion of medial NAS-projecting dopamine neurons co-release glutamate, which contributes to aversive motivational states (Morales and Root, 2014), and positions them to have unique functions compared to other striatal sub-regions. Within the medial portion of the NAS, there is also considerable heterogeneity in reward-related functions across dorsal/ventral and rostral/caudal axes, reflecting hotspots of endogenous opioid activity and dopamine signaling (Castro and Berridge, 2014; Hasani et al., 2015; Richard and Berridge, 2011). It remains unclear to what extent dopamine signals, and/or multiplexed signaling of multiple transmitters with dopamine, contribute to medial NAS heterogeneity. There is also some evidence that the lateral portion of the NAS, which lies ventral to the NAC sub-region, has unique functions and connectivity within the ventral striatum (Lammel et al., 2011, 2014; Yang et al., 2018). Lateral NAS projecting dopamine neurons sit in the ventrolateral portion of the VTA, contiguous with the SNc (Lammel et al., 2014; Yang et al., 2018), and may be positioned to relay information between the ventral and dorsal striatum, which could be important for learning-related plasticity as part of the striatal loop system.

In comparison to the medial NAS, the NAC receives input from dopamine neurons positioned more dorsally and laterally within the VTA (Breton et al., 2019; Ikemoto, 2007; Lammel et al., 2014; Saunders et al., 2018). Critically, across all brain regions receiving VTA input, the VTA-NAC projection is the only one for which a majority of neurons are dopaminergic (Breton et al., 2019). As such, most of the broad hypotheses about dopamine function in associative learning stem from measurement and manipulation of NAC dopamine. Generally, NAC dopamine signaling is critical for Pavlovian cue-reward learning and cue-directed or evoked motivation (Di Ciano et al., 2001; Day et al., 2007; Flagel et al., 2011; Hamid et al., 2016; Mohebi et al., 2019; Saddoris et al., 2015; Saunders and Robinson, 2012; Saunders et al., 2018; Wassum et al., 2013).

Dorsal Striatum.

The dorsal striatum is critical for associating stimuli, actions, and outcomes. This is particularly apparent in the context of instrumental learning, where dopamine release functions in concert with sensory-motor cortical inputs to the dorsal striatum. Within the dorsal striatum, there is considerable functional heterogeneity across the DMS and DLS sub-regions. Several studies show that dopamine in the DMS contributes to learning associations between actions and outcomes (Gremel and Costa, 2013; Hilario et al., 2012; Yin et al., 2005). This supports so called “goal-directed” behaviors, or those that are sensitive to changes in the value or content of subsequent outcomes. The DLS, in contrast, generally contributes to stimulus-response learning, which supports so called “habit-like” behaviors, those that are insensitive to changes in outcome value or delivery conditions (Balleine et al., 2007; Dickinson, 1994). Blockade of dopamine signaling within the DMS and DLS generally impairs goal-directed and habitual responding, respectively, under a variety of conditions (Balleine et al., 2007, 2009; Vandaele and Janak, 2018).

The distinction between goal-directed and habit-like behavioral control for DMS and DLS is not always clear cut (Malvaez et al., 2018; Vandaele and Janak, 2018), and recent functional anatomical work adds further complexity to our understanding of dorsal striatal function. Dopamine neurons projecting to the DMS and DLS are organized topographically across a medial-to-lateral gradient within the SNc (Lerner et al., 2015). DMS and DLS projecting dopamine neurons are similarly activated by reward receipt but show opposing responses to aversive stimuli. DLS dopamine neurons are activated by aversive stimuli, suggesting that they have unique functional relevance outside of a typical reward-learning framework (Lerner et al., 2015).

Striatal tail-projecting dopamine neurons, unlike many others, show relatively little value-related encoding and instead have activity that scales with the novelty and magnitude of stimuli. Unlike dopamine projections to the dorsal and ventral striatum (Ilango et al., 2014a; Saunders et al., 2018; Witten et al., 2011), tail-projecting dopamine neurons do not support positive reinforcement, as measured by optogenetic self-stimulation procedures. Activation of tail dopamine neurons reinforces the avoidance of potentially threatening novel stimuli, and tail dopamine stimulation can reinstate avoidance of a familiar object (Menegas et al., 2018). The tail of the striatum has been studied more thoroughly in non-human primates, where it has distinct motor control functions. For example, in monkeys, the tail portion of the caudate integrates the visual streams to guide visual saccades based on reward preference (Griggs et al., 2017; Hikosaka et al., 2000; Yamamoto et al., 2012). Whether the tail dopamine neuron threat responses and DLS dopamine neuron activity in response to aversive stimuli (Lerner et al., 2015; Matsumoto and Hikosaka, 2009) are functionally similar, or distinct, remains to be determined.

Striatal microcircuitry.

Local striatal microcircuits, which can modulate glutamatergic inputs, dopamine release, and MSN activity, also influence learning, motivation, and movement. Much of the research in this domain has focused on the function of cholinergic interneurons. For example, CIN activity can inhibit MSNs in vivo (Witten et al., 2010), and a pause in CIN activity induces learning-related plasticity mechanisms (Nair et al., 2015). Striatal CIN activity becomes synchronized during reward learning, and activity patterns correlate with aspects of associative learning (Aosaki et al., 1994a, 1995; Apicella, 2007; Goldberg and Reynolds, 2011; Kimura et al., 1984). Within the ventral striatum, CINs regulate the motivational impact of reward-paired cues. Optogenetic stimulation of CINs concurrent with reward-paired cue presentations inhibits the ability of that cue to invigorate reward-seeking (Collins et al., 2016a). Brief suppression of ventral striatal CIN activity can also enhance the acquisition of Pavlovian fear learning (Brown et al., 2012), suggesting a broader role in learning beyond appetitive motivation.

The functional significance of dorsal striatal cholinergic-dopamine microciruitry is less clear, but some evidence suggests acetylcholine activity in the dorstal striatum is important for behavioral flexibility under changing reward contingencies (Ragozzino et al., 2009). Activity in dorsal striatal CINs and dopamine terminals encode distinct features of spontaneous movement (Howe et al., 2019). Given acetylcholine’s ability to modulate dopamine release, this suggests that dorsal striatum CINs can shape features of nigrostriatal dopamine-mediated actions. Dorsal striatal CINs may organize large-scale striatal activity, based on evidence that CIN activation can spread in wave-like patterns spanning considerable anatomical distances (~1mm) across the dorsal striatum (Rehani et al., 2019). Recent data suggests that dopamine terminal activity in the dorsal striatum can also organize into wave formations (Hamid et al., 2019), and MSNs show ramping activity associated with reward seeking (Cui et al., 2013; London et al., 2018). Thus, a possible role for dorsal striatal CINs is to coordinate dopamine release and MSN activity across medial-lateral/anterior-posterior domains of the striatum. In the DMS, CIN activity tracks decision state (Stalnaker et al., 2016), consistent with a role in flexible behavior. Given the reciprocal connectivity of the nigrostriatal system and divergent functions for DMS and DLS (Gremel and Costa, 2013; Haber et al., 2000; Lerner et al., 2015), coordinated CIN activity may be important for engaging spatially distinct goal-directed and habitual striatal modules, to dynamically shift between decisions states in reward seeking.

Local sculpting of dopamine signals within the ventral and dorsal striatum can modulate cue-triggered motivation, reward-related learning, movement invigoration, and action selection. Together, these data suggest that one powerful contributor to dopamine function is moment-to-moment control of its release by, and its effects on, local microcircuitry in striatum.

Circuit-level functional connectivity.

The looping architecture of the midbrain-striatal system is thought to have a functional role in reward learning and decision making over time. In this framework, experience-dependent plasticity results in a progressive engagement of the network, from initial reliance on ventral striatal dopamine to form cue-reward associations and assign value to stimuli to the formation of DLS-dependent stimulus-response associations (Keiflin and Janak, 2015). Initial dopamine-mediated learning may potentiate MSNs in the ventral striatum that project back to GABA neurons in the midbrain (Bocklisch et al., 2013; Ikemoto, 2007). Hypothetically, as learning continues, becoming more streamlined and efficient, DLS projecting dopamine systems become engaged through recruitment of these loops. This framework is consistent with general notions of the functional role of the DLS in the formation of habits and the execution of skilled motor plans, which usually occur after extended experience (Everitt and Robbins, 2005).

This functional organization of midbrain-striatal circuits is generally accepted, but surprisingly few studies have directly tested it. In vivo evidence for a progressive engagement of ventral to dorsal striatal systems, potentially via a serial looping architecture, comes from measurements of striatal dopamine release, using electrochemical recording methods. These studies show that initial responding for rewards, including food and cocaine, evokes large dopamine responses in the NAC (Aragona et al., 2009; Clark et al., 2013; Flagel et al., 2011; Willuhn et al., 2012). After extended training, cue or outcome-related ventral striatal dopamine signals generally diminish (Clark et al., 2013; Day et al., 2007), and in some cases, emerge in the dorsal striatum (Brown et al., 2011; Willuhn et al., 2012, 2014). Furthermore, dopamine signaling in the ventral striatum becomes less important for the execution of some instrumental behaviors after extended training, while dorsal striatal dopamine signaling becomes more important for well-learned instrumental actions (Belin and Everitt, 2008). How this happens is unclear, but cocaine exposure is known to potentiate the activity of D1 MSNs in the nucleus accumbens that synapse onto GABAergic VTA neurons, which could disinhibit more dorsal projecting dopamine neurons (Bocklisch et al., 2013). It is unclear how that plasticity actually affects SNc dopamine projections, or if a similar mechanism occurs in non-drug learning. In contrast to instrumental learning, ventral striatal dopamine signaling remains critical for the execution of Pavlovian conditioned behaviors, and functional control does not emerge in the dorsal striatum, even after extended training (Fraser and Janak, 2017), underscoring that distinct engagement of striatal networks occurs for different forms of learning. This issue is ripe for further study but, together, these results also highlight an important dissociation: cues, actions, and rewards may engage striatal circuits, but that does not necessarily mean that that signal has functional importance for the behavior.

Other work suggests that learning-related plasticity in the striatum does not necessarily follow in a ventromedial-to-dorsolateral progression. Within the nigrostriatal pathway, dopamine neurons projecting to the DMS and DLS are reciprocally connected to MSNs projecting back to SNc, but some lateral SNc dopamine neurons also project to the DMS (Lerner et al., 2015), suggesting that information can also flow in a lateral-to-medial direction across the striatum. The behavioral conditions in which this occurs are unclear, but it is important to note that the transition to habit-like behavior is not permanent. Depending on the behavioral conditions, animals can shift dynamically between states that are goal-directed (i.e., outcome-sensitive) and habitual (i.e., outcome-insensitive) (Barker et al., 2017; Gremel and Costa, 2013). Switching from habit-like to goal-directed behavior, at least in some situations, involves simultaneous disengagement of DLS activity and up-regulation of DMS activity (Gremel and Costa, 2013). Therefore, well-learned goal-directed behaviors could still be influenced by lateral SNc dopamine neurons, as predicted by serial spiraling architecture, but via their signaling into the DMS. If information flows lateral-to-medial across nigrostriatal circuits, this could be important for toggling between habitual and goal-directed behavioral states, to allow for adaptive action selection. Notably, in sequence learning, DMS and DLS activity evolves in parallel, rather than in series (Thorn et al., 2010), and extended training does not necessarily result in progressive loss of DMS activity (Vandaele et al., 2019), suggesting that behavioral features (e.g., responding to well-learned but changing contingencies, versus progressing in skill learning) determine how the dorsal striatum is engaged.

While the anatomical basis for striatal loop architecture is strong, and there is some functional evidence, there is as yet no clear demonstration that serial plasticity across a hypothetical loop starting in ventromedial striatum and ending in dorsolateral striatum occurs during learning. Broadly, it is unclear how VTA, SNc, and ventral and dorsal striatum all interact in vivo, and under what conditions changes in ventral versus dorsal striatal dopamine signaling, as described above, require greater network changes, such as engagement of corticostriatal or thalamostriatal inputs (Aoki et al., 2019; Athalye et al., 2018; Balleine and O’Doherty, 2010; Cover et al., 2019; Gremel and Costa, 2013; Mandelbaum et al., 2019; Rothwell et al., 2015; Smith and Graybiel, 2013). Notably, relatively little work has examined the central portion of the striatum. Investigations there could be informative for testing the functional serial midbrain-striatal loop architecture, which would implicate the central striatum as a transition zone between ventral and dorsal. If this were the case, we would expect dopamine signals there to emerge after the ventral striatum, but before the dorsal striatum. Alternatively, the central striatum could have unique functions and dopamine signaling dynamics during learning, which might suggest that ventral and dorsal striatal dopamine systems are comprised of parallel, but independent, loops rather than one continuous serial network.

Open questions and future directions

In this section, we reviewed emerging evidence for functional heterogeneity in striatal dopamine circuits. Together, this work demonstrates that dopamine functions are parcellated across a number of striatal circuits and subdomains, where localized dopamine signals mediate learning about rewarding and threating stimuli and invigorate the appropriate action to produce a dynamic behavioral output that animals use for adaptive reward seeking. There are a number of broad open questions that will be important to answer in order to better understand how diverse dopamine functions are integrated during behavior.

• Establishing network-level functional dynamics of midbrain-striatal circuits: As discussed above, a growing literature provides clear evidence for circuit and striatal sub-regional specific functions for dopamine. It is less clear how these sub-systems interact in vivo to control behavior. Moving forward, it will be important to better understand how midbrain-striatal systems are engaged at a broader network level during behavior. One limitation to progress on this front has been the technological bottleneck preventing widespread activity measurements across large areas of deep brain tissue, like the striatum. Recent advances in large-scale recording methods that can access subcortical structures – such as high density fiber photometry and electrophysiology (Hamid et al., 2019; Jun et al., 2017; Pisano et al., 2019; Sych et al., 2019) offers promise for uncovering novel functional dynamics in the striatum. These approaches, combined with brain-wide circuit- and network-level functional imaging (Decot et al., 2017; Heilbronner et al., 2016; Lohani et al., 2017) will also be important for translation of rodent findings to humans.

• Functions of neurotransmitter co-release from dopamine neurons: Many dopamine neurons corelease other neurotransmitters, including GABA and glutamate (Morales and Margolis, 2017; Morales and Root, 2014; Tecuapetla et al., 2010; Tritsch et al., 2012) and co-release varies depending on dopamine neuron subpopulation. There have been some important investigations (Berrios et al., 2016; Birgner et al., 2010; Seal and Edwards, 2006; Trudeau et al., 2014; Yoo et al., 2016), but in general less is known about how co-transmission functions in vivo.

• Functional consequences of genetic diversity of dopamine neurons: Dopamine neurons express genetic markers for a number of neuropeptides and other signaling molecules, with clear systematic anatomical heterogeneity (Morales and Margolis, 2017; Poulin et al., 2018). The functional consequences of this genetic diversity, and the extent to which dopamine neurons actually release non-traditional transmitter molecules in vivo, is an area ripe for more investigation.

• Investigating the function of dopamine neurons outside of the VTA and SNc: Dopamine neurons exist in several other brain areas, including the olfactory bulb, retrorubral field, hypothalamus, red nucleus, retina, and dorsal raphe (Björklund and Dunnett, 2007; Brown et al., 1976; Lindvall and Björklund, 1974; Nair-Roberts et al., 2008; Stratford and Wirtshafter, 1990). These subpopulations have important functional differences from the broad movement, learning, and motivational roles of VTA and SNc dopamine neurons (e.g., Matthews et al., 2016), but remain relatively understudied.

• Integrating functional insights from non-mammals: Dopamine systems are well studied in invertebrate model systems such as drosophila (Burke et al., 2012; Claridge-Chang et al., 2009; Handler et al., 2019; Liu et al., 2012). While they are broadly different in gross anatomy and phylogenic history from mammals, these studies demonstrate that many core mammalian functions of dopamine in associative learning and movement control are in place in invertebrates, as well as birds (Gadagkar et al., 2016). Invertebrate studies in particular will offer insights into in vivo genetic and molecular mechanisms of dopamine signaling that are difficult or impossible to study in mammals.

3. Modes of striatal dopamine system action: activity and release

The anatomical and functional organization of striatal dopamine circuits is a major focus of study, but to fully appreciate the complexity of this network, it is important to consider the various modes of signaling and mechanisms of dopamine release modulation that are in place in the striatum. In this section, we will review classic work on dopamine neuron activity states, and more recent studies demonstrating complex release patterns and local control of dopamine signaling within striatal compartments.

A key aspect of dopamine heterogeneity stems from cellular physiology, as subpopulations of dopamine neurons exhibit different electrophysiological properties and diverse activity patterns (Lammel et al., 2011; Margolis et al., 2008). Canonically, dopamine neurons engage in two activity states in vivo: slow, low-frequency firing, called ‘tonic’, or rapid bursts of activity, called ‘phasic’ (Grace, 1991; Grace et al., 2007). Similarly, dopamine release at the terminal is usually divided into short (seconds) phasic and long (minutes to hours) tonic release, thought to usually reflect changes in activity within the dopamine cell body. Experimentally, these signaling modes correspond to dopamine’s functions in trial-by-trial learning versus extended motivational states, respectively. The distinction between phasic and tonic dopamine profiles is not clear-cut, however, as a brief phasic dopamine neuron activity can be followed by a minutes-long increase in dopamine concentration in the ventral striatum (Lohani et al., 2018). Growing evidence suggests a potential third profile, a ‘ramp-like’ pattern of dopamine release (Collins et al., 2016b; Hamid et al., 2016; Howe et al., 2013). Critically, ramping dopamine release can occur independently of ramping of dopamine neuron activity (Mohebi et al., 2019), suggesting terminal modulation mechanisms refine release characteristics locally within the striatum. We will now discuss these modes of dopamine cell activity and release, and mechanisms for terminal modulation, focusing on the striatal cholinergic system.

Tonic activity and release.

Low-frequency tonic dopamine cell body activity occurs in the 2–5 Hz range (Goto et al., 2007). Tonic dopamine release induced by an increase in population activity is distinct from neurotransmission induced by burst-firing (Floresco et al., 2003). However, recent work showed that tonic-scale dopamine release is not necessarily due to changes in tonic cell body activity (Mohebi et al., 2019), illustrating that additional refinement of the signal occurs at the terminal. One regulatory mechanism occurs via cholinergic signaling, as blockade of β2 nicotinic receptors located on dopamine terminals suppresses tonic release (Zhang et al., 2009). Many studies report a slow rise in striatal dopamine levels during extended reward seeking (McCullough et al., 1993; Sokolowski et al., 1998; Cousins et al., 1999; Ostlund et al., 2011). This slow increase in dopamine release correlates with the amount of behavior output, but not amount of reward delivered (Sokolowski et al., 1998), suggesting a role in mediating effort expenditure. NAC dopamine depletions suppress effortful reward seeking for a high-palatable food when a less palatable, low-effort option is also available (Sokolowski and Salamone, 1998). Chronic dopamine depletions alter reward seeking particularly when the effort required to pursue reward is high (Salamone et al., 2007). Additionally, knocking down striatal dopamine transporters (DATs), which increases tonic activity without altering phasic activity, increases reward seeking (Cagniard et al., 2006).

Tonic dopamine is thought to encode net costs and benefits of the current situation, tracking reward rate and any associated costs over long time periods (Niv et al., 2007). In line with this theory, a change in tonic dopamine negatively correlates with effort required to obtain a reward (Ostlund et al., 2011). Tonic dopamine also tracks motivational state information, as tonic dopamine efflux is reduced when animals are sated, compared to a hungry state (Ostlund et al., 2011). Together, these data suggest that slower, tonic changes in dopamine concentrations in the ventral striatum control motivation broadly, to regulate effort expenditure based on physiological state and work demands (Figure 2C).

Figure 2: Working model of how diverse dopamine release profiles emerge from CIN-dopamine neuron interactions in NAC.

A. A decrease in NAC CIN activity, leading to inactivation of β2 nicotinic receptors on dopamine terminals, allows burst-firing of dopamine neurons to produce phasic release in response to the presentation of reward-paired cues, which invigorates reward-seeking actions. B. When dopamine neurons engage in burst-firing that coincides with bursts in CIN activity, dopamine release to the cues and reward-seeking actions, is suppressed, in part, via activation of β2 nicotinic receptors. C. When dopamine neurons and CINs are in both low firing states, this results in tonic dopamine release, mediating long-term motivation/effort allocation. D. When dopamine neurons are in a low firing state, NAC CINs may increase activity under conditions wherein sustained motivation is required to procure a distal reward, producing a ramp-like increase of dopamine release to mediate sustained motivation, via activation of β2 nicotinic receptors located on dopamine terminals.

Phasic activity and release.

Many studies show that phasic midbrain dopamine neuron activity (Cohen et al., 2012; Eshel et al., 2016; Keiflin and Janak, 2015; Schultz et al., 1997; Waelti et al., 2001) and phasic striatal dopamine release in the NAC (Day et al., 2007; Hart et al., 2014) closely correlate with the reward-prediction error term proposed by temporal difference reinforcement learning theories to mediate some forms of associative learning. As an animal learns that cues predict rewards, dopamine activity diminishes to the reward and emerges to the predictive cue (Cohen et al., 2012; Day et al., 2007; Schultz et al., 1997), although not always (Coddington and Dudman, 2018). In situations where multiple cues predict rewards, with extended training, phasic dopamine release diminishes to the cues most proximal to reward (Clark et al., 2013) and emerges to earlier cues, to contribute to reward expectation (Collins et al., 2016b).

Recent work has emphasized that the fundamentals of this system can operate independently of sensory input that typically accompanies physical interaction with rewards. Direct optogenetic stimulation of dopamine neurons, in the absence of an external reward, elicits phasic dopamine neuron activity (Saunders et al., 2018) and phasic dopamine release in the NAC (Covey and Cheer, 2019) that back-propagates to a cue predicting stimulation. In natural cue-reward learning, expectation-related suppression of reward-evoked dopamine signals partially arises from plasticity that recruits local GABA neurons in the midbrain, which inhibit dopamine neurons during the period of expectation after cue delivery (Eshel et al., 2015). Remarkably, dopamine release in response to direct optogenetic dopamine cell body stimulation, which largely circumvents the pre-synaptic inhibitory mechanism within the VTA (Covey and Cheer, 2019), diminishes when it becomes well predicted, suggesting that expectation-related suppression of dopamine release evoked by learned rewards likely also occurs via mechanisms at the terminal.

The role of burst activity or rapid release patterns in response to aversive events is informative to this notion, as the presence of an unexpected aversive outcome is thought to be encoded by this system the same way as the absence of an expected appetitive outcome, i.e., ‘worse than expected’. In line with this, dopamine neuron activity within the VTA generally decreases to a noxious stimulus (Ungless et al., 2004), though this is not entirely uniform within VTA, as some neurons are activated by the same noxious stimuli (Brischoux et al., 2009). Phasic release to aversive events is also heterogeneous depending on ventral striatal sub-region. For example, dopamine release is suppressed in the NAC and lateral NAS, but increased in the medial NAS, by foot-shock. Interestingly, after learning, a cue predicting shock also increases medial NAS dopamine release (Badrinarayan et al., 2012), suggesting that cue-evoked dopamine may not always encode the valence of a prediction error, but rather an absolute change between expectation and outcome (Yang et al., 2018).

In addition to mediating learning about predictive cues, phasic activity (Ilango et al., 2014b; Jin and Costa, 2010; Zweifel et al., 2009) and release (Phillips et al., 2003; Roitman et al., 2004; Wassum et al., 2011, 2012) encode key aspects of reward-seeking actions. Phasic NAC dopamine release precedes self-initiated action and relates to both effort and vigor of responding (Collins et al., 2016b; Ko and Wanat, 2016; Syed et al., 2016). NAC dopamine release preceding action initiation is observed under circumstances where the action is self-guided, as opposed to driven by external cues, wherein behavioral movement precedes dopamine release (Coddington and Dudman, 2019; Satoh et al., 2003; Syed et al., 2016). Additionally, phasic NAC dopamine release encodes the value of impending rewards but not costs to obtain them (Gan et al., 2010).

Ramp-like release.

A majority of studies investigating phasic or tonic dopamine profiles use experimental designs in which the actions and cues are closely linked to rewards, either temporally or spatially. Recent reports have profiled a potential third type of dopamine release in the ventral striatum that gradually increases as an animal navigates their environment or completes a series of actions to procure a distal reward, peaking near reward consumption (Collins et al., 2016b; Hamid et al., 2016; Howe et al., 2013). This ramp-like profile is more sustained than typical (<1sec) phasic release profiles but is contained within a single reward-seeking behavioral sequence, in contrast to a classic tonic dopamine profile.

Ramping release profiles may provide the motivational drive necessary to continue goal pursuit, as they scale with distance to reward (Howe et al., 2013). In addition to spatial navigation, dopamine ramps occur during instrumental action sequences, in which persistent motivation is also required to earn a distal reward (Collins et al., 2016b; Hamid et al., 2016). In this situation, ramping dopamine release occurs over time scales that bridge choices. Within ramps, dopamine release is dynamic with learning and expectation violation (Collins et al., 2016b), and overall ramping profiles encode a multiplexed signal that is thought to include prediction errors and current motivational state (Hamid et al., 2016). It is interesting to note that with extended instrumental training, the ramp-like profile of dopamine release disappears and only re-emerges when an animal takes an atypical path to navigate through a sequence task, which has implications for the role of these signals in motivation and value encoding with stereotyped actions (Collins et al., 2016b).

It remains unclear how the dopamine system generates an extended, but not exactly tonic, release profile. Understanding this is important, because dopamine ramps most directly coincide with the expression of reward seeking behavioral sequences. One potential explanation is ramping release follows from a ramping in cell body activity. Indeed, single unit dopamine neuron activity can show a positive slope ramp-like profile during a probabilistic Pavlovian task, where reward delivery is uncertain (Fiorillo et al., 2003) and in a self-guided instrumental task (Romo and Schultz, 1990). Other studies, however, have found a negatively sloped ramp-like change in activity when the probability of reward delivery is uncertain (Bromberg-Martin et al., 2010a; Fiorillo et al., 2008; Starkweather et al., 2017, 2018). Given the uncertainty of reward delivery, this negative slope has been interpreted to reflect a gradual negative reward prediction error, encoding the accumulation of ‘worse than expected’ across trials. That negatively sloped ramps encode a reward prediction error, as opposed to motivational or value signal, suggests a distinction between ramping profiles of dopamine cell activity and ramp-like release signals. Several other reports do not demonstrate a ramping-up profile in single unit activity (Bromberg-Martin and Hikosaka, 2009; Hollerman and Schultz, 1998; Hollerman et al., 1998; Matsumoto and Hikosaka, 2007, 2009; Mirenowicz and Schultz, 1996; Morris et al., 2004; Satoh et al., 2003), and recent work showed that dopamine release ramps do not correspond to phasic cell body activity (Berke, 2018; Mohebi et al., 2019). Thus, the evidence for single unit dopamine neuron ramping activity is mixed. Population-level activity may instead produce ramps, which could reflect sequential, overlapping activation patterns across single units that individually encode distinct aspects of a behavioral sequence, such as spatial, sensory, movement, and decision features (Engelhard et al., 2019). Collectively, such a serial cell body activation pattern could produce a ramp in dopamine release in the ventral striatum that encodes multiple aspects of the ongoing behavior.

Local modulation for diverse dopaminergic profiles.

Critically, dopamine cell body firing does not always track with or explain release (Cachope and Cheer, 2014; Mohebi et al., 2019; Sulzer et al., 2016), and thus an important feature of phasic/ramping release dynamics likely occurs via modulation at the terminal. Dopamine terminals contain a number of mechanisms for local modulation, including dopamine reuptake by DATs (Giros et al., 1996; Jones et al., 1998, 1999; Rice and Cragg, 2008) and suppression of release via activation of D2 dopamine auto-receptors (Beaulieu and Gainetdinov, 2011; Ford, 2014; Missale et al., 1998; Romanelli et al., 2010).

Local interneuron circuits near dopamine terminals also play a key role in control of dopamine release dynamics. Recent work implicates striatal CINs in a variety of dopamine regulatory mechanisms. Critically, the dynamics of cholinergic regulation of dopamine release at the terminal are dependent on the activity state of both neuronal populations, which has occluded a clear mechanistic understanding. In anesthetized rats, optogenetic stimulation of CINs in the NAC produces dopamine release independent of dopamine neuron stimulation (Cachope and Cheer, 2014), suggesting that acetylcholine release facilitates striatal dopamine release. The activity state of dopamine neurons matters, however, as a series of in vitro studies suggests a more dynamic mechanism (Exley and Cragg, 2008; Rice and Cragg, 2004; Threlfell et al., 2012; Zhang and Sulzer, 2004; Zhang et al., 2009). Blockade of nicotinic receptors coupled with low-frequency dopamine neuron stimulation suppresses dopamine release, in line with the notion that acetylcholine facilitates dopamine release. In contrast, blockade of nicotinic receptors coupled with high-frequency stimulation increases dopamine release, suggesting that acetylcholine can serve a gating function to suppress release when dopamine neurons burst-fire (Exley and Cragg, 2008; Threlfell and Cragg, 2011; Sulzer et al., 2016). Indeed, disrupting nicotinic receptor signaling can actually increase high-frequency dopamine release (Rice and Cragg, 2004b; Exley et al., 2008). Together, this work suggests that, based on the current firing mode of a dopamine neuron, acetylcholine release in the ventral striatum can bi-directionally modulate dopamine release. While the details will require direct testing, Figure 2 summarizes our working model of how dopamine release in the ventral striatum might be regulated by CINs across different behavioral and neuronal activity states.

A caveat of the in vitro work is that it assumes high-frequency electrical stimulation of dopamine neurons effectively mimics burst firing of cell bodies that occurs in vivo. Until recently it was largely unclear how acetylcholine modulates dopamine release in freely behaving animals during reward seeking. One hurdle is difficulty in examining the effects of acetylcholine receptor activity on dopamine release locally within the striatum, without simultaneously affecting dopamine cell bodies. A recent paper circumvented this issue by combining local pharmacological manipulation of acetylcholine signaling with simultaneous electrochemical measurements of dopamine release during behavior (Collins et al., 2016a). This work demonstrated that intra-NAC blockade of nicotinic receptors can enhance dopamine release to an unexpected reward, as well as to a reward-associated cue, and also increases the ability of that cue to invigorate reward-seeking actions (Collins et al., 2016a), suggesting that a pause in CIN transmission allows for dopamine neuron burst activity to produce phasic dopamine release and promote cue-motivated behavior (Figure 2A). Given that reward-paired cues elicit burst firing from dopamine neurons (Schultz et al., 1997), this result suggests that acetylcholine suppresses dopamine release in vivo when dopamine neurons are in a high, but not low, activity mode (Figure 2B-C), in line with the in vitro data. Additionally, rapid cholinergic modulation of striatal dopamine release has important behavioral consequences. Brief optogenetic stimulation of CINs in the NAC during presentation of a reward-paired cue suppresses the ability of that cue to invigorate reward seeking (Collins et al., 2019). This effect occurs via local acetylcholine action, as simultaneous blockade of β2 nicotinic receptors in NAC reversed the behavioral effect of optogenetic stimulation (Collins et al., 2019).

How dopamine and acetylcholine interact to produce heterogeneous functional profiles of release is not fully understood. Acetylcholine, via β2 activation, can suppress dopamine release, but if burst dopamine activity occurs prior to that, there is a rapid shutdown of the CINs via D2 receptor activity (DeBoer and Abercrombie, 1996; Maurice et al., 2004), suggesting that reciprocal suppression of either acetylcholine or dopamine release can occur depending on the current activity state of either cell type. Optogenetically stimulated acetylcholine induces dopamine release in vivo in anesthetized subjects (Cachope and Cheer, 2014), suggesting that activation of nicotinic receptors on dopamine terminals enhances release when dopamine neurons are firing in a low-frequency state, while blockade of β2 nicotinic receptors suppresses tonic release (Zhang et al., 2009). This is a potential mechanism by which the dopaminergic ramp-like profile could be generated at the terminal, given that release ramps dissociate from high-frequency dopamine neuronal firing (Mohebi et al., 2019). After an initial burst of dopamine neuron activity in response to a cue, reflecting expectation, dopamine neurons then shift to a low firing rate. If sustained motivation is then necessary to acquire the reward, CINs could sustain release at the terminal, creating a ramping release profile (Figure 2D).

It remains unclear how specific activity patterns of ventral striatal CINs could contribute to a ramp-like profile of dopamine release, as there is no direct evidence that CINs themselves display a ramp activity pattern. In the ventral striatum, most CIN studies to date involve single cell recordings, preventing understanding of the broader population activity that could regulate dopamine release, and CIN activity patterns typically aren’t measured in behavioral tasks where sustained motivation is required, that is, the conditions where dopamine ramps are likely to emerge. This requires further study, along with the possibility that acetylcholine release dissociates from action potentials in CINs, as it appears to do in dopamine neurons (Mohebi et al., 2019; Trulson, 1985). In the dorsal striatum, CIN and dopamine terminal activity can propagate in waves across relatively large swaths of tissue (Hamid et al., 2019; Rehani et al., 2018). It is unknown if ventral striatal CINs can also act synchronously, or if ventral striatal dopamine terminals activate in waves. One possibility, which requires direct testing, is that the consequences of dopamine-acetylcholine interactions in dorsal and ventral striatum are broadly distinct where, in dorsal striatum they result in wave patterns, and in ventral striatum they result in ramp patterns. Alternatively, given the similar activity-dependent functioning of α6/α4β2 nicotinic receptors across striatal sub-regions (Threlfell and Cragg, 2011), if dopamine ramps also occur in the dorsal striatum, CINs there may have a similar role in local modulation.

How are ventral striatal CINs regulated to facilitate or suppress dopamine release? CINs fire at low frequencies, from 3–10 Hz, maintained by intrinsic pacemaker functions (Bennett and Wilson, 1999; Wilson and Goldberg, 2006). CIN activity is also altered in response to a variety of inputs. Stimulation of glutamatergic thalamic inputs induces a brief increase of CIN activity followed by a cessation, a ‘burst-pause’ profile (Ding et al., 2010). Interestingly, recent work demonstrated that CIN pauses can also be driven by a decline of excitatory drive (Zhang et al., 2018). Lesion of dopaminergic inputs prevents CIN pauses (Aosaki et al., 1994b), while an increase in dopaminergic activity can induce a pause in CINs via D2 receptor activation (DeBoer and Abercrombie, 1996; Maurice et al., 2004). This varies by striatal subregion. In the dorsal striatum, dopamine stimulation induces a pause in CINs, whereas in the NAS, a burst-pause activity pattern is produced, occurring via co-release of glutamate (Chuhma et al., 2014, 2018). However, when measured together in vivo, CIN pause is not differentially regulated by dopamine neuron firing rate, suggesting this mechanism may be limited in its ability to induce dynamic pauses during behavior (Joshua et al., 2008; Morris et al., 2004). Long-range GABAergic projections from the VTA are also capable of inducing a pause in CINs (Brown et al., 2012). Additionally, an increase in acetylcholine can produce a suppression of CIN activity via M4 autoreceptors on CINs (Calabresi et al., 1998; Yan and Surmeier, 1996; Zhang et al., 2002). A number of other specific signaling molecules also influence CINs. Corticotrophin-releasing factor in the NAC or dorsal striatum, for example, can enhance CIN activity (Lemos et al., 2019). That multiple mechanisms modulate CINs suggests striatal circuitry is set up to tightly regulate acetylcholine signaling, allowing for broad behavioral effects via enhancement or suppression of dopamine release and MSN activity.

Broadly, dopaminergic signaling profiles, including kinetics, volume, and duration, differ across striatal sub-regions (Brown et al., 2011; Cragg et al., 2000; Howe and Dombeck, 2016; Jones et al., 1995; Sulzer et al., 2016), reflecting the combination of a number of factors, including differences in the intrinsic properties of dopamine neuron subpopulations, and sub-regional diversity in local microcircuitry control (Burke et al., 2017; Margolis et al., 2008; Threlfell and Cragg, 2011). A major influence is the distribution of DAT, which is in higher concentrations in the dorsal striatum (Ciliax et al., 1995). This means that on average, dorsal striatum dopamine is removed from extracellular space faster, relative to the ventral striatum, which has consequences for release profile dynamics (Calipari et al., 2012; Cragg and Rice, 2004; Garris and Wightman, 1994; Jones et al., 1995). For the striatal cholinergic system, the distribution and function of nicotinic (Exley and Cragg, 2008; Exley et al., 2011) and muscarinic (Threlfell et al., 2010) acetylcholine receptors differs across dorsal and ventral striatum (Threlfell and Cragg, 2011). M2 and M4 acetylcholine receptors, which can act as autoreceptors on CINs, are necessary for cholinergic regulation of dopamine release within the dorsal striatum, but only M4 receptors play a role in the ventral striatum (Threlfell et al., 2010). Other gross feature distinctions between the dorsal and ventral striatum, such as the organization of striosome and matrix compartments (Graybiel et al., 1981), which exhibit unique local circuitry, connectivity, and function (Crittenden and Graybiel, 2011; Smith et al., 2016), create microenvironments with unexplored functional dynamics.

Open questions and future directions

Dopamine neurons signal via multiple activity states, and dopamine release is under multiple regulatory mechanisms, many of which occur independent of cell body activity. This creates a complex landscape for describing dopamine neuron functions, because the activity of many additional cells must be taken into account. A number of open questions remain for developing a more complete picture of dopamine signaling dynamics.

• In vivo population-level recordings of dopamine neurons. The vast majority of dopamine neuron recordings studies have relied on measurement of single units, typically numbering in the 10s of cells. These have been formative for modern conceptions of dopamine physiology and function, but we still know little about how larger populations (dozens to 100s) of dopamine neurons are collectively engaged in vivo, during behavior. In the striatum, recent studies employing new methods for measuring activity across large areas of tissue are beginning to uncover novel activity dynamics that single cell studies cannot access (Hamid et al., 2019; Klaus et al., 2017; Owen et al., 2018; Parker et al., 2018). It will be important to extend these approaches to the midbrain, to assess large-scale dopamine dynamics, which could shed light on discrepancies between dopamine release and activity and unexpected functional dynamics.

• Multiplexed understanding of rapid dopamine release dynamics and interactions with striatal microcircuitry in vivo: Dopamine measurement methods have provided a wealth of information on how the dopamine system is engaged during behavior. For technical reasons, these methods have traditionally been unimodal (voltammetry, which measures dopamine release in isolation), or slow (microdialysis, which measures multiple chemical transmitters over minutes to hours). Recent technological advances in optical imaging methods (Engelhard et al., 2019; Kim et al., 2016) and genetically encoded biosensors (Patriarchi et al., 2018; Sun et al., 2018) will allow for a more comprehensive characterization of dopamine and local circuit dynamics in vivo. Activity indicators can be targeted to specific dopamine neuron subpopulations, circuits, and post-synaptic targets, which will provide insight that was previously impossible. Combined imaging of multiple sensors will allow for better understanding of how striatal microcircuitry is organized.

• Modulation of terminal release across striatal subdomains: The majority of the work discussed here on terminal modulation of dopamine release relates to mechanisms in the ventral and dorsal striatum, but it remains unclear if terminal modulation, via acetylcholine and other mechanisms, varies in a systematic way across striatum. Furthermore, how release may be differentially modulated across other striatal classifications, such as striosome and matrix compartments (Brimblecombe and Cragg, 2017), requires more investigation.

• Striatal microcircuit control of neurotransmitter co-release from dopamine terminals: Local inputs, like CINs are capable of stimulating GABA release (Nelson et al., 2014) and glutamate release (Gras et al., 2002) at dopamine terminals, which influences output from the striatum. It will be important to assess the functions of this non-canonical release mechanism, and how it might differ across striatal sub-regions.

• Neurotransmitter co-release from local striatal circuits: Interneurons in the striatum also co-release multiple transmitters (Burke et al., 2017), and little is known about the functional and plasticity-related consequences of these signals. For example, CINs release GABA (Saunders et al., 2015a) and glutamate (Fremeau et al., 2002; Gras et al., 2002, 2008; Higley et al., 2011) in addition to acetylcholine.

Conclusions:

In this review, we have discussed anatomical, functional, and mechanistic heterogeneity of dopamine neurons in the VTA and SNc, and their target regions in the striatum. This work highlights how a dopamine neuron’s (and, indeed, most neurons’) function stems from a complex interaction of anatomical location, intrinsic properties, molecular make-up, connectivity, activity state, and moment-to-moment behavioral conditions. Independent, parallel, and serial regulatory mechanisms shape dopamine activity across these dimensions, creating a substrate for wide behavioral influence. A more complete view of the multiple levels of heterogeneity within midbrain-striatal circuits will hopefully better explain dopamine’s diverse contributions to behavior, which is essential for treating diseases that rely on these systems.

Significance Statement.

Dopamine neurons projecting to the striatum are critical for reward-related behaviors, but defining their diverse functions has been a challenge, due to extensive anatomical and physiological heterogeneity across dopamine neuron subpopulations. In this review, we highlight emerging research characterizing dopamine circuit heterogeneity, to better understand how this system controls learning, motivation, and dynamic decision making.