Heterogeneous dopamine signals support distinct features of motivated actions: implications for learning and addiction

Abstract

Despite decades of research, investigations into effective neural and pharmacological therapies for many drugs of abuse, such as cocaine, have produced no FDA-approved approaches. This difficulty derives from the complexity of substance use disorders, which encompass a variety of behavioral, psychological, and neural circuit-based changes that occur as a result of repeated experience with the drug. Dopamine signaling has been demonstrated to play a key role in several aspects of drug abuse—from mediating its reinforcing properties and drug-seeking to triggering relapse—while also mediating a number of important aspects of normal (nondrug related) motivated behaviors and actions. Real-time recording methods such as in vivo voltammetry, electrophysiology, and calcium imaging demonstrate that the signaling properties of dopamine for motivationally relevant stimuli are highly dynamic and spatiotemporally circumscribed within afferent target regions. In this review, we identify the origins and functional consequences of heterogeneous dopamine release in the limbic system, and how these properties are persistently altered in the drug-experienced brain. We propose that these spatiotemporally parallel dopaminergic signals are simultaneously available to the animal, but that these circuits are impaired following prolonged drug experience by disrupting the location and content of dopamine signals in afferent target regions. These findings are discussed in the context of relapse and pathways to discovering new treatments for addiction disorders.

Substance abuse disorders are notoriously widespread in the human population, and inflict an astonishing toll on individuals, their social networks, and society as a whole. At the time of this writing, drug addiction has become a national emergency. So great is the toll on human lives that the Centers for Disease Control and Prevention reports that death tolls from drug overdoses in a single year now surpass the amount of American combat deaths since the Vietnam war combined. While much of the focus on this epidemic has centered on the stunning rise of opiate abuse, there has been a paralleled rise in abuse of other drugs like cocaine, where use rates have increased 61% in just 2 yr, along with overdose deaths that are among the highest since the late 1990s (Hughes et al. 2016).

However, there has been surprisingly little success in developing pharmacotherapies to treat these disorders. To date, there have been no FDA-approved treatments for cocaine abuse disorder. The reasons for this have to do with the complexity of drug abuse disorders in general, involving both powerful behavioral and neurobiological components. Behavioral elements involve specific drug-associated paraphernalia, social associations, behavioral rituals associated with drug taking, and the motivational cycles of bingeing and withdrawal, while the neurobiological components involve the interaction of a variety of limbic regions such as the ventral tegmental area (VTA), its afferent target regions (particularly the nucleus accumbens [NAc]), and various other inputs that modulate VTA function. However, the complexity of signaling regulation within these systems (e.g., interactions between dopamine, glutamate, acetylcholine, adenosine, and microglia) along with dynamic shifts in these regulatory properties as a result of chronic drug exposure highlight the complexity of substance abuse disorders and therefore why straightforward treatments are unlikely to succeed.

In this review, we propose that the signaling properties of dopamine are highly heterogeneous, and therefore, dopamine will have notably different functions within defined circuits. As such, the function of dopamine signals cannot be understood completely by general principles, but instead must be defined within context (behavior) and in place (neural circuit). We present three organizing principles that support this position. First, the organization of dopaminergic nuclei is highly heterogeneous across regions, terminal field targets, and transmission properties. Second, phasic dopamine release patterns widely vary by afferent target region, even during performance of identical tasks. Third, presynaptic inputs can significantly alter phasic release properties, allowing for local shaping of release within microcircuit regions. Finally, we present evidence that acute and repeated experience with drugs of abuse modifies the dopamine signaling properties of these pathways, thereby changing important features of learning and drug-associated behaviors. We believe that understanding the heterogeneity, complexity, and dynamic nature of these signaling properties will allow future studies to develop effective means of treating complex behavioral disorders like substance abuse.

Anatomy of the mesolimbic dopamine system

Dopaminergic output regions

Within the context of learning, motivation and drug taking behaviors, the majority of work has focused on signaling of the mesolimbic dopamine system. The VTA is intrinsically heterogeneous based on cell type, connectivity, and organization. Anatomically, this area is comprised of parabrachial pigmented nucleus, caudal linear nucleus, rostral linear nucleus, interfasicular nucleus, and paranigral nucleus (Swanson 1982; Fu et al. 2012). Each of these regions is thought to maintain topographical organizations that are biased toward specific forebrain targets. For example, topographic projections from the VTA to the striatum follow relatively consistent mapping between VTA locations and limbic targets (Ikemoto 2007), with cell bodies that target medial aspects of the striatum such as the olfactory tubercle and medial shell of the NAc located more medial and posterior in the VTA (rostral linear, interfasicular, paranigral, and caudal linear nuclei), while those that target the core and dorsal striatum are far more lateral and anterior (parabrachial pigmented area, substantia nigra). Indeed, these pathways show a surprisingly low degree of collateralization, with one recent report indicating only 0.4% of VTA dopamine cells projected to both medial and lateral accumbens locations (Yang et al. 2018). However, it is likely that axonal collateralization and other forms of cross-regional networks are present in dopamine pathways (Beier et al. 2015), which could act to coordinate phasic dopaminergic signaling across related regions. Beyond connectivity, these pathways may have additional functional considerations. For example, a subset of atypical fast-firing DA neurons that have low expression of the dopamine transporter may be able to use a higher frequency band to carry different signals to limbic targets than the conventional VTA dopamine neurons (Lammel et al. 2008).

Additionally, it is now recognized that dopaminergic populations do not exclusively release dopamine, but a subset can corelease dopamine along with traditional neurotransmitters like glutamate and GABA. For example, identified dopamine neurons can produce GABA and copackage these molecules along with dopamine (Tritsch et al. 2012, 2014; Kim et al. 2015). These dopamine-GABA coreleased mechanisms likely modulate a set of regions including a nigrostriatal pathway to the dorsal striatum (Tritsch et al. 2012, 2014) and a VTA pathway to the lateral habenula (LHb) which may ultimately act to regulate the VTA's own activity (Stamatakis et al. 2013; Lammel et al. 2015; Dolzani et al. 2016). As with GABA, glutamatergic cells in the VTA can be found that release exclusively glutamate, or glutamate that is coreleased with dopamine (Hnasko et al. 2010; Morales and Root 2014; Root et al. 2016). As with the general projection architecture of the VTA, most of these dopamine-glutamate positive cells have projections to limbic regions including the NAc and LHb (Stuber et al. 2010; Root et al. 2014a,b). In important ways, these cells may be further differentiated by other aspects of receptor expression, but likely provide important differences in signaling than dopamine signals alone may contribute, and will be considered in more detail below.

Differences in dopaminergic signaling: circuit-level and behavioral considerations

Work in my laboratory and many others have begun to characterize the role that phasic dopamine signaling contributes to motivated learning and action. The most salient of the theories that led many researchers (including myself) into the field of dopamine was the observation that dopamine neural activity could provide a neural correlate for learning, particularly as it related to classical conditioning (Schultz et al. 1997; Waelti et al. 2001). In these models (and in contrast to earlier hedonic-based theories of dopamine Wise et al. 1978), dopamine neurons increased or decreased activity at the time of rewards not based only on the value of the reward itself, but rather on the degree to which the outcome was unexpected. In this scenario, dopamine signals showed increased activity to encode positive prediction errors (better than expected) and decreased activity below baseline for negative prediction errors (worse than expected). This signaling pattern corresponded with a long history of learning theory principles of associativity associated with the Rescorla and Wagner “delta rule” model (Rescorla and Wagner 1972). Consistent with this, dopamine also signaled information about the value of the anticipated outcome at the time of predictive cues, and that signaling would thus “shift” from unexpected rewards (early in learning) to cues (late in learning), consistent with model predictions (Pan et al. 2005). These reward prediction errors (RPEs) were thus thought to provide a teaching signal to the brain, and indicated an important role for dopamine in a variety of learning processes based on value and prediction.

From this profound and elegant insight, researchers over the next two decades have been trying to understand how (or whether) these brief phasic signals come to control behavioral output of motivated animals. In most of these cases, the central assumption has been that dopamine signals act as a largely unitary signal that is broadly available to the limbic system in brief bursts of phasic release events, consistent with a general neuromodulatory function of monoamine signaling and also with the general architecture of the mesolimbic dopamine system. From this, it was argued that these complex patterns of release provide information about what signals the dopamine system was transmitting to the limbic system as a whole. Therefore, using real-time methods to record dopamine activity in behaving animals should provide the necessary tools to understand the important function(s) of the dopamine system.

Methods to understand real-time dopamine signaling have traditionally used electrophysiology of putative dopamine cell bodies in the midbrain based on properties of waveforms (Hollerman and Schultz 1998; but see Margolis et al. 2010; Schultz and Dickinson 2000) or pharmacological verification (Roesch et al. 2007). Recent genetic targeting of dopamine-specific populations using cre transgenic lines (TH::Cre or DAT::Cre) now permit more selective identification for both “phototagging” (light-responsive neural responses in opsin-expressing cells (Lima et al. 2009; Cohen et al. 2012; Kravitz et al. 2013; Stauffer et al. 2016)) or in vivo calcium imaging (Gunaydin et al. 2014). In addition, dopamine can be detected using electrochemical methods, and thus in vivo fast scan cyclic voltammetry (FSCV) can be used in terminal regions to assess release patterns (Phillips et al. 2003a; Heien et al. 2004; Roitman et al. 2004). FSCV has the advantage of assessing the real-time release kinetics within a specific region rather than inferring release from electrophysiological or calcium imaging methods which may not always align (London et al. 2018). This mismatch may arise from several factors, including terminal modulation factors such as dopamine transporter (DAT) availability (Jones et al. 1996), activity glutamatergic afferents from other regions (Jones et al. 2010), and/or cholinergic interneurons (Cachope et al. 2012), which could alter release properties.

Several reports have now confirmed that phasic dopamine signaling corresponds to important features of the RPE hypothesis. In general, these studies with Pavlovian conditioning designs demonstrate that unexpected rewards trigger increases in dopamine activity (Day et al. 2007; Nasrallah et al. 2011; Lak et al. 2014; Stauffer et al. 2014; Saddoris et al. 2017), while with additional training, dopamine signals “shift” to become time-locked to reward-predictive cues (Day et al. 2007; Hart et al. 2014). Furthermore, in more sophisticated choice-based tasks, dopamine signaling during predictive cues signaled subjective factors about predicted valuation of outcomes, including individual preference and utility (Roesch et al. 2007; Day et al. 2010; Gan et al. 2010; Sugam et al. 2010; Sackett et al. 2017). Specifically, these findings and others show that dopamine release during cues scales with the subjective preference of the anticipated outcome such that preferred options elicit greater dopamine release events, and that optogenetic manipulation of these dopamine signals is sufficient to alter subsequent choice and associative behaviors (Saddoris et al. 2015b; Chang et al. 2016; Schelp et al. 2017; Sharpe et al. 2017).

However, several challenges to an RPE-based interpretation of dopamine function have emerged based on several factors that include behavioral considerations and the dynamics of dopamine release within different target regions. First, there are limited data that dopamine signaling during the reward receipt necessarily relates to the development of cue-predictive properties, as is anticipated by learning theory and behavioral work. For example, we have now shown that phasic dopamine release in the NAc core fails to track the value of different reward options (Saddoris et al. 2017), mirroring similar effects obtained in primate VTA neurons (Stauffer et al. 2014). This effect is observed despite the established observation that dopamine signals in this same region encode differences in predicted outcome values during cue presentations.

Second, RPE methods fail to account for the many observations that are linked to motivation and individual differences in conditioned approach. For example, state-related changes in motivation (e.g., hunger versus satiety) can significantly alter phasic dopamine release rates during cues predictive of reinforcing stimuli (Saddoris et al. 2015a; Aitken et al. 2016; Papageorgiou et al. 2016). Furthermore, incentive salience theories (Berridge and Robinson 1998; Berridge 2012; Saunders and Robinson 2012) further posit that dopamine is more linked to motivational significance rather than reward prediction alone. For example, phasic release associated with conditioned cue approach (sign tracking) is greatest at the cue during approach, but limited during rewards. In contrast, animals who approach a foodcup during reward-related cues show more blunted dopamine responses to the cue but slightly elevated responses to reward receipt (relative to sign trackers) (Flagel et al. 2011). It is argued that this disparity can be attributed to dopamine increasing the incentive value of the cue in sign trackers (thus eliciting approach), whereas goal trackers are able to learn perfectly well without that dopaminergic signal. Consistent with this, pharmacological manipulations with dopamine antagonists block the expression of sign tracking but fail to alter goal tracking behaviors (Flagel et al. 2011; Saunders and Robinson 2012).

Finally, dopamine signaling is not consistent across brain regions even during the same behavioral task. In behavioral tests, the same stimuli or actions can produce highly divergent dopamine signals across subregions of the striatum. In Pavlovian tasks, for example, both NAc core and shell display phasic release events to cue-predictive stimuli. However, in the core, dopamine signaling is rapid with a large peak at cue presentation and a minimal response to predicted rewards. In contrast, NAc shell dopamine tends to remain elevated for the duration of the cue and also at the reward receipt, even when well predicted (Cacciapaglia et al. 2012; Saddoris et al. 2016). We have demonstrated that regional differences in DAT availability—while present (Yorgason et al. 2011; Ferris et al. 2013; Saddoris et al. 2015a; Saddoris 2016)—are insufficient to account for these signaling differences. A single burst of electrically elicited dopamine closely tracks stimulus-elicited dopamine in the core, while the same stimulation only captures a fraction of stimulus-related dopamine in the shell (Saddoris et al. 2015a).

These observations suggest that dopamine in the NAc shell encode information that is both incompatible with RPE theories and not congruent with dopamine release patterns in a directly adjacent region. Indeed, we have now shown that in a chained operant task where presses on one lever (seeking lever) grant access to a second (taking lever) that can be pressed for food, the differences between core and shell dopamine release become even more profound (Saddoris et al. 2015a). Here, dopamine in the NAc core tracked only the initial presentation of the seeking lever, but produced only modest release at the taking lever and reward receipt. In contrast, dopamine release in the NAc shell tracked both seeking and taking levers as well as the reward delivery at similar release concentrations. Furthermore, under extinction conditions with this task, dopamine release in the core showed changes at the cue that gradually decreased over trials and displayed negative prediction errors at the time of expected reward delivery. In contrast, shell dopamine release tended to decrease at times with changes in animal motivation, but did not display prediction errors at reward omission. We have argued that core dopamine release was quite consistent with RPEs, but that shell was in fact more congruent with Incentive Salience models (Tindell et al. 2006, 2009; Smith et al. 2011; Saddoris et al. 2015a).

Other factors beyond conditioning have also been shown to produce reliable differences in dopamine signaling between striatal targets. For example, phasic release in the NAc shell appears to encode important information about reward magnitude, both during unsignaled reward consumption (Roitman et al. 2008; Saddoris et al. 2017) and cues predictive of valued rewards (Beyene et al. 2010; Loriaux et al. 2011; Sackett et al. 2017). In contrast, shell dopamine release was much less sensitive to costs (such as risk or delay) that could devalue the expected value of expected outcomes (Day et al. 2010; Sugam et al. 2012). NAc core dopamine showed a different set of goal-directed signaling. In several settings, dopamine signals during reward-predictive cues scaled with the preference of the expected outcome, particularly those that were devalued by costs (Day et al. 2010; Sugam et al. 2012; Ko and Wanat 2016; Schelp et al. 2017). For example, as the delay to obtain a large reward increased across blocks, dopamine levels significantly decreased for the option (Saddoris et al. 2015b). Furthermore, stimulation of dopamine afferents selectively within the NAc core was able to alter choice behavior when the options differed only by a delay cost (immediate versus delay, same reward outcome), but not for magnitude-based choices (large versus small reward, same delay) (Saddoris et al. 2015b).

Early reports suggested a specific role for dopamine in reward-related learning (Mirenowicz and Schultz 1996), though more recent studies indicate a role for dopamine in aversive learning as well (Oleson et al. 2012b; Danjo et al. 2014). For example, dopamine appears to play a causal role in negative reinforcement, where optogenetically stimulated dopamine afferents in the NAc core facilitate active avoidance and fear extinction (Wenzel et al. 2018b). While less is known about region-specific aspects of dopamine signaling in fear learning, heterogeneity in signaling has been seen in this system as well. In the primate, putative dopaminergic cells often showed patterned firing to both cues predictive of rewards and aversive outcomes, while other midbrain neurons displayed activity that was biased toward either reward or aversion (Matsumoto and Hikosaka 2009). Furthermore, FSCV recordings in the NAc demonstrate a region-specific difference in dopamine release; while cues predicting unavoidable shocks elicit transient increases in dopamine in the shell, the same cue induces sustained pauses in release in the core (Badrinarayan et al. 2012). Thus, dopamine is providing multiple parallel signals in the NAc that can be used by the animal to guide behavior based on task and motivational demands. In general, this suggests that any given proposed function of dopamine must be understood within the region it is signaling, and that the dopamine system may at any time provide quite different and even competing signals.

In addition, virtually nothing is known about the functional real-time signaling of dopamine in behaving animals in extra-striatal targets such as the PFC or amygdala, and whether these signals differ from phasic dopamine patterns seen in the NAc. These difficulties stem not from a lack of interest—indeed, important work from several labs using pharmacological approaches have shown critical importance of dopamine signals in places like the PFC in choice behavior and other goal-directed behaviors (Floresco et al. 2008; St Onge et al. 2011; Cocker et al. 2014, 2016; Jenni et al. 2017)—but rather from low levels of signal and difficulty in isolating dopamine from other catecholamines like norepinephrine. However, future work with detailed FSCV analysis in awake and behaving subjects may provide important insight into these nonstriatal targets, and further highlight the heterogeneity of the dopamine response in behavior.

Drug taking and the dopamine signal: acute and persistent changes in neural circuits

Early work with this system showed that drugs of abuse ultimately act by increasing dopamine levels in the NAc (Di Chiara and Imperato 1988). In particular, psychostimulants provide their reinforcing mechanisms by increasing the availability of dopamine in the synapse by invigorating release (amphetamine) or preventing reuptake of dopamine overflow (cocaine); in contrast, opiates work in general by removing inhibitory tone on dopamine cells in the midbrain, effectively “releasing the break” on the system (Johnson and North 1992). Indeed, VTA dopamine signals are essential to mediating the reinforcing properties of drugs as seen by their involvement in mediating self-administration, conditioned place preference, behavioral sensitization, and reinstatement (Gong et al. 1996, 1997; Neisewander et al. 1996; Duvauchelle et al. 2000; Ikemoto 2002; Xi et al. 2004).

There is strong evidence that phasic dopamine signaling mediating drug-related behaviors is similarly heterogeneous as it was in normal (nondrug) related behaviors. For example, the rate of psychostimulant self-administration into the anterior aspect of the striatum differs strongly based on subregion. Though all regions receive dopaminergic VTA afferents (Ferreira et al. 2008), the highest rates of self-administration are found in more medial and dorsal locations that include the medial NAc shell and olfactory tubercle, but far less in the lateral NAc shell or NAc core (Ikemoto 2003, 2007).

Using FSCV, these structural differences can be seen in the phasic release patterns. In general, cocaine infusions induce a significant increase in phasic dopamine release (Heien et al. 2005; Stuber et al. 2005). This signaling can be seen in increases in the number of dopamine transient events and also the amount of release during each burst (Stuber et al. 2005). As above, there is considerable evidence for differences in phasic response patterns between striatal target regions. For example, dopamine transients in the NAc shell show a substantially greater increase in release probability and volume of release than in the NAc core following intravenous infusions of cocaine (Aragona et al. 2008). Consistent with dopamine's role in learning, phasic release is also elicited by drug-associated cues, as well as with drug-seeking behaviors, though this is again biased toward signaling within the NAc shell compared to the core (Phillips et al. 2003b; Aragona et al. 2009; Wheeler et al. 2011). Recent data also provide support for changes in signaling properties along the anterior–posterior aspect of the ventral striatum (Park et al. 2012) that may relate to different motivational properties of drugs of abuse. For example, optical stimulation of dopamine afferents in the rostral NAc shell can oppose negative hedonic shifts in cue-elicited drug craving, while the same stimulation in the caudal shell can amplify the negative components (Hurley et al. 2017).

Psychostimulants like cocaine create their reinforcing properties by generating persistently elevated levels of dopamine in the synaptic region of motivationally relevant regions. However, repeated experience with these abnormally high levels of sustained dopamine levels has been shown to induce plastic changes in signaling properties. For example, animals that are trained to self-administer cocaine initially display phasic release events in the NAc. However, following weeks of this self-administration behavior, phasic dopamine release is now seen in the dorsal striatum (Willuhn et al. 2012, 2014). This neuroplastic shift in signaling indicates that phasic dopamine may represent information about events that are at a minimum engaging other brain regions, and in some cases providing information that is inappropriate for newly engaged regions (Takahashi et al. 2007; Burton et al. 2017). These potentially pathological representations in novel circuits could be less susceptible to modification by new learning.

To identify how these persistent plastic changes affect behavior, we have recorded dopamine in rats with a history of cocaine self-administration. Subjects typically had relatively brief but regular access to cocaine (2 wk, 2 h access per day), followed by a period of enforced abstinence. These animals show striking changes in behavior when learning food-related (nondrug) tasks after this period of cocaine exposure. For example, rats show altered motivation-related impairments in performance in a Pavlovian-to-instrumental transfer (PIT) task (Saddoris et al. 2011; LeBlanc et al. 2013; Ostlund et al. 2014), second-order Pavlovian conditioning (Saddoris and Carelli 2014; Saddoris et al. 2016), and even discriminating between reward magnitudes (Saddoris et al. 2017). In addition, these animals showed an abnormal degree of conditioned cue approach (Saddoris et al. 2016), consistent with observations in animals following abstinence from other drugs of abuse like alcohol (McClory and Spear 2014; Spoelder et al. 2015) and psychostimulants (Yager and Robinson 2013; Robinson et al. 2015).

Phasic dopamine release in these cocaine-experienced subjects is significantly impaired in these subjects across a number of dimensions, including striatal target region. For example, dopamine release during a Pavlovian discrimination task showed region-specific changes in release patterns in cocaine-experienced animals (Saddoris et al. 2016). After conditioning, dopamine release in these animals was profoundly altered. In the core, cue-related dopamine signaling was biased toward the reward instead of the predictive cue, a pattern more often seen much earlier in learning in drug-naïve animals (Day et al. 2007). In contrast, dopamine release in the shell of cocaine-experienced subjects was virtually absent. Despite the fact that dopamine release could readily be evoked by electrical stimulation of dopamine afferents in these animals (Saddoris 2016), few transient release events were observed during the task and were not aligned to behavioral events. Similar observations of impaired associative dopamine release have been found following repeated access to cocaine and other drugs of abuse (Nasrallah et al. 2011; Spoelder et al. 2015). Surprisingly, these impairments in drug-related dopamine signaling can disconnect the well-characterized relationship between dopamine and conditioned approach. In drug-naïve animals, increases in dopamine positively correlate with the degree of sign tracking responses, while cocaine-experienced rats display robustly elevated sign tracking responses despite diminished dopamine release in the NAc (Saddoris et al. 2016). These observations appear to be conflicting, but instead may suggest that chronic exposure to drugs is altering not just the sensitivity of target neurons to dopamine input, but also the location, timing, and release volume of the dopamine input itself.

We suggest that these shifts are likely predictable, with striatal phasic release being biased toward regions that are more dorsal and lateral than in drug-naïve populations. In a recent experiment, electrically stimulated release of dopamine in awake and behaving rats produced evoked transients in the NAc core that appeared more “shell-like” after cocaine experience. Specifically, release and reuptake kinetics of stimulated release were far more similar to those typically seen in the shell of drug-naïve controls than in the core (Saddoris 2016). Related findings from other labs using in vivo electrically stimulated dopamine demonstrate similar heterogeneity in dopamine release kinetics based on drug type, duration of exposure and period of enforced abstinence (Addy et al. 2010; Calipari et al. 2014, 2015; Cameron et al. 2016; Siciliano et al. 2016). Behaviorally we see this same effect. In a simple experiment in our laboratory (Saddoris et al. 2017), animals received unsignaled rewards of different sizes (either one or two pellets). In normal controls, dopamine release patterns in the medial NAc shell (but not the core) reflected differences in reward value, with greater concentrations of dopamine during reuptake being seen for larger rewards. However, in cocaine-experienced animals, this reward magnitude dopamine encoding now unexpectedly appeared in the core while very little dopamine was elicited by the same stimuli in the shell. Despite this magnitude-related dopamine encoding in the core of cocaine animals, these subjects were unable to appropriately select between levers that produced either one or two pellets, suggesting that the preserved (but regionally inappropriate) dopamine signaling of reward magnitude was insufficient to guide motivated behavior.

Collectively, these findings further demonstrate the necessity of dopamine signals within specific circuits to produce adaptive and appropriate behaviors. The persistence of discriminative dopamine signals in the cocaine-experienced brain was of little use to the animal because they occurred in an abnormal target region. These observations are more consistent with the importance of dopamine's functional roles within specific circuits, but less so with a more general signal available to limbic targets as a whole. We thus expect that plasticity gated in target regions by this inappropriate signaling contributes to the persistence of drug-related associations.

Dopamine terminal mechanisms: release kinetics and synaptic modulation

A final note on the importance of heterogeneity of dopamine signaling must assess the effects of local modulation on dopamine signals. Work has shown that midbrain dopamine neurons (identified by waveform and/or genetic targeting) consistently display a strong bias toward RPE-type encoding in a number of different behavioral contexts (Cohen et al. 2012; Eshel et al. 2016; Stauffer et al. 2016). However, in the above discussion, we have highlighted the distinct release properties of dopamine within specific target regions, and how disruption of these patterns induced by chronic drug experience can cause profound changes in motivated behavior. This lack of correspondence is surprising, but we expect that differences could be attributable to any combination of several factors. First, recordings taken from midbrain regions could oversample from populations of dopaminergic cells that have biases in their projection fields in RPE-related terminal locations. For example, we and others have shown that NAc core dopamine release often encodes information (predicted outcome values, RPEs); perhaps these populations are more likely to be identified and characterized than in shell targeting (or other limbic) regions. Given the heterogeneity of dopaminergic pathways to the forebrain limbic system (Ikemoto 2007; Lammel et al. 2011, 2014), these pathways and their relationship to signaling properties have not yet been described.

A second factor could be local modulation of release by presynaptic mechanisms within targeted regions. In this scenario, dopamine cell bodies could signal a relatively uniform output signal that is related to RPE information, but local factors could augment this signal to be more appropriately used within the afferent region. Previous work has already suggested that even within a given dopamine target region (such as NAc core), there is a lack of uniformity of signaling. For example, some NAc regions contain “hot spots” of stimulus-related release, while nearby locations produce very little stimulus-related release (Wightman et al. 2007). However, electrical stimulation of VTA afferents produces dopamine in both locations, suggesting that targeted heterogeneous regions have some functional purpose in the behaving animal. Indeed, neural populations in the NAc without phasic dopamine release either do not develop (Owesson-White et al. 2009) or rapidly lose phasic excitatory responses to external stimuli (Cacciapaglia et al. 2011). It has recently been suggested that these intrinsic differences in release properties between hot/cold spots could produce biased sampling approaches; for example, researchers with acutely placed electrodes could seek out and prioritize recordings in regions with the greatest level of release and naturally occurring transients (Rodeberg et al. 2017). However, these “hot spot” regions may not be representative of the region as a whole, and could indicate important and currently uncharacterized differences in local modulation of signaling.

A large number of local factors within dopamine terminal fields have now been demonstrated to modulate the release, and a full discussion of these factors is beyond the scope of this review. These factors include potential modulation by endocannabinoids (Cheer et al. 2004; Oleson et al. 2012a; Wenzel and Cheer 2018a), glutamatergic inputs from limbic regions (Jones et al. 2010), neuroinflammation (Schindler et al. 2017; Brown et al. 2018), adenosine (Ross and Venton 2015), and acetylcholine (Threlfell and Cragg 2011; Cachope et al. 2012). In this latter situation, cholinergic interneurons (ChIs), comprise <5% of the total population of striatal neurons, and exert presynaptic control over neurotransmitter release from DA neurons. ChIs can enhance DA release through the activation of ß2-subunit-containing nicotinic acetylcholine receptors on DA axons (Cachope et al. 2012; Threlfell et al. 2012). While ChIs are tonically active, they can exhibit synchronized firing following the receipt of excitatory input, primarily of cortical and thalamic origin (Kosillo et al. 2016). ChI synchrony has notable consequences for DA release kinetics, as local pharmacological manipulation of cholinergic receptors induced significant suppression (scopolamine) or facilitation (mecamylamine) of DA release to motivationally salient cues and unexpected rewards (Collins et al. 2016). Crucially, this control of DA activity can be driven independently of midbrain firing activity. It is therefore insensitive to the frequency of phasic DA signals that might occur in response to reward-relevant stimuli or that might be pathologically altered following chronic drug administration.

While ChIs are present throughout the striatum, their functional interactions with DA activity are not necessarily comparable across subregions. Due to the cessation of phasic DA activity, ChIs briefly pause firing (Zhang et al. 2018). However, in slice preparation, ChIs in the dorsal striatum exhibit a hyperpolarization, followed by an elongated pause in activity, while ChIs in the NAc core are at most slowed by phasic bursts of DA firing. In contrast, ChIs in the NAc shell respond to DA activity with a burst of action potentials, followed by a pause and rebound activity (Chuhma et al. 2014). The ChI pauses may be protective against loss of nAChR sensitivity and therefore degradation of cholinergic control of terminal DA (Shin et al. 2017). As such, cholinergic modulation of shell signaling may yield greater, longer-lasting responses to salient events that induce phasic bursts of VTA neurons. Moreover, as these pauses represent an aspect of ChI activity that is sensitive to midbrain excitatory activity, they too can evolve over the course of conditioning. In doing so, ChI can improve the temporal specificity of signals that may be critical for learning.

From phasic dopamine to striatal encoding; or, what does dopamine do?

The great preponderance of the work investigating phasic dopamine signaling in behaving animals has focused on the kinetics of dopamine signal itself rather than the consequences of this signaling on receiving neurons. This is largely a necessity of the methods that have been optimized to detect properties of the dopaminergic system (e.g., FSCV, or optogenetically identifying the activity of dopamine neurons). One important aspect of dopamine signaling is that it acts to alter excitability of targeted neurons, consistent with its typically understood function as a neuromodulator rather than a neurotransmitter. Thus, an important consideration of dopamine signaling should be considered in the context of how these signals mediate changes in encoding in downstream target regions. Early reports using a simultaneous FSCV/electrophysiology recording technique in the same location revealed that phasic neural encoding was highly correlated with the presence of phasic dopamine release (Cheer et al. 2005, 2007; Owesson-White et al. 2009). However, several reports have now suggested that the relationship between dopamine signaling and neural encoding is often quite complex. For example, in a food self-administration task, NAc core neurons showed either phasic excitations or inhibitions related to task events (e.g., press, food reward). However, when phasic burst firing of dopamine cells is inhibited via intra-VTA infusions of APV, neural excitations in the NAc core disappear (i.e., become nonresponsive to task events), but phasic inhibitions appear to be unaffected (Cacciapaglia et al. 2011). Thus, the heterogeneous response of target neurons in response to dopamine release (perhaps corresponding to D1R- and D2R-expressing populations in the NAc), suggest that even a uniform dopamine signal can propagate highly specific circuit-level outcomes within the limbic system.

Few studies have investigated differences in the relationship between core and shell dopamine signaling and associated neural encoding. Work from our group used an identical decision-making task, using either FSCV (Sugam et al. 2012) or electrophysiology (Sugam et al. 2014) to record accumbal dopamine inputs and NAc neural outputs, respectively. Rats in this task were trained so that one response produced a small but certain reward, while the other “risky” response produced either a larger reward on half the trials or nothing on the other half. After a block of “forced choice” training trials, rats were then allowed to select their preferred responses in a free choice block. Using FSCV, Sugam et al. (2012) found that most animals develop a stable preference for either the safe or risky option. For those subjects, cues associated with the animals’ preferred choice elicited greater dopamine release than the nonpreferred option in the core, but both options produced similar release in the shell. However, dopamine failed to show any difference on free choice trials, suggesting that dopamine was signaling the overall preferred choice rather than the actually chosen action (Roesch et al. 2009). However, electrophysiological recordings in the NAc core and shell revealed a sharply different pattern of core and shell responses (Sugam et al. 2014). Unlike dopamine signals, phasic neural encoding in both core and shell failed to differentiate between preferred and nonpreferred cues on forced-choice trials, but instead showed a high degree of discriminative encoding based on the preferred action on free choice trials, an effect which was particularly salient in the core. This lack of congruence demonstrates that the information contained in the dopamine signal is not passively translated into action by target regions, but rather provides one of many important signals important for mediating motivated action. Understanding the extent (and indeed limits) of the dopamine signal on behavior will thus demand the context of location, convergent signals, and target cell type.

Conclusion

Drug use and abuse is known to involve the signaling of dopamine. These signals are involved in a variety of mechanisms that affect these drug-related behaviors, including the reinforcing properties of the drug, initiating the drug taking actions, encoding information about drug-related cues, modulating affect during craving, and triggering features of relapse. Despite the commonality of the dopamine molecule, the signals important for supporting these processes have distinct temporal, spatial and behavioral contexts in which they occur. Indeed, dopamine signaling arises from a set of highly heterogeneous populations and is released in patterns that differ based on terminal field locations. These patterns are then likely subject to highly specific terminal modulatory processes that can modify release further based on the availability and input structure of local microcircuits. Finally, these architectures are highly dynamic and can change significantly as a result of chronic experience with drugs of abuse.

No doubt dopamine gets a significant amount of attention in learning, motivation and drug abuse for both functional and historical reasons. However, the arising picture of dopamine's function in these processes also include important interactions with other neurotransmitter systems like norepinephrine (McElligott et al. 2013), glutamate, and peptides (Lemos et al. 2012; Wanat et al. 2013; Al-Hasani et al. 2015; Twining et al. 2015), among others. These factors point to the complexity of potential treatments for addiction, as the interaction and dynamic updating of components of dopamine-related processes present neurotherapies as a moving target. However, more precise approaches that can reveal the function of dopamine in context and in place should provide better insights to provide more effective treatment options moving forward.