Skip to main content
Springer logoLink to Springer
. 2017 Oct 26;125(3):565–574. doi: 10.1007/s00702-017-1793-9

Reward and value coding by dopamine neurons in non-human primates

Aydin Alikaya 1, Mackenzie Rack-Wildner 1, William R Stauffer 1,2,
PMCID: PMC5847197  PMID: 29076112

Abstract

Rewards are fundamental to everyday life. They confer pleasure, support learning, and mediate decisions. Dopamine-releasing neurons in the midbrain are critical for reward processing. These neurons receive input from more than 30 brain areas and send widespread projections to the basal ganglia and frontal cortex. Their phasic responses are tuned to rewards. Specifically, dopamine signals code reward prediction error, the difference between received and predicted rewards. Decades of research in awake, behaving non-human primates (NHP), have shown the importance of these neural signals for learning and decision making. In this review, we will provide an overview of the bedrock findings that support the reward prediction error hypothesis and examine evidence that this signal plays a role in learning and decision making. In addition, we will highlight some of the conceptual challenges in dopamine neurophysiology and identify future areas of research to address these challenges. Keeping with the theme of this special issue, we will focus on the role of NHP studies in understanding dopamine neurophysiology and make the argument that primate models are essential to this line of research.

Keywords: Dopamine, Learning, Decision making, Value, Monkey, NHP, Optogenetics, Reward prediction error

Introduction

Rewards are a central feature of everyday life that promote learning, incentivize value-based decisions, and provide a currency for social exchanges. Dopamine neurons respond to rewarding events. Specifically, phasic dopamine activity reflects the difference between received and predicted rewards, i.e., reward prediction errors. Systems neuroscience studies in non-human primates (NHP) have been critical to understanding this coding scheme, as well as investigating the behavioral implications of reward prediction error coding.

Right now, we stand at the threshold of a new neuroscientific age. This age will be defined by big data and molecular control of neural information processing. From this vantage point, this review endeavors to provide a brief introduction to dopamine neurons, examine the critical findings revealed by NHP studies that have shaped our understanding of dopamine function, and evaluate what role this valuable species should have in future investigations. We will attempt to identify outstanding challenges to our current understanding of this critical brain system, propose areas for future research, and reinforce the need to preserve and technologically advance behavioral neurophysiology studies in NHP. Keeping with the theme of this special issue on NHP studies of basal ganglia function, this review does not attempt to be comprehensive and passes over many of the recent advances observed in rodent models. In addition, we do not discuss the role of dopamine cell loss in Parkinson’s disease, nor the valuable contribution of the monkey MPTP Parkinson’s disease model (Langston et al. 1983).

The majority of dopamine neuron cell bodies are located in the midbrain, in cell groups designated A8, A9, and A10 (Dahlstroem and Fuxe 1964). These groups broadly overlap with the retrorubral field (RRF), substantia nigra pars compacta (SNc), and ventral tegmental area (VTA), respectively. (Dahlstroem and Fuxe 1964; German and Manaye 1993) (Fig. 1a). Dopamine neurons express the enzyme Tyrosine Hydroxylase (TH) that converts the amino acid tyrosine to l-3,4-dihydroxyphenylalanine (l-DOPA). l-DOPA is converted to dopamine via aromatic l-amino acid decarboxylase (AAAD) and packaged into vesicles by the vesicular monoamine transporter (VMAT) (Lovenberg et al. 1962; Yelin and Schuldiner 1995).

Fig. 1.

Fig. 1

Dopamine anatomy and physiology. a Dopamine cell bodies in the VTA and SNc and dopamine terminals in the putamen and caudate tail are marked by brown DAB staining. Cd t caudate tail, Put putamen, SNc substantia nigra pars compacta, VTA ventral tegmental area. bd Dopamine responses code for reward prediction error. b Peri-stimulus time histogram (PSTH) of dopamine activity shows a strong response to unpredicted reward (indicated by the drop of juice). c PSTH of dopamine activity when a conditioned stimulus fully predicts reward. Dopamine neurons respond to the unpredictable onset of conditioned stimulus (CS1), but not to the fully predicted reward. d PSTH of dopamine activity when a high order conditioned stimulus (CS2) predicts the temporal onset of CS1 and delivery of reward. Dopamine neurons respond to unpredictable onset of CS2, but not to the fully predicted CS1 or reward.

bd Adapted from Schultz et al. (1993)

Dopamine neurons are few in number, less than 200,000 in Rhesus macaque monkeys (Stark and Pakkenberg 2004). Despite this, dopamine terminals are found throughout the striatum (Lynd-Balta and Haber 1994a, b) (Fig. 1a) and primate frontal cortex (Smiley et al. 1994; Smith et al. 2014; Williams and Goldman-Rakic 1993, 1998). Dopamine neuron action potentials (AP) are recognized according to their distinct electrophysiological profile that features broad AP waveforms as well as low and irregular baseline impulse rates (Guyenet and Aghajanian 1978). Classical identification of dopamine neurons was based on the tight correlation between these distinctive waveform characteristics and apomorphine sensitivity (Bunney et al. 1973; Guyenet and Aghajanian 1978; Schultz 1986; Schultz and Romo 1987). A partial survey of the literature that used apomorphine injections to identify dopamine neurons revealed that 77 of 85 putative dopamine neurons were suppressed by apomorphine, whereas none of 39 putative non-dopamine neurons were inhibited by apomorphine (Aebischer and Schultz 1984; Bunney et al. 1973; Guyenet and Aghajanian 1978; Schultz 1986; Schultz and Romo 1987; Studer and Schultz 1987). New alternative techniques, such as optogenetic photo-identification and juxtacellular labeling, promise similar reliability (Brischoux et al. 2009; Cohen et al. 2012; Eshel et al. 2015; Stauffer et al. 2016; Ungless and Grace 2012). Photo-identification is especially promising, as it uses dopamine neuron selective expression of optogenetic channels coupled with optical stimulation to unambiguously identify many dopamine neurons in each animal (Cohen et al. 2012; Eshel et al. 2015; Lammel et al. 2012; Stauffer et al. 2016), but see (Lammel et al. 2015). This strategy promises a less-subjective criterion for dopamine neuron identification, compared to identification via waveform characteristics, and will be critical to achieve an unbiased picture of dopamine neuron diversity.

Dopamine neuron activity is traditionally divided into irregular, slow (0.3–8 imp/s) tonic activity and phasic (burst firing) activity when the impulse rate can briefly reach 20–30 imp/s (Grace and Bunney 1983). Phasic bursts of dopamine neurons are the most efficient way to change the dopamine concentration in their target structures (Gonon 1988). These phasic bursts are thought to respond to rewards, though there is significant disagreement about the degree of functional heterogeneity (see section “Challenges and future research directions”).

Dopamine neurons code reward prediction error

Early evidence that dopamine neurons are involved in reward came from self-stimulation studies in maze-running rats (Olds and Milner 1954). However, behavioral neuroscience experiments in awake, head-fixed monkeys revealed the fundamental insight that phasic dopamine responses reflect reward prediction errors (Bayer and Glimcher 2005; Bayer et al. 2007; Bromberg-Martin et al. 2010; Enomoto et al. 2011; Fiorillo 2013; Fiorillo et al. 2003, 2008, 2013a, b; Hollerman and Schultz 1998; Kobayashi and Schultz 2008; Lak et al. 2014, 2016; Ljungberg et al. 1992; Matsumoto and Hikosaka 2009; Mirenowicz and Schultz 1994, 1996; Nakahara et al. 2004; Nomoto et al. 2010; Schultz et al. 1993, 1997; Stauffer et al. 2014; Tobler et al. 2005; Waelti et al. 2001). Reward prediction errors are defined in animal learning theory and machine learning as the differences between received and predicted rewards (Sutton and Barto 1998). A central tenet of animal learning theory is that temporal contiguity between a conditioned stimulus (CS) and reward (unconditioned stimulus, US) is not sufficient to drive learning. Rather, the reward must be unexpected; it must evoke a prediction error (Rescorla and Wagner 1972). Accordingly, reward prediction errors are teaching signals (Sutton and Barto 1998), and the reward prediction error nature of phasic dopamine responses strongly implicates these responses in learning.

Phasic dopamine responses are dependent on learning, and are time locked to unpredicted rewards and stimuli that elicit behavioral reactions (Mirenowicz and Schultz 1994, 1996; Romo and Schultz 1990; Schultz 1986; Schultz and Romo 1990). Early on in learning, when the associative strength between CS and reward is low, reward delivery strongly activates dopamine neurons (Ljungberg et al. 1992; Schultz et al. 1993) (Fig. 1b). Later, as the rewards become well predicted by the CS, dopamine neurons respond more strongly to the CS and less strongly to rewards (Ljungberg et al. 1992; Schultz et al. 1993) (Fig. 1c). With enough training, even higher order reward predictors (higher order CS) can activate dopamine neurons (Pan et al. 2005; Schultz et al. 1993) (Fig. 1d). Together, these studies provide overwhelming evidence that the phasic activity of dopamine neurons encodes reward prediction errors.

Dopamine reward prediction error responses are an ideal neural mechanism to mediate behavioral reinforcement learning, because they indicate both the occurrence of prediction errors and the proper direction to updating predictions. Rewards that are better than predicted activate dopamine neurons (positive prediction error responses), whereas rewards that are worse than predicted inhibit dopamine activity (negative prediction error responses). Modeling studies demonstrated that the prediction error term in popular reinforcement learning (RL) algorithms closely resembles the phasic dopamine signal (Montague et al. 1996; Schultz et al. 1997).

Using RL models, many studies have shown that dopamine responses conform to key principles of learning theory. For instance, when a US is predicted by a CS, a second CS presented at the same time or later than the first CS is ‘blocked’ from forming an association with the already predicted US, and dopamine neurons consistently fail to develop a response to the secondary, blocked CS (Steinberg et al. 2013; Waelti et al. 2001). Similarly, dopamine neurons are sensitive to temporal jittering of reward delivery. Early delivery of a predicted reward causes dopamine activation, whereas later than, predicted delivery leads to a diminished response (Hollerman and Schultz 1998). Moreover, dopamine responses reflect the discounting of future rewards, as suggested by reinforcement learning and economic theory (Enomoto et al. 2011; Fiorillo et al. 2008; Kobayashi and Schultz 2008). Trial-by-trial dopamine responses reflect the reinforcement history, a weighted average of past outcomes, in simple learning contexts (Bayer and Glimcher 2005). When the experimental context involves more complicated inter-trial task structure, dopamine neurons use this task structure to quickly update their responses on one trial, for instance, during reversal learning (Bromberg-Martin et al. 2010). Together, these results demonstrate the fidelity of dopamine responses to predictions made by learning theory, and they provide compelling evidence that phasic dopamine responses play a role in learning.

Dopamine activity reflects economic value

The magnitudes of dopamine prediction error responses scale positively with reward parameters that increase value, including reward size (Bayer and Glimcher 2005; Tobler et al. 2005), and probability (Fiorillo et al. 2003; Lak et al. 2016; Nakahara et al. 2004; Nomoto et al. 2010), and negatively with reward parameters that decrease value, including delays (Fiorillo et al. 2008; Kobayashi and Schultz 2008) and bitter substances (Fiorillo et al. 2013b). Moreover, when monkeys indicate preference rankings between goods that have the same reward magnitude, expected value, and delay, dopamine responses vigorously reflect the preference rankings for reward type (Lak et al. 2014) and information content (Bromberg-Martin and Hikosaka 2009). These results indicate that dopamine reward prediction error responses reflect subjective value.

To demonstrate the functional relationship between subjective value and dopamine activity, it is necessary to measure a psychometric function of subjective value. Economic theory demonstrates that choices between risky options reveal subjective value (utility) as a function of physical value (Debreu 1959; von Neumann et al. 1944). Risk-avoiding individuals display concave utility functions, where the potential loss is greater than potential gain (Fig. 2a). In contrast, risk seekers have convex utility functions, where the potential utility gain outweighs the potential utility loss (Fig. 2b). A psychometric utility function with a consequential shape—a shape that can be meaningfully correlated with a neurometric function—can, therefore, be measured from choices under risk (Caraco et al. 1980; Machina 1987; Stauffer et al. 2014). Choices between risky rewards show that monkeys are risk seeking for small rewards (McCoy and Platt 2005; O’Neill and Schultz 2010; Yamada et al. 2013), but become more risk avoiding as rewards get larger (Genest et al. 2016; Stauffer et al. 2014). This behavioral pattern, risk seeking for small rewards and risk avoiding for large rewards, translates into a convex then concave utility function (Fig. 2c) (Genest et al. 2016; Stauffer et al. 2014). The magnitudes of dopamine responses to unpredicted rewards are highly correlated with the shapes of the measured utility functions (Fig. 2d). In addition, when more reward than delivered is predicted, the prediction error response of dopamine neurons depends on the local slope of the utility function (Stauffer et al. 2014). These results demonstrate that the fundamental variable coded by dopamine prediction error responses is the same variable used to make decisions.

Fig. 2.

Fig. 2

Phasic dopamine responses code value. ab Example utility functions predict preferences between equi-probable (50:50) two outcome gambles (0.1, 0.9, arbitrary units) and the gambles’ expected values (EV) (0.5 a.u.). a Concave utility function indicates risk avoiding. b Convex utility function indicates risk seeking. Orange and brown two-sided arrows indicate the potential utility gain (G) and loss (L), respectively, relative to the utility of the expected value (uEV). For concave (risk avoiding) functions G < L, whereas for risk seeking (convex) functions G > L. c Measured utility function shows the utility of juice rewards. Convex regions of the utility (lower reward sizes) represent reward ranges, where the monkey was risk seeking. Concave regions (larger reward sizes) represent reward ranges, where the monkey was risk avoiding. Black dots represent points of subjective equivalence—termed certainty equivalents—between risky and safe rewards, measured through binary choices between risky and safe rewards. Solid line was fitted to the certainty equivalent data using cubic splines. d Dopamine neuron action potential responses are strongly correlated with the shape of the utility function. Action potentials were measured, while unpredicted rewards were delivered to the animals (sized 0.1–1.2 ml in 0.1 ml increments). Black bars represent impulse rate in a 500 ms window following reward. Error bars are SEM across 17 neurons. Red line represents utility functions and corresponds to secondary y-axis.

c, d Adapted from Stauffer et al. (2014)

An outstanding question related to economic value coding is whether economic costs reduce the magnitude of dopamine responses. In one study, increasing the effort required to get a reward resulted in reduced responses of some dopamine neurons, but not others (Pasquereau and Turner 2013). The behavioral measures used to gauge effort, however—reaction time and error rate—do not map linearly onto economic value. Thus, the true economic costs remained unknown. More studies are required to determine whether dopamine neurons code a net utility signal that accounts for the economic costs associated with effort.

When behavioral decisions are made, dopamine responses reflect the chosen value, which is a post-decision variable (Lak et al. 2016; Morris et al. 2006). The current data suggest that dopamine neurons do not play a direct role in selecting options for a particular choice. Nevertheless, the close correlation between dopamine response to rewards and reward utility suggests that dopamine-teaching signals play a fundamental role in the choices we make over time. This functional role in value-based decisions was demonstrated by a recent study using optogenetic stimulation of dopamine neurons in a macaque monkey. Reward predicting objects that were followed by optogenetic activation of dopamine neurons were chosen more frequently than identical objects that were not followed by optogenetic activation (Stauffer et al. 2016) (Fig. 3). Thus, the likely role of the dopamine prediction error response is to train downstream brain structures about value.

Fig. 3.

Fig. 3

Optical stimulation of ChR2 expressing dopamine neurons leads to neuronal and behavioral correlates of value. a Top, monkeys viewed visual stimuli that predicted liquid reward delivered with (blue) or without (red) accompanying optical stimulation. a Bottom, larger neuronal response (blue) occurred to cues that predicted optical stimulation, compared to neuronal responses (red) to cues that did not predict optical stimulation. Blue raster plot and PSTH aligned onto the appearance of cues predicting reward plus optical stimulation. Red raster plot and PSTH aligned onto the appearance of cues predicting reward alone in the same neuron. b Monkeys made saccade guided choices between two visual cues (same reward scheme as in a). When the optical fiber was placed in the channelrhodopsin-infected hemisphere, monkeys learned to choose the cue that predicted optical stimulation, over the cue that did not predict optical stimulation (blue, ‘injected’). When the optical fiber was placed in the contralateral hemisphere, where no channelrhodopsin virus was injected, the monkeys continued to choose either option with equal frequency (red, ‘control). Thus, the monkeys’ choices indicated that optical stimulation added value. Two choice sessions are shown, one with the optical fiber in the infected hemisphere (blue) and one session with the optical fiber in the control, uninfected hemisphere (red). The ‘x’ indicates trial-by-trial choices in each session. The smoothed lines represent a running average of the choices (10 trial sliding window).

This figure was adapted from Stauffer et al. (2016)

Challenges and future research directions

Despite the overwhelming evidence for the reward prediction error hypothesis of dopamine function, several outstanding challenges remain unaddressed. Here, we will attempt to define and outline three critical challenges deserving of attention. These include (1) determining whether dopamine neurons are functionally homogenous, (2) elucidating the relationship between behavioral measures and striatal dopamine release, and (3) defining the functional role of dopamine signals in the cortex.

Functional diversity in dopamine neurons

Several studies have reported that a fraction of dopamine neurons are excited by novel or aversive stimuli and outcomes (Brischoux et al. 2009; Cohen et al. 2012; Fiorillo 2013; Fiorillo et al. 2013b; Lak et al. 2016; Matsumoto and Hikosaka 2009; Schultz and Romo 1987). Although there is significant anatomical diversity in the input–output pathways between medial and lateral dopamine neurons (Lynd-Balta and Haber 1994a, b, c; Watabe-Uchida et al. 2012), it remains unclear whether these non-reward-related activations represent the activity of distinct dopamine neuron circuits.

Multiple aspects of dopamine signaling may contribute to the observed complexity of dopamine responses, including context dependency and complex temporal dynamics. Dopamine neurons are exquisitely sensitive to the experimental context. For instance, when visual stimuli predict both appetitive and aversive stimuli, approximately 40% of dopamine neurons respond to the stimulus predicting the aversive outcome (Matsumoto and Hikosaka 2009; Mirenowicz and Schultz 1996). When the sensory stimuli are more perceptually distinct, as in when reward is predicted by an auditory cue and aversive outcome predicted by a visual cue, the number of dopamine neurons that respond to the aversive cue drops dramatically (Mirenowicz and Schultz 1996). This shows that stimulus context influences the activity of dopamine neurons. Likewise, the distribution of outcomes also alters dopamine responding. Highly rewarding contexts, such as behavioral situations with high reward probability, increase dopamine activations to neutral cues (Kobayashi and Schultz 2014; Matsumoto et al. 2016) and cues that predict aversive outcomes (Matsumoto et al. 2016). Even trial-by-trial behavioral measures in mice and monkeys predict whether dopamine neurons will respond to the current behavioral stimuli (Lak et al. 2017; Matsumoto et al. 2016). Such context-, and even trial-, specific effects seem to demand a systems neuroscience perspective that places focus on the behavior of the animal and not just the underlying neural circuits. Well-controlled experiments will be critical to understand the behavioral consequences of dopamine activity in real-world environments with complex emotional contexts.

The temporal dynamics of the dopamine response can further complicate the interpretation of these responses. Short latency activations can occur 50–90 ms following behavioral events that reflect physical impact, novelty, and stimulus generalization (Fiorillo 2013; Fiorillo et al. 2013b; Lak et al. 2014; Matsumoto and Hikosaka 2009; Nomoto et al. 2010). These short latency responses are not modulated by value, whereas later response components are (Fiorillo et al. 2013a; Lak et al. 2016). Likewise, robust rebound activations are often observed following negative prediction error responses when dopamine neurons can be silent for 200–500 ms (Bayer et al. 2007; Fiorillo et al. 2013b). Together, these complex dynamics, such as short latency and rebound activations, can complicate the interpretation of negative prediction error responses. It is important to note that an abundance of caution should be exercised with the interpretation of this neuronal behavior. Despite the various conclusions that can be reached by applying statistics to selected time windows, it is unclear how an aversive outcome-predicting stimulus that evokes a short latency activation, a long pause in firing, and then a rebound activation would influence dopamine release in the striatum.

The relationship between behavior and striatal dopamine release

Classic studies have repeatedly shown that, even during operant paradigms, dopamine responses are time-locked to external reward predictors, rather than to the onset of well controlled, single joint movements or associated EMG activity (Fig. 4a) (Ljungberg et al. 1992; Schultz et al. 1993; Schultz and Romo 1990). In contrast, larger, multi-muscle movements in monkeys (Schultz et al. 1983) or whole-body movements in rodents (Dodson et al. 2016; Howe and Dombeck 2016; Howe et al. 2013) are correlated with increased dopamine activity in midbrain cell bodies and striatal dopamine release sites. For instance, a recent study in freely behaving rodents found that phasic dopamine release was time-locked to behavior as well as stimuli (Fig. 2d) (Hamid et al. 2016). Investigating this discrepancy and exploring the larger question of how information processing in the striatum is modulated by incoming dopamine signals are of critical importance. Local striatal neurons (cholinergic interneurons) and afferent connections can influence dopamine release at dopamine terminals in the striatum (Cachope and Cheer 2014; Threlfell et al. 2012). This local influence might be especially significant in primates, because the basal ganglia are spatially organized according to cortical inputs (Alexander and DeLong 1985a, b; Alexander et al. 1986). The activity of dopamine neurons and cholinergic neurons is correlated in the NHP basal ganglia, but it is unclear how the behavioral variables coded by cholinergic neurons influence dopamine release (Morris et al. 2004). It is, therefore, important to characterize dopamine release in different functional regions of the striatum and observe the relationship between release and well-controlled behaviors. Recently, cyclic voltammetry was used to monitor dopamine reward responses in NHP striatum (Min et al. 2016; Schluter et al. 2014; Yoshimi et al. 2015), and this technique can shed light on how local network effects influence dopamine release and whether release reflects behavioral parameters other than reward, such as movements and actions.

Fig. 4.

Fig. 4

Temporal discrepancy between dopamine action potential responses recorded in the midbrain and dopamine release monitored in the striatum. a PSTH (top) and raster plot (bottom) of dopamine response to reward predicting cues. Responses were aligned onto cue onset (solid line). The time of movement onset during each trial is indicated by the dark hatches in the raster plot. This panel was adapted from (Schultz et al. 1993). b Profile of dopamine concentration change in the striatum of a rat after reward prediction. Dopamine concentration profiles are aligned to the time when the rats inserted their nose into a center port (white dashed lines). The time of instruction cues for each trial is indicated by the red ticks. This figure panel was adapted from Hamid et al. (2016)

The role of fast dopamine signals in the frontal cortex

It is widely believed that phasic dopamine signals update action values in the striatum (Shen et al. 2008), but it is less well known what role these signals have in the frontal cortex. There was an extensive expansion of frontal cortex dopamine projections that accompanied the evolution of higher cognitive abilities in primates (Lewis et al. 1987; Smiley et al. 1994; Williams and Goldman-Rakic 1993, 1998). Accordingly, several studies have examined the role of dopamine in attention, working memory, and associative learning (Jacob et al. 2013, 2016; Noudoost and Moore 2011a; Puig and Miller 2012; Vijayraghavan et al. 2007; Williams and Goldman-Rakic 1995). These studies and others like them have relied upon the (relatively) slow process of agonist or antagonist infusion. Even when done with high spatial and temporal precision (Noudoost and Moore 2011b), these manipulations cannot approximate the natural dynamics of phasic dopamine signals. Instead, optogenetics can be employed for millisecond timescale control of dopamine release (Boyden et al. 2005; Tsai et al. 2009). Recently, a dual virus injection was shown to selectively label wild-type NHP dopamine neurons. Optical stimulation of cell bodies positively modulated behavioral read-outs of value. It was not known from that study whether the opsin was expressed in neuron terminals (Stauffer et al. 2016), but future research using next generation molecular tools in NHPs will permit projection specific recording and perturbation of neural activity.

Conclusions

Electrophysiological recordings from dopamine cell bodies in the midbrain have demonstrated that phasic dopamine responses code reward prediction error, the difference between received and predicted reward. Studies in awake, behaving NHP have been critical to this endeavor, because they are highly trainable and can provide a wealth of data through in-depth exploration of single unit dopamine activity. NHP possess a rich and complex behavioral repertoire which has led to advanced understanding of the role of dopamine in learning, movement, and decision making. Not discussed here but worth mentioning, the MPTP monkey model has been critical to the study of neuronal and behavioral deficits associated with Parkinson’s disease. In short, the unique properties of NHP models have made them essential to understanding midbrain dopamine function and dysfunction.

The findings reviewed here demonstrate that, even for studying a relatively simple and evolutionarily old neural structure like dopamine neurons, there are significant advantages to using NHP models. Non-human primates possess behavioral and anatomical characteristics that are more similar to humans than any other experimental animal model. From a behavioral standpoint, the cognitive capability and choice flexibility reviewed here and demonstrated elsewhere (Eiselt and Nieder 2013; Stauffer et al. 2015) resembles human choice behavior. From an anatomical perspective, NHP dopamine projections to the striatum and frontal cortex are most analogous to those in humans. The NHP striatum contains the densest concentration of dopamine terminals (Lynd-Balta and Haber 1994a, b) and is functionally organized according to cortical inputs (Alexander and DeLong 1985a, b; Alexander et al. 1986). Likewise, the dopamine projections to the frontal cortex are massively expanded in NHP, where they primarily target executive and motor regions (Smiley et al. 1994; Williams and Goldman-Rakic 1993, 1998). For these reasons, and because of the clinical relevance of dopamine to numerous movement and mental health disorders including but not limited to Parkinson’s disease, dystonia, ADHD, OCD, psychosis, depression, and schizophrenia, it is critical to maintain and advance behavioral neurophysiology in awake, behaving primates.

A new generation of molecular tools—including optogenetics and in vivo single cell imaging—has revolutionized how we ask questions and even what questions we can ask. These technologies, however, have not been widely incorporated into monkey neurophysiology studies. Although progress is being made, as reviewed elsewhere in this issue (Galvan et al. 2017), there are many technical challenges impeding easy implementation of next generation molecular tools in NHP. Efficient light delivery, large-scale viral infection, and the lack of genetically modified NHP lines all pose significant challenges. Nevertheless, recent developments, including red shifted opsins and improved optical fibers (Acker et al. 2016), better recording/stimulating devices (Yazdan-Shahmorad et al. 2016), and virally mediated cell-type-specific ChR2 expression (El-Shamayleh et al. 2017; Klein et al. 2016; Stauffer et al. 2016), point towards a promising future for optogenetic NHP studies. These new technologies coupled with the enormous and oft-demonstrated utility of the NHP model should ensure continued focus on and research in this most evolutionarily relevant experimental species.

Acknowledgements

The authors would like to thank Andreea Bostan, Jing He, and Amber Torrise for comments and discussion. This work was supported by the University of Pittsburgh Brain Institute and by the National Institutes of Health through the NIH Director’s New Innovator Award 1DP2MH113095.

References

  1. Acker L, Pino EN, Boyden ES, Desimone R. FEF inactivation with improved optogenetic methods. Proc Natl Acad Sci USA. 2016;113:E7297–E7306. doi: 10.1073/pnas.1610784113. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Aebischer P, Schultz W. The activity of pars compacta neurons of the monkey substantia nigra is depressed by apomorphine. Neurosci Lett. 1984;50:25–29. doi: 10.1016/0304-3940(84)90456-7. [DOI] [PubMed] [Google Scholar]
  3. Alexander GE, DeLong MR. Microstimulation of the primate neostriatum. I. Physiological properties of striatal microexcitable zones. J Neurophysiol. 1985;53:1401–1416. doi: 10.1152/jn.1985.53.6.1401. [DOI] [PubMed] [Google Scholar]
  4. Alexander GE, DeLong MR. Microstimulation of the primate neostriatum. II. Somatotopic organization of striatal microexcitable zones and their relation to neuronal response properties. J Neurophysiol. 1985;53:1417–1430. doi: 10.1152/jn.1985.53.6.1417. [DOI] [PubMed] [Google Scholar]
  5. Alexander GE, DeLong MR, Strick PL. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu Rev Neurosci. 1986;9:357–381. doi: 10.1146/annurev.ne.09.030186.002041. [DOI] [PubMed] [Google Scholar]
  6. Bayer HM, Glimcher PW. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron. 2005;47:129–141. doi: 10.1016/j.neuron.2005.05.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bayer HM, Lau B, Glimcher PW. Statistics of midbrain dopamine neuron spike trains in the awake primate. J Neurophysiol. 2007;98:1428–1439. doi: 10.1152/jn.01140.2006. [DOI] [PubMed] [Google Scholar]
  8. Boyden ES, Zhang F, Bamberg E, Nagel G, Deisseroth K. Millisecond-timescale, genetically targeted optical control of neural activity. Nat Neurosci. 2005;8:1263–1268. doi: 10.1038/nn1525. [DOI] [PubMed] [Google Scholar]
  9. Brischoux F, Chakraborty S, Brierley DI, Ungless MA. Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc Natl Acad Sci. 2009;106:4894–4899. doi: 10.1073/pnas.0811507106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Bromberg-Martin ES, Hikosaka O. Midbrain dopamine neurons signal preference for advance information about upcoming rewards. Neuron. 2009;63:119–126. doi: 10.1016/j.neuron.2009.06.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Bromberg-Martin ES, Matsumoto M, Hong S, Hikosaka O. A pallidus-habenula-dopamine pathway signals inferred stimulus values. J Neurophysiol. 2010;104:1068–1076. doi: 10.1152/jn.00158.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Bunney BS, Aghajanian GK, Roth RH. Comparison of effects of l-dopa, amphetamine and apomorphine on firing rate of rat dopaminergic neurones. Nature. 1973;245:123–125. doi: 10.1038/newbio245123a0. [DOI] [PubMed] [Google Scholar]
  13. Cachope R, Cheer JF. Local control of striatal dopamine release. Front Behav Neurosci. 2014;8:188. doi: 10.3389/fnbeh.2014.00188. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Caraco T, Martindale S, Whittam TS. An empirical demonstration of risk-sensitive foraging preferences. Anim Behav. 1980;28:820–830. [Google Scholar]
  15. Cohen JY, Haesler S, Vong L, Lowell BB, Uchida N. Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature. 2012;482:85–88. doi: 10.1038/nature10754. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Dahlstroem A, Fuxe K. Evidence for the existence of monoamine-containing neurons in the central nervous system. I. Demonstration of monoamines in the cell bodies of brain stem neurons. Acta Physiol Scand Suppl. 1964;232:231–255. [PubMed] [Google Scholar]
  17. Debreu G. Cardinal utility for even-chance mixtures of pairs of sure prospects. Rev Econ Stud. 1959;26:174–177. [Google Scholar]
  18. Dodson PD, Dreyer JK, Jennings KA, Syed ECJ, Wade-Martins R, Cragg SJ, Bolam JP, Magill PJ. Representation of spontaneous movement by dopaminergic neurons is cell-type selective and disrupted in parkinsonism. Proc Natl Acad Sci. 2016;113:E2180–E2188. doi: 10.1073/pnas.1515941113. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Eiselt AK, Nieder A. Representation of abstract quantitative rules applied to spatial and numerical magnitudes in primate prefrontal cortex. J Neurosci. 2013;33:7526–7534. doi: 10.1523/JNEUROSCI.5827-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. El-Shamayleh Y, Kojima Y, Soetedjo R, Horwitz GD. Selective optogenetic control of Purkinje cells in monkey cerebellum. Neuron. 2017;95:51–62. doi: 10.1016/j.neuron.2017.06.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Enomoto K, Matsumoto N, Nakai S, Satoh T, Sato TK, Ueda Y, Inokawa H, Haruno M, Kimura M. Dopamine neurons learn to encode the long-term value of multiple future rewards. Proc Natl Acad Sci USA. 2011;108:15462–15467. doi: 10.1073/pnas.1014457108. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Eshel N, Bukwich M, Rao V, Hemmelder V, Tian J, Uchida N. Arithmetic and local circuitry underlying dopamine prediction errors. Nature. 2015;525:243–246. doi: 10.1038/nature14855. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Fiorillo CD. Two dimensions of value: dopamine neurons represent reward but not aversiveness. Science. 2013;341:546–549. doi: 10.1126/science.1238699. [DOI] [PubMed] [Google Scholar]
  24. Fiorillo CD, Tobler PN, Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science. 2003;299:1898–1902. doi: 10.1126/science.1077349. [DOI] [PubMed] [Google Scholar]
  25. Fiorillo CD, Newsome WT, Schultz W. The temporal precision of reward prediction in dopamine neurons. Nat Neurosci. 2008;11:966–973. doi: 10.1038/nn.2159. [DOI] [PubMed] [Google Scholar]
  26. Fiorillo CD, Song MR, Yun SR. Multiphasic temporal dynamics in responses of midbrain dopamine neurons to appetitive and aversive stimuli. J Neurosci. 2013;33:4710–4725. doi: 10.1523/JNEUROSCI.3883-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Fiorillo CD, Yun SR, Song MR. Diversity and homogeneity in responses of midbrain dopamine neurons. J Neurosci. 2013;33:4693–4709. doi: 10.1523/JNEUROSCI.3886-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Galvan A, Caiola MJ, Albaugh DL. Advances in optogenetic and chemogenetic methods to study brain circuits in non-human primates. J Neural Transm (Vienna) 2017 doi: 10.1007/s00702-017-1697-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Genest W, Stauffer WR, Schultz W. Utility functions predict variance and skewness risk preferences in monkeys. Proc Natl Acad Sci USA. 2016;113:8402–8407. doi: 10.1073/pnas.1602217113. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. German DC, Manaye KF. Midbrain dopaminergic neurons (nuclei A8, A9, and A10): three-dimensional reconstruction in the rat. J Comp Neurol. 1993;331:297–309. doi: 10.1002/cne.903310302. [DOI] [PubMed] [Google Scholar]
  31. Gonon FG. Nonlinear relationship between impulse flow and dopamine released by rat midbrain dopaminergic neurons as studied by in vivo electrochemistry. Neuroscience. 1988;24:19–28. doi: 10.1016/0306-4522(88)90307-7. [DOI] [PubMed] [Google Scholar]
  32. Grace AA, Bunney BS. Intracellular and extracellular electrophysiology of nigral dopaminergic neurons—1. Identification and characterization. Neuroscience. 1983;10:301–315. doi: 10.1016/0306-4522(83)90135-5. [DOI] [PubMed] [Google Scholar]
  33. Guyenet PG, Aghajanian GK. Antidromic identification of dopaminergic and other output neurons of the rat substantia nigra. Brain Res. 1978;150:69–84. doi: 10.1016/0006-8993(78)90654-6. [DOI] [PubMed] [Google Scholar]
  34. Hamid AA, Pettibone JR, Mabrouk OS, Hetrick VL, Schmidt R, Vander Weele CM, Kennedy RT, Aragona BJ, Berke JD. Mesolimbic dopamine signals the value of work. Nat Neurosci. 2016;19:117–126. doi: 10.1038/nn.4173. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Hollerman JR, Schultz W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci. 1998;1:304–309. doi: 10.1038/1124. [DOI] [PubMed] [Google Scholar]
  36. Howe MW, Dombeck DA. Rapid signalling in distinct dopaminergic axons during locomotion and reward. Nature. 2016;535:505–510. doi: 10.1038/nature18942. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Howe MW, Tierney PL, Sandberg SG, Phillips PE, Graybiel AM. Prolonged dopamine signalling in striatum signals proximity and value of distant rewards. Nature. 2013;500:575–579. doi: 10.1038/nature12475. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Jacob SN, Ott T, Nieder A. Dopamine regulates two classes of primate prefrontal neurons that represent sensory signals. J Neurosci. 2013;33:13724–13734. doi: 10.1523/JNEUROSCI.0210-13.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Jacob SN, Stalter M, Nieder A. Cell-type-specific modulation of targets and distractors by dopamine D1 receptors in primate prefrontal cortex. Nat Commun. 2016;7:13218. doi: 10.1038/ncomms13218. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Klein C, Evrard HC, Shapcott KA, Haverkamp S, Logothetis NK, Schmid MC. Cell-targeted optogenetics and electrical microstimulation reveal the primate koniocellular projection to supra-granular visual cortex. Neuron. 2016;90:143–151. doi: 10.1016/j.neuron.2016.02.036. [DOI] [PubMed] [Google Scholar]
  41. Kobayashi S, Schultz W. Influence of reward delays on responses of dopamine neurons. J Neurosci. 2008;28:7837–7846. doi: 10.1523/JNEUROSCI.1600-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Kobayashi S, Schultz W. Reward contexts extend dopamine signals to unrewarded stimuli. Curr Biol. 2014;24:56–62. doi: 10.1016/j.cub.2013.10.061. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Lak A, Stauffer WR, Schultz W. Dopamine prediction error responses integrate subjective value from different reward dimensions. Proc Natl Acad Sci USA. 2014;111:2343–2348. doi: 10.1073/pnas.1321596111. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Lak A, Stauffer WR, Schultz W. Dopamine neurons learn relative chosen value from probabilistic rewards. Elife. 2016;5:e18044. doi: 10.7554/eLife.18044. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Lak A, Nomoto K, Keramati M, Sakagami M, Kepecs A. Midbrain dopamine neurons signal belief in choice accuracy during a perceptual decision. Curr Biol. 2017;27:821–832. doi: 10.1016/j.cub.2017.02.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Lammel S, Lim BK, Ran C, Huang KW, Betley MJ, Tye KM, Deisseroth K, Malenka RC. Input-specific control of reward and aversion in the ventral tegmental area. Nature. 2012;491:212–217. doi: 10.1038/nature11527. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Lammel S, Steinberg EE, Foldy C, Wall NR, Beier K, Luo L, Malenka RC. Diversity of transgenic mouse models for selective targeting of midbrain dopamine neurons. Neuron. 2015;85:429–438. doi: 10.1016/j.neuron.2014.12.036. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Langston JW, Ballard P, Tetrud JW, Irwin I. Chronic Parkinsonism in humans due to a product of meperidine-analog synthesis. Science. 1983;219:979–980. doi: 10.1126/science.6823561. [DOI] [PubMed] [Google Scholar]
  49. Lewis DA, Campbell MJ, Foote SL, Goldstein M, Morrison JH. The distribution of tyrosine hydroxylase-immunoreactive fibers in primate neocortex is widespread but regionally specific. J Neurosci. 1987;7:279–290. doi: 10.1523/JNEUROSCI.07-01-00279.1987. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Ljungberg T, Apicella P, Schultz W. Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol. 1992;67:145–163. doi: 10.1152/jn.1992.67.1.145. [DOI] [PubMed] [Google Scholar]
  51. Lovenberg W, Weissbach H, Udenfriend S. Aromatic l-amino acid decarboxylase. J Biol Chem. 1962;237:89–93. [PubMed] [Google Scholar]
  52. Lynd-Balta E, Haber SN. The organization of midbrain projections to the striatum in the primate: sensorimotor-related striatum versus ventral striatum. Neuroscience. 1994;59:625–640. doi: 10.1016/0306-4522(94)90182-1. [DOI] [PubMed] [Google Scholar]
  53. Lynd-Balta E, Haber SN. The organization of midbrain projections to the ventral striatum in the primate. Neuroscience. 1994;59:609–623. doi: 10.1016/0306-4522(94)90181-3. [DOI] [PubMed] [Google Scholar]
  54. Lynd-Balta E, Haber SN. Primate striatonigral projections: a comparison of the sensorimotor-related striatum and the ventral striatum. J Comp Neurol. 1994;345:562–578. doi: 10.1002/cne.903450407. [DOI] [PubMed] [Google Scholar]
  55. Machina MJ. Choice under uncertainty: problems solved and unsolved. J Econ Perspect. 1987;1:121–154. [Google Scholar]
  56. Matsumoto M, Hikosaka O. Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature. 2009;459:837–841. doi: 10.1038/nature08028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Matsumoto H, Tian J, Uchida N, Watabe-Uchida M. Midbrain dopamine neurons signal aversion in a reward-context-dependent manner. Elife. 2016;5:e17328. doi: 10.7554/eLife.17328. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. McCoy AN, Platt ML. Risk-sensitive neurons in macaque posterior cingulate cortex. Nat Neurosci. 2005;8:1220–1227. doi: 10.1038/nn1523. [DOI] [PubMed] [Google Scholar]
  59. Min HK, Ross EK, Jo HJ, Cho S, Settell ML, Jeong JH, Duffy PS, Chang SY, Bennet KE, Blaha CD, et al. Dopamine release in the nonhuman primate caudate and putamen depends upon site of stimulation in the subthalamic nucleus. J Neurosci. 2016;36:6022–6029. doi: 10.1523/JNEUROSCI.0403-16.2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Mirenowicz J, Schultz W. Importance of unpredictability for reward responses in primate dopamine neurons. J Neurophysiol. 1994;72:1024–1027. doi: 10.1152/jn.1994.72.2.1024. [DOI] [PubMed] [Google Scholar]
  61. Mirenowicz J, Schultz W. Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature. 1996;379:449–451. doi: 10.1038/379449a0. [DOI] [PubMed] [Google Scholar]
  62. Montague PR, Dayan P, Sejnowski TJ. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci. 1996;16:1936–1947. doi: 10.1523/JNEUROSCI.16-05-01936.1996. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Morris G, Arkadir D, Nevet A, Vaadia E, Bergman H. Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron. 2004;43:133–143. doi: 10.1016/j.neuron.2004.06.012. [DOI] [PubMed] [Google Scholar]
  64. Morris G, Nevet A, Arkadir D, Vaadia E, Bergman H. Midbrain dopamine neurons encode decisions for future action. Nat Neurosci. 2006;9:1057–1063. doi: 10.1038/nn1743. [DOI] [PubMed] [Google Scholar]
  65. Nakahara H, Itoh H, Kawagoe R, Takikawa Y, Hikosaka O. Dopamine neurons can represent context-dependent prediction error. Neuron. 2004;41:269–280. doi: 10.1016/s0896-6273(03)00869-9. [DOI] [PubMed] [Google Scholar]
  66. Nomoto K, Schultz W, Watanabe T, Sakagami M. Temporally extended dopamine responses to perceptually demanding reward-predictive stimuli. J Neurosci. 2010;30:10692–10702. doi: 10.1523/JNEUROSCI.4828-09.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Noudoost B, Moore T. Control of visual cortical signals by prefrontal dopamine. Nature. 2011;474:372–375. doi: 10.1038/nature09995. [DOI] [PMC free article] [PubMed] [Google Scholar]
  68. Noudoost B, Moore T. A reliable microinjectrode system for use in behaving monkeys. J Neurosci Methods. 2011;194:218–223. doi: 10.1016/j.jneumeth.2010.10.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  69. Olds J, Milner P. Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J Comp Physiol Psychol. 1954;47:419–427. doi: 10.1037/h0058775. [DOI] [PubMed] [Google Scholar]
  70. O’Neill M, Schultz W. Coding of reward risk by orbitofrontal neurons is mostly distinct from coding of reward value. Neuron. 2010;68:789–800. doi: 10.1016/j.neuron.2010.09.031. [DOI] [PubMed] [Google Scholar]
  71. Pan WX, Schmidt R, Wickens JR, Hyland BI. Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J Neurosci. 2005;25:6235–6242. doi: 10.1523/JNEUROSCI.1478-05.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  72. Pasquereau B, Turner RS. Limited encoding of effort by dopamine neurons in a cost-benefit trade-off task. J Neurosci. 2013;33:8288–8300. doi: 10.1523/JNEUROSCI.4619-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  73. Puig MV, Miller EK. The role of prefrontal dopamine D1 receptors in the neural mechanisms of associative learning. Neuron. 2012;74:874–886. doi: 10.1016/j.neuron.2012.04.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  74. Rescorla RA, Wagner AR. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and non reinforcement. In: Black AH, Prokasy WF, editors. Classical conditioning II: current research and theory. New York: Appleton-Century-Crofts; 1972. pp. 64–99. [Google Scholar]
  75. Romo R, Schultz W. Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. J Neurophysiol. 1990;63:592–606. doi: 10.1152/jn.1990.63.3.592. [DOI] [PubMed] [Google Scholar]
  76. Schluter EW, Mitz AR, Cheer JF, Averbeck BB. Real-time dopamine measurement in awake monkeys. PLoS One. 2014;9:e98692. doi: 10.1371/journal.pone.0098692. [DOI] [PMC free article] [PubMed] [Google Scholar]
  77. Schultz W. Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. J Neurophysiol. 1986;56:1439–1461. doi: 10.1152/jn.1986.56.5.1439. [DOI] [PubMed] [Google Scholar]
  78. Schultz W, Romo R. Responses of nigrostriatal dopamine neurons to high-intensity somatosensory stimulation in the anesthetized monkey. J Neurophysiol. 1987;57:201–217. doi: 10.1152/jn.1987.57.1.201. [DOI] [PubMed] [Google Scholar]
  79. Schultz W, Romo R. Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioral reactions. J Neurophysiol. 1990;63:607–624. doi: 10.1152/jn.1990.63.3.607. [DOI] [PubMed] [Google Scholar]
  80. Schultz W, Ruffieux A, Aebischer P. The activity of pars compacta neurons of the monkey substantia nigra in relation to motor activation. Exp Brain Res. 1983;51:377–387. [Google Scholar]
  81. Schultz W, Apicella P, Ljungberg T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci. 1993;13:900–913. doi: 10.1523/JNEUROSCI.13-03-00900.1993. [DOI] [PMC free article] [PubMed] [Google Scholar]
  82. Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275:1593–1599. doi: 10.1126/science.275.5306.1593. [DOI] [PubMed] [Google Scholar]
  83. Shen W, Flajolet M, Greengard P, Surmeier DJ. Dichotomous dopaminergic control of striatal synaptic plasticity. Science. 2008;321:848–851. doi: 10.1126/science.1160575. [DOI] [PMC free article] [PubMed] [Google Scholar]
  84. Smiley JF, Levey AI, Ciliax BJ, Goldman-Rakic PS. D1 dopamine receptor immunoreactivity in human and monkey cerebral cortex: predominant and extrasynaptic localization in dendritic spines. Proc Natl Acad Sci USA. 1994;91:5720–5724. doi: 10.1073/pnas.91.12.5720. [DOI] [PMC free article] [PubMed] [Google Scholar]
  85. Smith Y, Wichmann T, DeLong MR. Corticostriatal and mesocortical dopamine systems: do species differences matter? Nat Rev Neurosci. 2014;15:63. doi: 10.1038/nrn3469-c1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  86. Stark AK, Pakkenberg B. Histological changes of the dopaminergic nigrostriatal system in aging. Cell Tissue Res. 2004;318:81–92. doi: 10.1007/s00441-004-0972-9. [DOI] [PubMed] [Google Scholar]
  87. Stauffer WR, Lak A, Schultz W. Dopamine reward prediction error responses reflect marginal utility. Curr Biol. 2014;24:2491–2500. doi: 10.1016/j.cub.2014.08.064. [DOI] [PMC free article] [PubMed] [Google Scholar]
  88. Stauffer WR, Lak A, Bossaerts P, Schultz W. Economic choices reveal probability distortion in macaque monkeys. J Neurosci. 2015;35:3146–3154. doi: 10.1523/JNEUROSCI.3653-14.2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  89. Stauffer WR, Lak A, Yang A, Borel M, Paulsen O, Boyden ES, Schultz W. Dopamine neuron-specific optogenetic stimulation in rhesus macaques. Cell. 2016;166(1564–1571):e1566. doi: 10.1016/j.cell.2016.08.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
  90. Steinberg EE, Keiflin R, Boivin JR, Witten IB, Deisseroth K, Janak PH. A causal link between prediction errors, dopamine neurons and learning. Nat Neurosci. 2013;16:966–973. doi: 10.1038/nn.3413. [DOI] [PMC free article] [PubMed] [Google Scholar]
  91. Studer A, Schultz W. The catecholamine uptake inhibitor nomifensine depresses impulse activity of dopamine neurons in mouse substantia nigra. Neurosci Lett. 1987;80:207–212. doi: 10.1016/0304-3940(87)90655-0. [DOI] [PubMed] [Google Scholar]
  92. Sutton R, Barto A. Reinforcement learning: an introduction. Cambridge: MIT Press; 1998. [Google Scholar]
  93. Threlfell S, Lalic T, Platt NJ, Jennings KA, Deisseroth K, Cragg SJ. Striatal dopamine release is triggered by synchronized activity in cholinergic interneurons. Neuron. 2012;75:58–64. doi: 10.1016/j.neuron.2012.04.038. [DOI] [PubMed] [Google Scholar]
  94. Tobler PN, Fiorillo CD, Schultz W. Adaptive coding of reward value by dopamine neurons. Science. 2005;307:1642–1645. doi: 10.1126/science.1105370. [DOI] [PubMed] [Google Scholar]
  95. Tsai HC, Zhang F, Adamantidis A, Stuber GD, Bonci A, de Lecea L, Deisseroth K. Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science. 2009;324:1080–1084. doi: 10.1126/science.1168878. [DOI] [PMC free article] [PubMed] [Google Scholar]
  96. Ungless MA, Grace AA. Are you or aren’t you? Challenges associated with physiologically identifying dopamine neurons. Trends Neurosci. 2012;35:422–430. doi: 10.1016/j.tins.2012.02.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  97. Vijayraghavan S, Wang M, Birnbaum SG, Williams GV, Arnsten AF. Inverted-U dopamine D1 receptor actions on prefrontal neurons engaged in working memory. Nat Neurosci. 2007;10:376–384. doi: 10.1038/nn1846. [DOI] [PubMed] [Google Scholar]
  98. von Neumann J, Morgenstern O, Kuhn HW, Rubinstein A. Theory of games and economic behavior (60th anniversary commemorative edition) Princeton: Princeton University Press; 1944. [Google Scholar]
  99. Waelti P, Dickinson A, Schultz W. Dopamine responses comply with basic assumptions of formal learning theory. Nature. 2001;412:43–48. doi: 10.1038/35083500. [DOI] [PubMed] [Google Scholar]
  100. Watabe-Uchida M, Zhu L, Ogawa SK, Vamanrao A, Uchida N. Whole-brain mapping of direct inputs to midbrain dopamine neurons. Neuron. 2012;74:858–873. doi: 10.1016/j.neuron.2012.03.017. [DOI] [PubMed] [Google Scholar]
  101. Williams SM, Goldman-Rakic PS. Characterization of the dopaminergic innervation of the primate frontal cortex using a dopamine-specific antibody. Cereb Cortex. 1993;3:199–222. doi: 10.1093/cercor/3.3.199. [DOI] [PubMed] [Google Scholar]
  102. Williams GV, Goldman-Rakic PS. Modulation of memory fields by dopamine D1 receptors in prefrontal cortex. Nature. 1995;376:572–575. doi: 10.1038/376572a0. [DOI] [PubMed] [Google Scholar]
  103. Williams SM, Goldman-Rakic PS. Widespread origin of the primate mesofrontal dopamine system. Cereb Cortex. 1998;8:321–345. doi: 10.1093/cercor/8.4.321. [DOI] [PubMed] [Google Scholar]
  104. Yamada H, Tymula A, Louie K, Glimcher PW. Thirst-dependent risk preferences in monkeys identify a primitive form of wealth. Proc Natl Acad Sci USA. 2013;110:15788–15793. doi: 10.1073/pnas.1308718110. [DOI] [PMC free article] [PubMed] [Google Scholar]
  105. Yazdan-Shahmorad A, Diaz-Botia C, Hanson TL, Kharazia V, Ledochowitsch P, Maharbiz MM, Sabes PN. A large-scale interface for optogenetic stimulation and recording in nonhuman primates. Neuron. 2016;89:927–939. doi: 10.1016/j.neuron.2016.01.013. [DOI] [PubMed] [Google Scholar]
  106. Yelin R, Schuldiner S. The pharmacological profile of the vesicular monoamine transporter resembles that of multidrug transporters. FEBS Lett. 1995;377:201–207. doi: 10.1016/0014-5793(95)01346-6. [DOI] [PubMed] [Google Scholar]
  107. Yoshimi K, Kumada S, Weitemier A, Jo T, Inoue M. Reward-induced phasic dopamine release in the monkey ventral striatum and putamen. PLoS One. 2015;10:e0130443. doi: 10.1371/journal.pone.0130443. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Journal of Neural Transmission are provided here courtesy of Springer

RESOURCES