Roles for nigrosriatal—not just mesocorticolimbic—dopamine in reward and addiction

Abstract

Forebrain dopamine circuitry has traditionally been studied by two largely independent specialist groups: students of Parkinson’s disease who study the nigrostriatal dopamine system that originates in the substantia nigra (SN), and students of motivation and addiction who study the role of the mesolimbic and mesocortical dopamine systems that originate in the ventral tegmental area (VTA). The anatomical evidence for independent nigrostriatal and mesolimbic dopamine systems has, however, long been obsolete. There is now compelling evidence that both nominal “systems” participate in reward function and addiction. Electrical stimulation of both SN and VTA is rewarding, blockade of glutamatergic or cholinergic input to either SN or VTA attenuates the habit-forming effects of intravenous cocaine, and dopamine in both nigrostriatal and mesocorticolimbic terminal fields participates in the defining property of rewarding events: the reinforcement of memory consolidation. Thus the similarities between nigrostriatal and mesolimbic dopamine systems may be as important as their differences.

Introduction

The midbrain dopamine neurons that project to the forebrain were initially identified as a single continuous layer [1], arising from a single embryological cell group [2]. However, perhaps because the lateral and medial portions were largely restricted to established brain regions—the dorsal substantia nigra (SN) and the ventral tegmental area of Tsai (VTA)—the lateral and medial portions of this layer were given different labels (A9 and A10, respectively) and eventually became identified with two distinct nominal systems (a nigrostriatal system and a mesolimbic system [3]. The two nominal systems in turn became identified with different functions: the nigrostriatal system—known to degenerate in Parkinson’s disease—with motor function, and the mesolimbic system—important for the habit-forming effects of cocaine and for approach behaviors—with motivation and reward function. While the simple dichotomy between a nigrostriatal motor system and a meso-limbic reward and motivational system has had long-lasting influence, it is a misleading dichotomy. The nigrostriatal and mesolimbic dopamine “systems” are not simply differentiated anatomically, and significant functional interactions between the two systems have been recently suggested [4, 5]. Here I discuss the anatomical complications and lines of functional evidence that the substantia nigra dopamine neurons, and not just those of the ventral tegmental area, play a significant role in reward and addiction.

Anatomical complications

As was noted from the start [1], there is no clear boundary between the two nominal midbrain dopamine systems (Figure 1). Anterograde and retrograde tracing studies show that the SN and VTA dopamine cells have overlapping, not distinct, projection fields [6]. The discovery of dopamine terminals in prefrontal cortex—initially thought to arise uniquely from the VTA—prompted postulation of a third (mesocortical) or an expanded (mesocorticolimbic) system, but projections to the prefrontal cortex were subsequently found to arise from the medial SN as well as from the VTA [6]. Further, what is usually referred to SN pars compacta (SNc) is now known to comprise two layers or “tiers,” a ventral tier projecting mainly to the dorsal striatum, and a dorsal tier projecting to limbic [7] as well as striatal [8] targets. It is only the ventral tier or “densocellular layer” that has the tight packing appropriate for the label “pars compacta” [9]. The most lateral dopaminergic neurons, in SN pars lateralis (SNl, a loosely compacted lateral extension of SNc, and thus more similar to the dorsal than to the ventral tier of SNc) projects not to the dorsal striatum but rather to a traditionally limbic target (but see [10]): the amygdala [6]. Thus it is no longer possible to think of the limbic and striatal dopamine systems as arising from anatomically distinct lateral (SN) and medial (VTA) origins.

Figure 1.

Coronal sections of the ventral midbrain of the rat. (a) The blue line outlines the layer of dopaminergic cell bodies and dendrites (dark brown) as revealed by tyrosine hydroxylase immunohistochemistry. The green line outlines the GABAergic cell bodies (purple) of the substantia nigra pars reticulata as revealed by in situ hybridization. (b) In the same section, mRNA expression is shown for corticotropin-binding factor (white grain aggregates), a protein expressed in sub-populations of dopaminergic and GABAergic cells in regions with limbic and cortical projections: the VTA; the dorsal tier of SNc (a lateral extension of the parabrachial pigmented area of the VTA); and SNl (a regon projecting primarily to the amygdala). This protein is never expressed in the dopaminergic cells of the densocellular ventral tier of SNc (outlined in white), the origin of the nigrostriatal system, or in any of the cells of the SNr. (Note that the spacing of the cells in the dorsal and ventral tiers is not easily seen with TH immunohistochemistry, which labels dendrites as well as somata; the spacing is evident, however, from in situ hybridization or nissl-stained material: see [9], Fig. 3). Photomicrographs courtesy of Dr. Marisela Morales and Dr. Hui-Ling Wang; see [9] for full details.

Moreover, the terminal fields also fail to segregate simply. The major terminal field of the mesolimbic dopamine system—the unfortunately named nucleus accumbens septi—is not an unambiguously limbic structure. While some early investigators assumed accumbens to share olfactory functions (smell brain) with the septum and olfactory tubercle, others saw accumbens as a ventral extension of the striatum. The major portion of accumbens, the core subregion, is now understood to be an extension of the striatum and is, with the olfactory tubercle, known as the ventral striatum [11]. The medial “shell” portion of accumbens, however, can still be considered “limbic” as it is continuous with, has similar connections to, and is now often considered part of an extended delineation of the amygdala [11]. Part of the problem is that the so-called limbic system has never been consistently defined.

The nigrostriatal and mesolimbic dopamine systems are not anatomically distinct at either the level of their cells of origin or at the level of their terminal fields. Thus it should not be surprising that they are also not distinct functionally. In the following sections I discuss functional overlap between the two systems with respect to brain stimulation reward, cocaine reward, reward prediction, and reinforcement of memory consolidation.

Brain Stimulation Reward

The discovery that rats and humans will learn to work for direct electrical stimulation of certain brain regions (termed “intracranial self-stimulation”), coupled with the observation that humans find such stimulation pleasurable, led to the mapping of reward-related circuitry throughout the brain. Rats learn to work for stimulation of dozens of brain sites, many of which are presumably linked in series to form one or more reward-related circuits or sub-circuits. The most unambiguous self-stimulation involves stimulation sites along the medial forebrain bundle (MFB).

Dopamine antagonists (neuroleptics) attenuate the rewarding effects of MFB stimulation at doses that do not cause significant motoric debilitation [12, 13]. Conversely, the indirect dopamine agonist amphetamine enhances the rewarding effects of the stimulation and antagonizes the effects of neuroleptics in a manner that can be differentiated from simple augmentation of motor function [14]. Thus some subset of midbrain dopamine neurons plays a critical role in the rewarding effects of medial forebrain bundle electrical stimulation. While it was originally thought that rewarding MFB stimulation activated dopamine fibers directly, it is now believed that—at traditional stimulation parameters—it selectively activates afferent input to the dopamine system [15]. Thus the midbrain dopamine neurons are currently seen as a final common path for the rewarding effects of MFB stimulation.

While it is the mesocorticolimbic dopamine system that is most frequently associated with brain stimulation reward, the dopaminergic fibers from the SN and the VTA each enter and project along the MFB [3], and reward sites are found in both SN and VTA [16, 17]. Within-subjects mapping with a moveable electrode confirms that the boundaries of these reward sites correspond to the boundaries of the dopaminergic cell body region of SN and VTA, including SN pars compacta (SNc) and SN pars lateralis [18, 19]. The refractory periods for the directly depolarized substrate of ventral midbrain self-stimulation are similar to those of MFB self-stimulation, suggesting again that what are activated directly in this region are the sensitive terminals of the dopamine afferents rather than the relatively insensitive dopamine neurons themselves.

Moveable-electrode mapping studies also confirm that stimulation in the primary terminal fields of both the nigrostriatal and the mesocorticolimbic systems is rewarding [20, 21]. While there are regional differences in thresholds, response rates, and time to learn, these differences are graded across both dorsal (caudate) and ventral (accumbens, olfactory tubercle) striatum. Threshold and time to learn are not significantly different between nucleus accumbens and the adjacent anterior ventromedial caudate-putamen; indeed the differences between these two subregions of the striatum are no greater than any other adjacent pair of striatal subregions [21]. In the region of the dopamine cell bodies, thresholds between the best SNc and VTA placements are comparable. Response rates associated with VTA and SN stimulation differ considerably, but this appears to be largely influenced by motor (turning) artifacts due to spread of current dorsal to SNc [18]. Moreover, differences in response rate associated with positive sites in SNc and VTA are minimal compared to differences associated with sites within SNc, where stimulation at rostral placements tend to be positive and stimulation at caudal placements tends to be ineffective [18]. Thus comparisons of the effects of rewarding electrical stimulation between SN and VTA, like those between the dorsal (caudate-putamen) and ventral (nucleus accumbens), implicate both the nigrostriatal and the mesolimbic systems in reward function.

Intravenous cocaine reward

Early studies of intravenous cocaine self-administration emphasized the ventral striatum (nucleus accumbens) as a substrate of cocaine reward. First, like the effects of systemic dopamine receptor blockade, dopamine-specific lesions of the ventral striatum were found to disrupt self-administration of intravenous cocaine and amphetamine. This, of course, did not rule out a similar role for the dorsal striatum; in neither case were dorsal striatal lesions tested for comparison. Small non-specific lesions of the dorsal striatum had been shown to affect intravenous morphine reward [22], and recent examination of the effects of small lesions within the dorsal striatum (caudate), reveals decreases in progressive ratio responding for either intravenous cocaine or intravenous heroin [23]. Thus even lesion studies suggest some degree of involvement of the dorsal striatum in drug reward. Early lesion studies also implicated dopaminergic neurons of the VTA in cocaine self-administration, but lesions of the SN have not been tested, perhaps because complete lesions of this structure render animals aphagic, adipsic, and akinetic.

Whereas high systemic doses of neuroleptics cause extinction of intravenous cocaine self-administration—with an initial increase in response rate (“frustration” responding) followed by response cessation—the effect of low and moderate doses of systemic neuroleptics is simply to accelerate responding. The increased intake of cocaine caused by low doses of neuroleptics is interpreted as reflecting decreased rewarding efficacy the drug; the animals appear to take more cocaine, over a wide range of doses per injection, in compensation for competitive dopamine antagonists. Thus they maintain increased concentrations of cocaine in the blood (and increased concentrations of dopamine in the brain) when the dopamine system is partially blocked. This interpretation of response rate increases—reduced rewarding drug impact—is confirmed by progressive ratio studies in which the animal is required to respond progressively more for each successive injection. Here, the measure of drug reward is how long the animal will continue to respond as the “price” of reward escalates. The highest response requirement that the animal meets before ceasing to respond is termed the “break point,” and neuroleptics reduce such break points. Thus despite their inclination to take more cocaine when it is free, neuroleptic-treated animals are willing to pay a lower price for cocaine when they have to work for it [24]. The decrease in break point confirms that cocaine is less rewarding even when animals show the motivation to take more of it.

The same pattern—increased responding for cocaine under fixed ratio schedules contrasted with decreased responding under progressive ratio schedules—is seen when neuroleptics are infused locally into nucleus accumbens [25, 26]. Dorsal striatal dopamine antagonists also increase response rate on fixed ratio schedules, although they do so less effectively and with greater latency than do ventral striatal antagonists [27]. This was suggested to reflect diffusion of the antagonist to the ventral striatum, but it may merely reflect that greater spread within the dorsal striatum is required because of its greater volume or because of weaker sensitivity. Recent studies show that dorsal [28] but not ventral [29] striatal neuroleptic microinjections attenuate cocaine-seeking in animals tested with a second-order schedule, a sensitive schedule that depends heavily on the contribution of conditioned reinforcement by cocaine-associated cues.

While the evidence for dorsal striatal participation in cocaine reward is relatively weak (compare [28] with [27]), stronger evidence comes from comparisons of VTA and SN manipulations. First, like systemic or ventral striatal treatments, microinjections of a D1-type dopamine antagonist into the VTA increase fixed-ratio cocaine self-administration and decrease progressive ratio responding [30]. The D1-type receptors in the VTA are on the terminals of GABAergic and glutamatergic afferents to this region, and because presynaptic control of glutamate release by D1 agonists is an order of magnitude more sensitive than presynaptic control of GABA release [31], the dominant local mechanism of D1 antagonists appears to involve a decrease in excitatory drive to the dopamine neurons. Glutamate input to the VTA activates VTA dopamine neurons as reflected in release of dopamine from dopaminergic dendrites [32], a correlate of dopaminergic cell firing and of dopamine release in nucleus accumbens [33–35]. Blockade of VTA glutamate receptors, like blockade of VTA D1-type receptors, increases cocaine self-administration on fixed-ratio schedules but attenuates responding under extinction conditions, presumably by blocking a glutamatergic contribution of reward-predictive stimuli that serve as conditioned incentive stimuli or conditioned reinforcers [32].

For the thesis of the present paper, the important point is that similar effects on fixed ratio responding are seen when the D1-type antagonist is microinjected into the SN [36]. The same doses of the D1 antagonist are effective in both VTA [30] and SN [36], and the effects of antagonists at the two sites have similar latencies. In the studies cited, the tips of the angled VTA injection cannulae and the vertical SN cannulae were 1.5 mm apart, and control injections 1mm dorsal or rostral to effective sites were ineffective; thus it is very unlikely that the antagonist injected to the SN diffused to a site of action in the VTA. Rather, it appears that by attenuating excitatory input to SN dopamine neurons, antagonism of D1 dopamine receptors in the SN attenuates the rewarding impact of intravenous cocaine and does so in a similar manner to antagonism of D1 dopamine receptors in the VTA.

Cholinergic manipulations of SN and VTA further suggest contributions of both regions to the habit-forming—and thus the addictive—actions of cocaine. Cholinergic projections from the laterodorsal and pedunculopontine tegmental nuclei provide a second source of excitatory input to VTA dopamine neurons [37], and acetylcholine release in the VTA and the SN are each elevated during intravenous cocaine self-administration. This elevation is associated with increased dopaminergic activation (as reflected in local dendritic dopamine release) that is attenuated by local cholinergic antagonists [38]. As with D1 dopamine antagonists, cholinergic antagonists microinjected into either SN or VTA increase fixed ratio responding for cocaine; they each also increase local dendritic dopamine release [38]. Cholinergic antagonists injected into either SN or VTA—like glutamate antagonists injected into the same regions [32]—decrease responding for cocaine-predictive conditioned stimuli under extinction conditions [38]. As with the effects of the D1 dopamine antagonists, the effects of the cholinergic antagonists are assumed to reflect decreased excitatory input to the SN and VTA dopamine neurons. Thus, again, manipulations of SN dopamine cells, like those of VTA dopamine cells, modify the rewarding impact of intravenous cocaine and its associated cues [38].

Reward prediction

If the activation of dopaminergic neurons plays a significant role in reward function, these neurons should be responsive to the presentation of rewarding stimuli. That they are has been well established by electrophysiological studies; neurons presumed to be dopaminergic by their electrophysiological characteristics are excited by unexpected food rewards [39]. When, after many repeated trials, an animal learns that some distal environmental stimulus reliably predicts the presentation of reward, dopaminergic neurons come to respond to the distal predictive stimulus and cease responding to the proximal reward signal itself [39]. The distal signal, in effect, becomes the reward signal. The responses of dopamine neurons to reward-predictive stimuli occur prior to any movements that the distal stimuli evoke and thus do not reflect motor commands. Despite the fact that dopaminergic neurons cease to be excited by predicted rewards as they come to be excited by reward-predictors, their activity remains sensitive to reward events: if a reward is expected but not presented, the dopamine cell that responded to the reward predictor goes briefly silent [39]. While such studies only identify correlates, not causes, of reward function, these characteristics of dopaminergic responsiveness fit remarkably well with what is required of a reinforcement signal. Phasic dopamine activation occurs when there is something to learn (the association between a reward and an earlier predictor); dopaminergic pacemaker firing continues normally when there is nothing to be learned (when a reward occurs that was fully predicted); and phasic inhibition of dopaminergic firing occurs when a reward-predictor fails (and should no longer be reinforced).

For the present thesis, the important finding is that such responses are seen in both SN and VTA dopamine neurons. Despite several kinds of heterogeneity both between SN and VTA dopamine neurons [9, 40–42] and also within SN [43] and VTA [44–46] dopamine groupings, subsets of both SN and VTA dopamine neurons show the characteristic responses to reward and reward-predictive stimuli and to reward omission [39]. Indeed, the responses of SN and VTA neurons to rewards and reward-predictors are so indistinguishable that the electrophysiological data from cells in the two regions are traditionally pooled [39]. SN and VTA dopamine neurons share the same qualifying characteristics for whatever role dopamine neurons play in reward function.

The same can be said for the primary target neurons of the SN and VTA dopamine populations. Target neurons in the primary terminal field of the SN—dorsal striatal medium spiny neurons—and target neurons in the primary terminal field of the VTA—ventral striatal medium spiny neurons—respond similarly to rewards [47] and reward-predictors [48]. Thus, at each end of the both the nigrostriatal and the mesolimbic dopamine systems, significant populations dopamine target-neurons are implicated in the processing of reward and reward-predictor information.

Reinforcement and memory consolidation

Those interested in the elusive mechanism of the essential feature of reward function—reinforcement—have studied it at both the behavioral and the cellular level. The concept of reinforcement was first and most clearly articulated by Thorndike [49], who initially described it as the “stamping-in” of associations and subsequently elaborated it as “the aftereffects of a connection upon it.” In his view of reinforcement as strengthening the connections between neurons that have been recently active, Thorndike anticipated the “Hebb Synapse” that is central to modern notions of the cellular basis of memory. At the cellular level, reinforcement is the stamping-in of functional connections between synaptic inputs and synaptic outputs. In its most fundamental sense, reinforcement refers to any treatment that enhances the permanence or “consolidation” of a memory trace in the nervous system [50]. While reward stimuli do more than simply reinforce learning (they also attract and motivate [51]), the stamping in of memory traces is their core characteristic [49]. In both the rapid time-frame of cellular models of synaptic potentiation and the more prolonged time-frame of consolidation of behavioral learning and memory, dopamine has an important and frequently necessary role. And at each of these levels of analysis, dopamine in the nigrostriatal system and dopamine in the mesolimbic system have each been implicated.

The most elemental demonstration of the stamping-in of memory consolidation is in behavioral paradigms where the reinforcing event is presented after the learning trial and is irrelevant to the task or association that is learned. Using this approach, Krivanek and McGaugh [52] established, for example, that amphetamine injections given after (post-trial) the training session can enhance the subsequent memory for food-rewarded discrimination learning. In this case the reinforcer of memory consolidation (amphetamine) was not only irrelevant but, as an appetite-suppressant drug, seemingly antithetical to the task. Nonetheless, when given after the task amphetamine enhanced consolidation of the association between the food and the food-predictive discriminative stimulus. Similarly, post-trial sucrose [53] or post-trial rewarding brain stimulation [54, 55] can enhance memory consolidation in avoidance tasks where the glucose or stimulation is irrelevant to the task itself. If amphetamine, glucose, or rewarding brain stimulation are given in the hour after a significant learning event, consolidation of the relevant memories will be stronger and the evidence of learning will, when subsequently tested, be more evident.

The post-trial enhancement of memory consolidation by rewarding electrical stimulation is more clearly linked to stimulation of the origins of the nigrostriatal than of the mesocorticolimbic dopamine system [56]. Other post-trial manipulations of dopamine systems implicate both mesolimbic and nigrostriatal dopamine actions in the stamping in of memory consolidation. Post-trial administration of either amphetamine or a selective D2-type dopamine agonist enhances retention of a win-stay radial maze habit when injected into the hippocampus and enhances retention of a win-shift habit when injected into the dorsal striatum [57, 58]. Memory consolidation in a Pavlovian autoshaping task is disrupted by post-trial administration of a D1 antagonist into the ventral striatum [59]. While the consolidation of memory in different kinds of task appears to be influenced differentially by dorsal striatal and limbic manipulations—suggesting that the memory for different associations is laid down in different sub-regions of the dopamine terminal fields—memory function itself is clearly affected by manipulations of both nigro-striatal and mesolimbic projection targets.

At the neuronal level, two forms of modification of synaptic input-output relations serve as cellular models of learning and memory: long-term potentiation (LTP) and long-term depression (LTD) of glutamatergic neurotransmssion. LTP and LTD occur at synapses in widespread regions of the brain, including striatal limbic and cortical targets of the VTA dopamine neurons, such as the nucleus accumbens [60], hippocampus [61], and medial prefrontal cortex [62], as well as in the primary striatal target of the SN dopamine neurons, the caudate-putamen [63, 64]. In most brain regions studied, mammalian LTP and LTD are each known to be dependent on dopaminergic input. Hippocampal LTP is blocked by dopamine D1 antagonists [65–67] and potentiated by D1 agonists [68]. Hippocampal LTD is potentiated by D1 agonists and D2 antagonists and blocked by D1 antagonists and D2 agonists [69]. Striatal LTD is blocked by either D1- or D2-type antagonists and can be restored in dopamine-depleted slice preparations by a combination of D1 and D2 agonists [70]. Striatal LTP is also dopamine-dependent [71], but in this case D1 and D2 receptor activations are antagonistic [72, 73]. LTP and LTD are also dopamine-dependent in amygdala [74], frontal cortex [62], and even the VTA [75, 76] and SN [77]. Perhaps surprisingly, while LTP and LTD have each been demonstrated at glutamatergic synapses in nucleus accumbens, dopamine appears to play an important role only in LTD in this structure [75, 78]. Thus dopamine appears to play important roles in the reinforcement of synaptic transmission in several but perhaps not all projections of the mesencephalic dopamine system. That the details of the roles played differ somewhat from site to site should perhaps not be surprising in light of the fact that, while the same intracellular events appear to be critical for memory consolidation in different structures, they do not necessarily occur in the same sequence in different structures [79].

A particularly interesting demonstration of striatal LTP involved the use of rewarding electrical stimulation at the level of mesencephalic dopamine neurons. Here rats were trained to lever-press for rewarding brain stimulation (confirming that the electrodes were in the right place), and then, after extinction of the response habit, stimulation with the same parameters was shown to potentiate subsequent corticostriatal glutamatergic neurotransmission. In this case the rewarding electrical stimulation sites were in the SN, and the effectiveness of the stimulation to potentiate corticostriatal transmission was correlated with the effectiveness of the stimulation to serve as a reward in the independent intracranial self-stimulation task [80]. Here a reinforcer involving the nigrostriatal system and established in the behavioral domain was found to influence memory consolidation in the cellular domain, and the memory that was consolidated was the memory of GABAergic medium spiny neurons for glutamatergic cortical input signals.

Summary and conclusions

While the present suggestion is that nigrostriatal and mesolimbic dopamine systems play common roles in reward function, this is not to imply that they are specialized for reward function or that they play a necessary role in reward. Dopamine-deficient mice are still capable of rudimentary learning. Forebrain dopamine release is caused not only by reward stimuli but also by stress and aversive stimuli. Near-total depletions of the forebrain dopamine cause aphagia, adipsia, and akinesia; thus basal dopamine levels play an essential role in all behavior and particularly in sensorimotor responsiveness to environmental stimuli. In addition to rewarding effects, phasic increases in dopamine have drive-like motivational effects that enhance of the salience of reward-predictive “incentive-motivational” stimuli in the environment, leading to reward-seeking behaviors. Conversely, competitive dopamine antagonists not only attenuate the effectiveness of rewards as reinforcers; they also attenuate an animal’s motivation or apparent willingness to exert effort. Even the importance of dopamine for memory consolidation appears not to depend on a single mechanism. LTP and LTD depend closely on the timing of synaptic inputs and thus probably depend on burst firing in dopamine neurons and actions of dopamine within the synapse [81]; post-trial memory consolidation in behavioral studies occurs over tens of minutes or hours and thus probably depends on extrasynaptic dopamine, perhaps more influenced by pacemaker firing of dopamine neurons and release from extra-synaptic varicosities along terminal axons [82]. Actions of dopamine at D1 and D2 receptors have different roles in LTP and LTD, and these roles can differ from structure to structure [83].

Thus dopamine participates in more than just reward or reinforcement function, and within reward and reinforcement function it contributes in multiple ways. However, it is not just mesocorticolimbic dopamine that is important for reward function; nigrostriatal dopamine is similarly important. Nigrostriatal and mesolimbic dopamine do not play equal or identical roles; they appear to contribute differentially to the learning of, for example, habit, spatial, and emotional learning [84]. In part, this must be due to differences in the information received in the different dopaminergic terminal fields; even within the striatum, different sub-regions receive inputs from different parts of the cerebral cortex [4]. While the number and nature of the roles dopamine plays remain to be fully understood, the habit-forming and habit-maintaining effects of intravenous cocaine—the properties that lead the animal from experimental cocaine self-administration to compulsive cocaine self-administration and addiction—appear to depend on both mesolimbic and nigrostriatal dopamine function.

Box 1. Reward versus Reinforcement.

What is reward and how does it differ from reinforcement? The term reward has many meanings, and they differ from writer to writer [51, 85, 86]. Reward is the more general term, referring as a noun to the stimulus or event that is rewarding, and referring as a verb to the consequence of presenting a rewarding stimulus in the right temporal relation with a response or another stimulus. When present in the environment or when presented prior to a given behavior, rewards also serve as incentives, attracting attention and approach responses. They can also serve a “priming” function, as when the eating of a salted peanut triggers the motivation for more salted peanuts. Priming effects of a reward are short-lived and not stored in long-term memory, but they can influence the probability and vigor of the next response in a series when animals are responding rapidly or when they are responding for slowly decaying rewards like addictive drugs [87, 88]. That rewards have incentive and priming effects as well as reinforcing (see below) effects make reward an ambiguous term when taken out of context. The term reward is used as a noun when we refer to the object or event. It is also used as a verb, both in common parlance and also by some specialists (such as brain stimulation reward specialists) who are sensitive to the potential confounding of priming effects with reinforcing effects under conditions where they cannot specify the relative contributions of the two [13, 51].

The term “reinforcement” is unambiguous. It refers only to the seemingly retroactive “stamping in” of recent associations between stimuli, stimuli and responses, and responses and outcomes that occurs when the reinforcer is given after the stimulus or response in question. Reinforcing actions of a reward depend on this temporal sequence. To reinforce the association between a conditioned and an unconditioned stimulus, the reinforcer (the unconditioned stimulus in this case) is most effective when given a fraction of a second after the conditioned stimulus so that it is stimulus-contingent; to reinforce a stimulus-response or a response-outcome association, the reinforcer (the reward) is also most effective when given very shortly after the response, so that it is response-contingent. Even a very short delay of reinforcement can reduce reinforcing effectiveness dramatically. The term “reinforcement,” then, is much more narrowly defined than the term “reward.” Often, unfortunately, the terms reward and reinforcement are used interchangeably, presumably by authors who are not concerned with, or who are careless with, these distinctions. For the purposes of the present paper, “reward” is used as a verb in situations where the presentation of a stimulus or event might have proactive “priming” effects as well as retroactive “reinforcing” effects. “Reinforcement” is used to refer selectively to the stamping in of learned associations with prior stimuli or responses.

Box 2. Motivation.

While it is not the primary topic of the present paper, motivation and reward are interdependent functions and dopamine is important for each [13]. Motivation is a state variable, an inferred variable that precedes, instigates, and invigorates goal-directed behavior [89]. The prototypical motivational states are thirst, hunger, and hormonal states such as those that orchestrate courtship behaviors. Motivation comes in two forms: drive and incentive motivation. Thirst and hunger are considered drive states, assumed to reflect the state of the organism and easily manipulated by water or food deprivation. Hormonal states are traditionally assumed to be drive states as well, but their status is ambiguous inasmuch as they can be triggered by environmental events [90]. Motivational states that are triggered by environmental stimuli are termed incentive-motivational states, a term that is usually used in the context of conditioned motivational or incentive stimuli. An incentive stimulus is a stimulus the animal tends to approach or work for; thus the term “incentive stimulus” is equivalent to the term “reward” when “reward” is used as a noun. A conditioned incentive motivational stimulus is a reward-predictor. Reward-predictive stimuli instigate and invigorate goal-directed behavior, and inasmuch reward-prediction depends on past learning, incentive motivation depends on past reinforcement. To the degree that past reinforcement depends on brain dopamine, incentive motivation is also dopamine-dependent. However, while dopamine is important for establishing incentive motivation (and while dopamine can enhance incentive motivation), incentive motivation (as reflected, for example in prolonged extinction responding) can survive for long periods without the need for any phasic dopamine contribution [13].

Box 3. Brain dopamine and reward: caveats.

It is widely accepted that brain dopamine plays a major role in reward function. However, there continue to be challenges to this widely held view, as discussed in detail elsewhere [13]. For those who do not follow this literature in detail, several caveats should be briefly mentioned. First, brain dopamine is not necessary for reward function; some degree of reward learning can be demonstrated in dopamine-deficient mice [91]. Second, brain dopamine is involved in more than merely the response to rewards. Dopamine systems are activated in response to aversive and arousing stimuli, as well as to stimuli we normally think of as rewarding (positive valence) stimuli [92]. Dopamine release can precipitate [93] or enhance [94] as well as reinforce responding, serving as a drive or priming stimulus in addition to its stamping in function. Third, while I once advanced the view that dopamine is important for the pleasure that is frequently associated with reinforcement, this is a side issue with respect to the theme of the present paper. Finally, consistent with the central issue of the present paper, while reward function is importantly dependent on dopamine function, it is not necessarily dependent on dopamine function in nucleus accumbens [13].