THE MYSTERIOUS MOTIVATIONAL FUNCTIONS OF MESOLIMBIC DOPAMINE

Summary

Nucleus accumbens dopamine is known to play a role in motivational processes, and dysfunctions of mesolimbic dopamine may contribute to motivational symptoms of depression and other disorders, as well as features of substance abuse. Although it has become traditional to label dopamine neurons as “reward” neurons, this is an over-generalization, and it is important to distinguish between aspects of motivation that are differentially affected by dopaminergic manipulations. For example, accumbens dopamine does not mediate primary food motivation or appetite, but is involved in appetitive and aversive motivational processes including behavioral activation, exertion of effort, approach behavior, sustained task engagement, Pavlovian processes and instrumental learning. In this review, we discuss the complex roles of dopamine in behavioral functions related to motivation.

Nucleus accumbens dopamine (DA) has been implicated in several behavioral functions related to motivation. Yet, the specifics of this involvement are complex, and at times can be difficult to disentangle. An important consideration in interpreting these findings is the ability to distinguish between diverse aspects of motivational function that are differentially affected by dopaminergic manipulations. Although ventral tegmental neurons have traditionally been labeled “reward” neurons, and mesolimbic DA referred to as the “reward” system, this vague generalization is not matched by the specific findings that have been observed. The scientific meaning of the term “reward” is unclear, and its relation to concepts such as reinforcement and motivation is often ill defined. Pharmacological and DA depletion studies demonstrate that mesolimbic DA is critical for some aspects of motivational function, but of little or no importance for others. Some of the motivational functions of mesolimbic DA represent areas of overlap between aspects of motivation and features of motor control, which is consistent with the well known involvement of nucleus accumbens in locomotion and related processes. Furthermore, despite an enormous literature linking mesolimbic DA to aspects of aversive motivation and learning, a literature which goes back several decades (e.g., Salamone 1994), the established tendency has been to emphasize dopaminergic involvement in reward, pleasure, addiction, and reward-related learning, with less consideration of the involvement of mesolimbic DA in aversive processes. The present review will discuss the involvement of mesolimbic DA in diverse aspects of motivation, with an emphasis on experiments that interfere with DA transmission, particularly in nucleus accumbens.

Mesolimbic DA and Motivation: The Changing Theoretical Landscape

If nothing else, humans are inveterate story tellers; we are, after all, the descendants of people who sat around the fire at night being regaled by vivid myths, tales, and oral histories. Human memory is more efficacious if random facts or events can be woven into the meaningful tapestry of a coherent story. Scientists are no different. An effective university lecture, or a scientific seminar, is often referred to as “a good story”. So it is with scientific hypotheses and theories. Our brain seems to crave the order and coherence of thought offered by a simple and clear scientific hypothesis, backed up by just enough evidence to make it plausible. The problem is-- what if the coherence of the story is being enhanced by over interpreting some findings, and ignoring others? Gradually, the pieces of the puzzle that do not fit continue to eat away at the whole, eventually rendering the entire story woefully inadequate.

One can argue that this kind of evolution has taken place with regards to the DA hypothesis of “reward”. A “story” could be constructed, which would proceed as follows: the main symptom of depression is anhedonia, and since DA is a “reward transmitter” that mediates hedonic reactions, then depression is due to a reduction of DA-regulated experience of pleasure. Likewise, it has been suggested that drug addiction depends upon the experience of pleasure induced by drugs that hijack the brain’s “reward system”, which is mediated by DA transmission, and evolved to convey the pleasure produced by natural stimuli such as food. This would even suggest that blocking DA receptors could offer a readily effective treatment for addiction. Finally, one could also offer a “story” constructed on the premise that DA neurons exclusively respond to pleasurable stimuli such as food, and that this activity mediates the emotional response to these stimuli, which in turn underlies the appetite for food consumption. Such stories are not “straw men” that are artificially constructed for these passages. But unfortunately, despite their popularity, none of these ideas is fully supported by a close examination of the literature.

To take the example of dopaminergic involvement in depression, one could begin to deconstruct this idea by pointing out that “anhedonia” in depression is often misinterpreted or mislabeled by clinicians (Treadway and Zald, 2011). Several studies show that depressed people often have a relatively normal self-rated experience of encounters with pleasurable stimuli, and that, over and above any problems with the experience of pleasure, depressed people appear to have impairments in behavioral activation, reward seeking behavior and exertion of effort (Treadway and Zald, 2011). Indeed, most depressed people suffer from a crippling constellation of motivational impairments that include psychomotor retardation, anergia and fatigue (Demyttenaere et al., 2005; Salamone et al., 2006), and considerable evidence implicates DA in these symptoms (Salamone et al., 2006, 2007). These observations, coupled with the literature indicating that there is not a simple correspondence between DA activity and hedonic experience (e.g. Smith et al. 2011), and the studies linking DA to behavioral activation and exertion of effort (Salamone et al. 2007; see discussion below), lead one to conclude that dopaminergic involvement in depression seems to be more complicated than the simple story would have allowed.

Similarly, it is clear that a substantial body of research on drug dependence and addiction does not comply with the traditional tenets of the DA hypothesis of reward. Several studies have shown that blockade of DA receptors or inhibition of DA synthesis does not consistently blunt the self-reported euphoria or “high” induced by drugs of abuse (Gawin 1986; Brauer and DeWit, 1998; Nann-Vernotica et al. 2001; Wachtel et al. 2002; Leyton et al. 2005; Venugopalan et al. 2011). Recent research has identified individual differences in behavioral patterns shown by rats during Pavlovian approach conditioning, which are related to the propensity to self-administer drugs. Rats that show greater response to conditioned cues (sign-trackers) display different patterns of dopaminergic adaptation to training as compared to animals that are more responsive to the primary reinforcer (goal trackers; Flagel et al., 2007). Interestingly, the rats that show greater Pavlovian conditioned approach to an appetitive stimulus, and show greater incentive conditioning to drug cues, also tend to show greater fear in response to cues predicting shock, and greater contextual fear conditioning (Morrow et al., 2011). Additional research has challenged some long held views about the neural mechanisms underlying addiction, as opposed to the initial reinforcing characteristics of drugs. It has become more common to view addiction in terms of neostriatal habit-formation mechanisms built upon extensive drug taking, which can be relatively independent of instrumental reinforcement contingencies or the initial motivational characteristics of drug reinforcers (Kalivas, 2008; Belin et al., 2009). These emerging views about the neural basis of drug addiction, and its potential treatment, have moved well beyond the original story offered by the DA hypothesis of “reward”.

After decades of research, and continuing theoretical developments, there has been a substantial conceptual restructuring in the field of DA research. Considerable evidence indicates that interference with mesolimbic DA transmission leaves fundamental aspects of the motivational and hedonic response to food intact (Berridge 2007; Salamone et al. 2007). Behavioral measures such as progressive ratio break points and self-stimulation thresholds, which were once thought to be useful as markers of the “reward” or “hedonia” functions of DA, are now considered to reflect processes involving exertion of effort, perception of effort-related or opportunity costs, and decision making (Salamone, 2006; Hernandez et al., 2010). Several recent electrophysiology papers have demonstrated responsiveness of either presumed or identified ventral tegmental DA neurons to aversive stimuli (Anstrom and Woodward, 2005; Brischoux et al., 2009; Matsumoto and Hikosaka, 2009; Bromberg-Martin et al., 2010; Schulz, 2010; Lammel et al., 2011). Many investigators now emphasize the involvement of mesolimbic and nigrostriatal DA in reinforcement learning or habit formation (Wise 2004; Yin et al., 2008; Belin et al. 2009), rather than hedonia per se. These trends have all contributed to a dramatic rewriting of the story of dopaminergic involvement in motivation.

Motivational Processes: Historical and Conceptual Background

The term motivation refers to a construct that is widely used in psychology, psychiatry, and neuroscience. As is the case with many psychological concepts, the discussion of motivation had its origins in philosophy. In describing causal factors that control behavior, the German philosopher Schopenhauer (1999) discussed the concept of motivation in relation to the way that organisms must be in a position to “choose, seize, and even seek out the means of satisfaction”. Motivation also was a vital area of interest during the initial development of psychology. Early scientific psychologists, including Wundt and James, included motivation as a subject in their textbooks. Neobehaviorists such as Hull and Spence frequently employed motivational concepts such as incentive and drive. Young (1961) defined motivation as “the process of arousing actions, sustaining the activity in progress, and regulating the pattern of activity”. According to a more recent definition, motivation is “the set of processes through which organisms regulate the probability, proximity and availability of stimuli” (Salamone, 1992). Generally speaking, the modern psychological construct of motivation refers to the behaviorally-relevant processes that enable organisms to regulate both their external and internal environment (Salamone, 2010).

Perhaps the main utility of the construct of motivation is that it provides a convenient summary and organizational structure for observable features of behavior (Salamone, 2010). Behavior is directed towards or away from particular stimuli, as well as activities that involve interacting with those stimuli. Organisms seek access to some stimulus conditions (i.e., food, water, sex) and avoid others (i.e., pain, discomfort), in both active and passive ways. Moreover, motivated behavior typically takes place in phases (Figure 1). The terminal stage of motivated behavior, which reflects the direct interaction with the goal stimulus, is commonly referred to as the consummatory phase. The word “consummatory” (Craig, 1918), does not refer to “consumption”, but instead to “consummation”, which means “to complete” or “to finish”. In view of the fact that motivational stimuli usually are available at some physical or psychological distance from the organism, the only way to gain access to these stimuli is to engage in behavior that brings them closer, or makes their occurrence more likely. This phase of motivated behavior often is referred to as “appetitive”, “preparatory”, “instrumental”, “approach”, or “seeking”. Thus, researchers sometimes distinguish between “taking” vs. “seeking” of a natural stimulus such as food (e.g. Foltin et al. 2001), or of a drug reinforcer; indeed, the term “drug seeking behavior” has become a common phrase in the language of psychopharmacology. As discussed below, this set of distinctions (e.g. instrumental vs. consummatory or seeking vs. taking) is important for understanding the effects of dopaminergic manipulations on motivation for natural stimuli such as food.

Figure 1.

For several years, researchers have been making distinctions between aspects of motivated behavior, many of which are differentially affected by dopaminergic manipulations. Motivated behavior takes places in phases, in which the animal first must approach or seek the reinforcing goal stimulus (e.g. appetitive, instrumental, approach, preparatory or seeking behavior). Eventually, the organism gains access to the motivational stimulus, and directly interacts with it (consummatory or taking behavior). In addition, the distinction between activational (vigor, persistence, stimulation of sustained activity) and directional (i.e., behavior is directed towards or away from a particular stimulus) aspects of motivation has been made in the behavioral literature for many years. More recently, Berridge and colleagues have emphasized the distinction between liking (i.e., the hedonic reaction to the stimulus) and wanting (the desire for the stimulus, the tendency to consume or pursue the stimulus). These distinctions are highly relevant for characterizing the effects of DA antagonists and accumbens DA depletions on motivated behavior; several papers indicate that DA antagonism and accumbens DA depletions have a greater effect on appetitive, instrumental, preparatory or seeking behavior, as well as behavioral activation and “wanting”, while having less effect on consummatory behavior, directional aspects of motivation, and “liking”.

In addition to “directional” aspects of motivation (i.e., that behavior is directed towards or away from stimuli), motivated behavior also is said to have “activational” aspects (Cofer and Appley, 1964; Salamone 1988, 2010; Parkinson et al., 2002; Figure 1). Because organisms are usually separated from motivational stimuli by a long distance, or by various obstacles or response costs, engaging in instrumental behavior often involves work (e.g. foraging, maze running, lever pressing). Animals must allocate considerable resources towards stimulus-seeking behavior, which therefore can be characterized by substantial effort, i.e., speed, persistence, and high levels of work output. Although the exertion of this effort can at times be relatively brief (e.g., a predator pouncing upon its prey), under many circumstances it must be sustained over long periods of time. Effort-related capabilities are highly adaptive, because in the natural environment survival can depend upon the extent to which an organism overcomes time- or work-related response costs. For these reasons, behavioral activation has been considered a fundamental aspect of motivation for several decades. Psychologists have long used the concepts of drive and incentive to emphasize the energizing effects of motivational conditions on measures of instrumental behavior, such as run speed in a maze. Cofer and Apley (1964) suggested that there was an anticipation-invigoration mechanism that could be activated by conditioned stimuli, and which functioned to invigorate instrumental behavior. Scheduled non-contingent presentation of primary motivational stimuli such as food reinforcement pellets can induce various activities, including drinking, locomotion and wheel-running (Robbins and Koob, 1980; Salamone 1988). Several researchers have studied the impact of work requirements on the performance of instrumental tasks, which ultimately helped to lay the groundwork for the development of economic models of operant behavior (e.g. Hursh, 1988). Ethologists also have employed similar concepts. Foraging animals need to expend energy to gain access to food, water, or nesting material, and optimal foraging theory describes how the amount of effort or time expended to obtain these stimuli is an important determinant of choice behavior.

There is a considerable degree of conceptual overlap between motor control processes and activational aspects of motivation. For example, food deprivation can accelerate run speed in a maze. Does this reflect conditions that are motivational, motoric, or some combination of the two? Locomotor activity clearly is under the control of neural systems that regulate movement. Nevertheless, locomotor activity in rodents also is very sensitive to the impact of motivational conditions such as novelty, food deprivation, or periodic presentation of small food pellets. In addition, if an organism is presented with a work-related challenge during instrumental performance, it often responds to that challenge by exerting greater effort. Increasing ratio requirements on operant schedules, up to a point, can create substantial upward pressures on response rates. Facing an obstacle, such as a barrier in a maze, can lead rodents to increase their exertion of effort and jump over the barrier. Furthermore, presentation of a Pavlovian conditioned stimulus associated with a primary motivational stimulus such as food can serve to instigate approach or amplify instrumental activity, an effect known as Pavlovian to instrumental transfer (Colwill and Rescorla, 1988). Thus, the neural systems that regulate motor output appear to operate at the behest of those neural systems that direct behavior towards or away from particular stimuli (Salamone, 2010). Of course, the terms “motor control” and “motivation” do not mean precisely the same thing, and one can easily find points of non-overlap. Nevertheless, it is evident that there is a fundamental overlap as well (Salamone, 1992, 2010). In light of this observation, it is informative to consider that the English words motivation and movement both are ultimately derived from the Latin word movere, to move (i.e., moti is the past participle of movere). As with the distinction between instrumental vs. consummatory behavior (or seeking vs. taking), differentiation between activational vs. directional aspects of motivation is widely used to describe the effects of dopaminergic manipulations (Figure 1). The diverse nature of motivational processes is an important feature of the literature discussing the behavioral effects of dopaminergic manipulations, as well as that focusing on the dynamic activity of mesolimbic DA neurons.

Dissociative Nature of the Effects of Interfering with Nucleus Accumbens DA Transmission

In trying to understand the literature on the motivational functions of accumbens DA, we should consider several of the conceptual principles highlighted above. On the one hand, we should recognize that motivational processes are dissociable into component parts, and that manipulations of accumbens DA transmission are sometimes able to cleave these components like the application of a diamond cutter, substantially altering some while leaving others largely unaffected (Salamone and Correa, 2002; Berridge and Robinson, 2003; Smith et al., 2011). On the other hand, we also must realize that motivational processes interact with mechanisms related to emotion, learning, and other functions, and that there is not a precise point-to-point mapping between behavioral processes and neural systems. Thus, some of the effects of dopaminergic manipulations may be most effectively understood in terms of actions on specific aspects of motivation, motor function or learning, while other effects may be more squarely in areas of overlap between these functions. Finally, one also should consider that it is highly unlikely that accumbens DA performs only one very specific function; it is difficult to conceive of a complex machine like the mammalian brain operating in such a simple manner. Thus, accumbens DA probably performs several functions, and any particular behavioral or neuroscience method may be well suited for characterizing some of these functions, but poorly suited for others. In view of this, it can be challenging to assemble a coherent view.

Brain manipulations can alter subcomponents of a behavioral process in a highly specific manner. This principle has been very useful in cognitive neuroscience, and has led to important distinctions in terms of dissociable memory processes (i.e., declarative vs. procedural memory, working vs. reference memory, hippocampal-dependent vs. independent processes). In contrast, the tendency in much of the literature discussing the behavioral functions of accumbens DA has instead been to use rather blunt conceptual instruments, i.e., very general and vague terms such as “reward”, to summarize the actions of drugs or other manipulations. Indeed, the term “reward” has been criticized in detail elsewhere (Cannon and Bseikri, 2004; Salamone, 2006; Yin et al., 2008; Salamone et al., 2012). Though the term reward has meaning as a synonym for “reinforcer”, there is no consistent scientific meaning of “reward” when used to describe a neurobehavioral process; some employ it as a synonym for “reinforcement”, while others use it to mean “primary motivation” or “appetite”, or as a thinly disguised synonym for “pleasure” or “hedonia” (for an historical overview of the “anhedonia hypothesis”, see Wise, 2008). In many cases, the word “reward” seems to be used as a general term that refers to all aspects of appetitive learning, motivation and emotion, including both conditioned and unconditioned aspects; this usage is so broad as to be essentially meaningless. One can argue that the overuse of the term “reward” is a source of tremendous confusion in this area. While one article may use reward to mean pleasure, another may employ the term to refer to reinforcement learning but not pleasure, and a third may be referring to appetitive motivation in a very general way. These are three very different meanings of the word, which obfuscates the discussion of the behavioral functions of mesolimbic DA. Moreover, labeling mesolimbic DA as a “reward system” serves to downplay its role in aversive motivation. Perhaps the biggest problem with the term “reward” is that it evokes the concept of pleasure or hedonia in many readers, even if this is unintended by the author.

The present review is focused upon the involvement of accumbens DA in features of motivation for natural reinforcers such as food. In general, there is little doubt that accumbens DA is involved in some aspects of food motivation; but which aspects? As we shall see below, the effects of interference with accumbens DA transmission are highly selective or dissociative in nature, impairing some aspects of motivation while leaving others intact. The remainder of this section will focus on the results of experiments in which dopaminergic drugs or neurotoxic agents are used to alter behavioral function.

Although it is generally recognized that forebrain DA depletions can impair eating, this effect is closely linked to depletions or antagonism of DA in the sensorimotor or motor-related areas of lateral or ventrolateral neostriatum, but not nucleus accumbens (Dunnett and Iversen, 1982; Salamone et al., 1993). A recent optogenetics study showed that stimulating ventral tegmental GABA neurons, which results in the inhibition of DA neurons, acted to suppress food intake (Van Essen et al., 2012). However, it is not clear if this effect is specifically due to dopaminergic actions, or if it is dependent upon aversive effects that also are produced with this manipulation (Tan et al., 2012). In fact, accumbens DA depletion and antagonism have been shown repeatedly not to substantially impair food intake (Ungerstedt, 1971; Koob et al., 1978; Salamone et al., 1993; Baldo et al., 2002; Baldo and Kelley, 2007). Based upon their findings that injections of D₁ or D₂ family antagonists into accumbens core or shell impaired motor activity, but did not suppress food intake, Baldo et al., (2002) stated that accumbens DA antagonism “did not abolish the primary motivation to eat” (p 176). Accumbens DA depletions failed to reduce food intake or feeding rate, and did not impair food handling, although similar depletions of ventrolateral neostriatum did affect these measures (Salamone et al., 1993). In addition, the effects of DA antagonists or accumbens DA depletions on food-reinforced instrumental behavior do not closely resemble the effects of appetite suppressant drugs (Salamone et al., 2002; Sink et al., 2008), or the reinforcer devaluation provided by prefeeding (Salamone et al., 1991; Aberman and Salamone 1999; Pardo et al. 2012). Lex and Hauber (2010) demonstrated that rats with accumbens DA depletions were sensitive to devaluation of food reinforcement during an instrumental task. Furthermore, Wassum et al. (2011) showed that the DA antagonist flupenthixol did not affect the palatability of food reward or the increase in reward palatability induced by the upshift in motivational state produced by increased food deprivation.

Considerable evidence also indicates that nucleus accumbens DA does not directly mediate hedonic reactivity to food. An enormous body of work from Berridge and colleagues has demonstrated that systemic administration of DA antagonists, as well DA depletions in whole forebrain or nucleus accumbens, do not blunt appetitive taste reactivity for food, which is a widely accepted measure of hedonic reactivity to sweet solutions (Berridge and Robinson, 1998, 2003; Berridge 2007). Moreover, knockdown of the DA transporter (Pescina et al., 2002), as well as microinjections of amphetamine into nucleus accumbens (Smith et al., 2011), which both elevate extracellular DA, failed to enhance appetitive taste reactivity for sucrose. Sederholm et al. (2002) reported that D2 receptors in the nucleus accumbens shell regulate aversive taste reactivity, and that brainstem D2 receptor stimulation suppressed sucrose consumption, but neither population of receptors mediated the hedonic display of taste.

If nucleus accumbens DA does not mediate appetite for food per se, or food-induced hedonic reactions, then what is its involvement in food motivation? There is considerable agreement that accumbens DA depletions or antagonism leave core aspects of food-induced hedonia, appetite, or primary food motivation intact, but nevertheless affect critical features of the instrumental (i.e., food seeking) behavior (Figures 1 and 2). Investigators have suggested that nucleus accumbens DA is particularly important for behavioral activation (Koob et al., 1978; Robbins and Koob, 1980; Salamone 1988, Salamone et al. 1991, 1992, 2005, 2007; Calaminus and Hauber, 2007; Lex and Hauber, 2010), exertion of effort during instrumental behavior (Salamone et al., 1994, 2007, 2012; Mai et al., 2012), Pavlovian to instrumental transfer (Parkinson et al. 2002; Everitt and Robbins, 2005; Lex and Hauber 2008), flexible approach behavior (Nicola, 2010), energy expenditure and regulation (Salamone, 1987; Beeler et al., 2012), and exploitation of reward learning (Beeler et al., 2010). Accumbens DA depletions and antagonism reduce spontaneous and novelty-induced locomotor activity and rearing, as well as stimulant-induced activity (Koob et al., 1978; Cousins et al., 1993; Baldo et al., 2002). Activities such as excessive drinking, wheel-running, or locomotor activity that are induced by periodic presentation of food pellets to food-deprived animals are reduced by accumbens DA depletions (Robbins and Koob, 1980; McCullough and Salamone, 1992). In addition, low doses of DA antagonists, as well as accumbens DA antagonism or depletions, reduce food-reinforced responding on some tasks despite the fact that food intake is preserved under those conditions (Salamone et al., 1991, 2002; Ikemoto and Panksepp, 1996; Koch et al., 2000). The effects of accumbens DA depletions on food-reinforced behavior vary greatly depending upon the task requirements or reinforcement schedule. If the primary effects of accumbens DA depletions were related to a reduction in appetite for food, then one would expect that the fixed ratio 1 (FR1) schedule should be highly sensitive to this manipulation. Nevertheless, this schedule is relatively insensitive to the effects of compromised DA transmission in accumbens (Aberman and Salamone, 1999; Salamone et al., 2007; Nicola 2010). One of the critical factors yielding sensitivity to the effects of accumbens DA depletions on food reinforced behavior is the size of the ratio requirement (i.e., number of lever presses required per reinforcer; Aberman and Salamone, 1999; Mingote et al., 2005). In addition, blockade of accumbens DA receptors impairs performance of instrumental approach instigated by presentation of cues (Wakabayashi et al., 2004; Nicola, 2010).

Figure 2.

This figure emphasizes the dependence of some aspects of appetitive and aversive instrumental (i.e., seeking) behavior on nucleus accumbens DA transmission. Salamone (1991) noted that highly active instrumental behaviors elicited and supported by conditioned stimuli are very sensitive to disruption of accumbens DA transmission. Koob et al. (1978) reported that neurotoxic depletions of accumbens DA decreased behavioral activation, but actually tended to increase food consumption. Nicola (2010) emphasized the importance of accumbens DA for flexible approach to the reinforcing stimulus.

The ability of DA antagonists or accumbens DA depletions to dissociate between food consumption and food-reinforced instrumental behavior, or between different instrumental tasks, is not some trivial detail or epiphenomenal result. Rather, it demonstrates that under conditions in which food-reinforced instrumental behavior can be disrupted, fundamental aspects of food motivation are nevertheless intact. A number of investigators who have written about the fundamental characteristics of reinforcing stimuli have concluded that stimuli acting as positive reinforcers tend to be relatively preferred, or to elicit approach, goal directed, or consummatory behavior, or generate a high degree of demand, and that these effects are a fundamental aspect of positive reinforcement (Dickinson and Balleine, 1994; Salamone and Correa, 2002; Salamone et al., 2012). As stated in the behavioral economic analysis offered by Hursh (1993; p 166) “responding is regarded as a secondary dependent variable that is important because it is instrumental in controlling consumption”. Thus, the results described above demonstrate that low doses of DA antagonists and accumbens DA depletions do not impair fundamental aspects of primary or unconditioned food motivation and reinforcement, but do make animals sensitive to some features of the instrumental response requirement, blunt responsiveness to conditioned cues, and reduce the tendency of the animals to work for food reinforcement.

One of the manifestations of the dissociative nature of the behavioral effects of low systemic doses of DA antagonists, and depletion or antagonism of accumbens DA, is that these conditions affect the relative allocation of behavior in animals responding on tasks that assess effort-based decision making (Salamone et al., 2007; Floresco et al., 2008; Mai et al., 2012). One task that has been used to assess the effects of dopaminergic manipulations on response allocation offers rats a choice between lever pressing reinforced by delivery of a relatively preferred food, versus approaching and consuming a concurrently available but less preferred food (Salamone et al., 1991, 2007). Under baseline or control conditions, trained rats get most of their food by lever pressing, and consume small quantities of chow. Low-to-moderate doses of DA antagonists that block either D1 or D2 family receptor subtypes produce a substantial alteration of response allocation in rats performing on this task, decreasing food-reinforced lever pressing but substantially increasing chow intake (Salamone et al., 1991; Koch et al., 2000; Sink et al., 2008). This task has been validated in several experiments. Doses of DA antagonists that produce the shift from lever pressing to chow intake do not affect total food intake or alter preference between these two specific foods in free-feeding choice tests (Salamone et al., 1991; Koch et al., 2000). In contrast, appetite suppressants from different classes, including fenfluramine and cannabinoid CB1 antagonists (Salamone et al., 2007; Sink et al., 2008), failed to increase chow intake at doses that suppressed lever pressing. In contrast to the effects of DA antagonism, pre-feeding, which is a type of reinforcer devaluation, reduced both lever pressing and chow intake (Salamone et al., 1991). These results indicate that interference with DA transmission does not simply reduce primary food motivation or intake, but instead alters response allocation between alternative sources of food that are obtained through different responses. These behavioral effects are dependent upon accumbens DA, and are produced by accumbens DA depletions and local infusions of D1 or D2 family antagonists into accumbens core or shell (Salamone et al., 1991; Koch et al., 2000; Nowend et al., 2001; Farrar et al., 2010; Mai et al., 2012).

A T-maze procedure also has been developed to study effort-related choice. For this task, the two choice arms of the maze lead to different reinforcement densities (e.g. 4 vs. 2 food pellets, or 4 vs. 0), and under some conditions, a barrier is placed in the arm with the higher density of food reinforcement to impose an effort-related challenge (Salamone et al., 1994). When the high density arm has the barrier in place, and the arm without the barrier contains fewer reinforcers, accumbens DA depletions or antagonism decrease choice of the high cost/high reward arm, and increase selection of the low cost/low reward arm (Salamone et al., 1994; Denk et al., 2005; Pardo et al., 2012; Mai et al., 2012). When there was no barrier in the maze, rodents prefered the high reinforcement density arm, and neither DA receptor antagonism nor accumbens DA depletion altered their choice (Salamone et al., 1994). When the arm with the barrier contained 4 pellets, but the other arm contained no pellets, rats with accumbens DA depletions still chose the high density arm, climbed the barrier, and consumed the pellets. In a recent T-maze study with mice, while haloperidol reduced choice of the arm with the barrier, this drug had no effect on choice when both arms had a barrier in place (Pardo et al., 2012). Thus, dopaminergic manipulations did not alter the preference based upon reinforcement magnitude, and did not affect discrimination, memory or instrumental learning processes related to arm preference. Bardgett et al. (2009) developed a T-maze effort discounting task, in which the amount of food in the high density arm of the maze was diminished each trial on which the rats selected that arm. Effort discounting was altered by administration of D1 and D2 family antagonists, which made it more likely that rats would choose the low reinforcement/low cost arm. Increasing DA transmission by administration of amphetamine blocked the effects of SCH23390 and haloperidol, and also biased rats towards choosing the high reinforcement/high cost arm, which is consistent with operant choice studies using DA transporter knockdown mice (Cagniard et al., 2006).

One of the important issues in this area is the extent to which animals with impaired DA transmission are sensitive to the work requirements in effort-related tasks, or to other factors such as time delays (e.g. Denk et al., 2005; Wanat et al., 2010). Overall, the effects of DA antagonism on delay discounting have proven to be rather mixed (Wade et al. 2000; Koffarnus et al., 2011), and Winstanley et al. (2005) reported that accumbens DA depletions did not affect delay discounting. Floresco et al. (2008) demonstrated that the DA antagonist haloperidol altered effort discounting even when they controlled for the effects of the drug on response to delays. Wakabayashi et al. (2004) found that blockade of nucleus accumbens D1 or D2 receptors did not impair performance on a progressive interval schedule, which involves waiting for longer and longer time intervals in order to receive reinforcement. Furthermore, studies with tandem schedules of reinforcement that have ratio requirements attached to time interval requirements indicate that accumbens DA depletions make animals more sensitive to added ratio requirements, but do not make animals sensitive to time interval requirements from 30-120 sec (Correa et al., 2002; Mingote et al., 2005).

In summary, the results of the T-maze and operant choice studies in rodents support the idea that low doses of DA antagonists and accumbens DA depletions leave fundamental aspects of primary motivation and reinforcement intact, but nevertheless reduce behavioral activation and cause animals to reallocate their instrumental response selection based upon the response requirements of the task, and select lower cost alternatives for obtaining reinforcers (Salamone et al., 2007, 2012). Considerable evidence indicates that mesolimbic DA is part of a broader circuitry regulating behavioral activation and effort-related functions, which includes other transmitters (adenosine, GABA; Mingote et al., 2008; Farrar et al., 2008, 2010; Nunes et al., 2010; Salamone et al., 2012) and brain areas (basolateral amygdala, anterior cingulate cortex, ventral pallidum; Walton et al., 2003; Floresco and Ghods-Sharifi, 2007; Mingote et al., 2008; Farrar et al., 2008; Hauber and Sommer, 2009).

Involvement of Mesolimbic DA in Appetitive Motivation: Dynamic Activity of DA Systems

Although it is sometimes said that nucleus accumbens DA release or the activity of ventral tegmental DA neurons is instigated by presentation of positive reinforcers such as food, the literature describing the response of mesolimbic DA to appetitive stimuli is actually quite complicated (Hauber, 2010). In a general sense, does food presentation increase DA neuron activity or accumbens DA release? Across a broad range of conditions, and through different phases of motivated behavior, which phases or aspects of motivation are closely linked to the instigation of dopaminergic activity? The answer to these questions depends upon the timescale of measurement, and the specific behavioral conditions being studied. Fluctuations in DA activity can take place over multiple timescales, and a distinction often is made between “phasic” and “tonic” activity (Grace, 2000; Floresco et al., 2003; Goto and Grace, 2005). Electrophysiological recording techniques are capable of measuring fast phasic activity of putative DA neurons (e.g. Schultz, 2010), and voltammetry methods (e.g. fast cyclic voltammetry) record DA “transients” that are fast phasic changes in extracellular DA, which are thought to represent the release from bursts of DA neuron activity (e.g., Roitman et al., 2004; Sombers et al., 2009; Brown et al., 2011). It also has been suggested that fast phasic changes in DA release can be relatively independent of DA neuron firing, and can instead reflect synchronized firing of cholinergic striatal interneurons that promote DA release through a presynaptic nicotinic receptor mechanism (Rice et al., 2011; Threlfell et al., 2012; Surmier and Graybiel, 2012). Microdialysis methods, on the other hand, measure extracellular DA in a way that represents the net effect of release and uptake mechanisms integrated over larger units of time and space relative to electrophysiology or voltammetry (e.g. Hauber, 2010). Thus, it is often suggested that microdialysis methods measure “tonic” DA levels. Nevertheless, in view of the fact that microdialysis can measure behavior- or drug- related fluctuations (e.g., increases followed by decreases) in extracellular DA that take place over minutes, it is perhaps most useful to employ the term “fast phasic” to talk about the rapid changes in DA-related activity that can be measured with electrophysiology or voltammetery, and “slow phasic” in reference to the changes that take place over the slower time scale measured with microdialysis methods (e.g. Hauber, 2010; Segovia et al., 2011).

Electrophysiology studies have shown that presentation of novel or unexpected food reinforcers is accompanied by transient increases in the activity of putative ventral tegmental DA neurons, but that this effect goes away with regular presentation, or repeated exposure through training (Schultz et al., 1993; Schultz, 2010). Employing voltammetry methods to measure fast phasic changes in DA release, Roitman et al. (2004) showed that, in trained animals, exposure to a conditioned stimulus signaling that lever pressing would result in sucrose delivery was accompanied by an increase in DA transients, however, the actual presentation of the sucrose reinforcer was not. A similar finding was reported years ago by Nishino et al. (1987), who studied free-operant fixed ratio lever pressing in monkeys, and observed that activity of putative ventral tegmental DA neurons was increased during lever pressing in trained animals, but actually decreased during reinforcer presentation. Unpredicted food delivery, as well as presentation of cues that predicted food delivery, increased fast phasic signaling as measured by voltammetry in the nucleus accumbens core (Brown et al., 2011). DiChiara and colleagues showed that exposure to novel palatable foods transiently increased extracellular DA in nucleus accumbens shell as measured by microdialysis, but that this response rapidly habituated (e.g. Bassareo et al., 2002). A recent microdialysis paper demonstrated that presentation of high carbohydrate food reinforcers to previously exposed rats did not produce any change in extracellular DA in accumbens core or shell (Segovia et al., 2011). In contrast, both the acquisition and maintenance of fixed ratio lever pressing was associated with increases in DA release (Segovia et al., 2011). A similar pattern was shown when markers of DA-related signal transduction (c-Fos and DARPP-32) were measured (Segovia et al., 2012). Taken together, these studies do not support the idea that food presentation per se, including that of palatable foods, uniformly increases accumbens DA release across a broad range of conditions.

Nevertheless, considerable evidence does indicate that increases in DA transmission are associated with presentation of stimuli associated with natural reinforcers such as food, or the performance of instrumental behavior; this has been seen in studies involving microdialysis (Sokolowski et al., 1998; Ostlund et al., 2010; Hauber, 2010; Segovia et al. 2011), voltammetry (Roitman et al., 2004; Brown et al., 2011; Cacciapaglia et al., 2011), and electrophysiological recordings during free operant responding (Nishino et al., 1987; Kosobud et al., 1994). Cacciapaglia et al. (2011) reported that fast phasic DA release in nucleus accumbens as measured by voltammetry occurred during onset of a cue that signaled reinforcer availability, as well as lever press responding, and that the excitatory effects of this phasic release on accumbens neurons were blunted by inactivation of burst firing in ventral tegmental DA neurons. Furthermore, a substantial body of electrophysiology research has identified some of the conditions that activate burst firing in putative ventral tegmental DA neurons, including presentation of stimuli that are associated with the primary reinforcer, as well as conditions that have a higher reinforcement value relative to the expectation generated by previous experience (Schultz et al., 1997). The later observation has led to the hypothesis that DA neuron activity could represent the kind of prediction error signal described by some models of learning (e.g. Rescorla and Wagner, 1972). This pattern of activity in putative DA neurons has provided a formal theoretical basis for the involvement of fast phasic DA signaling in reinforcement learning models (Schultz et al., 1997; Bayer and Glimcher, 2005; Niv, 2009; Schultz, 2010).

Although the primary focus of the present paper is on the effects of dopaminergic manipulations on distinct aspects of motivation, it is useful to consider the importance of fast phasic and slow phasic (i.e., “tonic”) signaling for interpreting the effects of conditions that interfere with DA transmission. The different timescales of dopaminergic activity could serve very different functions, and therefore, the effects of a particular manipulation could very much depend upon whether it is altering fast or slow phasic activity, or baseline levels of DA. Researchers have used various pharmacological or genetic manipulations to differentially affect fast phasic DA activity vs. DA release on slower timesecales (Zweifel et al., 2009; Parker et al., 2011; Grieder et al., 2012), and have reported that these manipulations can exert distinct behavioral effects. For example, Grieder et al. (2012) showed that selective interference with phasic DA activity prevented the expression of conditioned place aversions to withdrawal from a single acute dose of nicotine, but not to withdrawal from chronic nicotine. In contrast, blockade of D2 receptors impaired the expression of conditioned aversion during chronic, but not acute withdrawal. Zweifel et al. (2009) reported that selective genetic inactivation of NMDA receptors, which blunted burst firing in VTA DA neurons, impaired the acquisition of cue dependent appetitive learning but did not disrupt the behavior of working for food reinforcement on a progressive ratio schedule. In fact, a number of DA-related behavioral functions are preserved in animals with impaired fast phasic DA activity (Zweifel et al., 2009; Wall et al., 2011; Parker et al., 2011). These observations have implications for integrating information from studies of fast phasic activity with those that focus on the effects of DA antagonism or depletion. First of all, they suggest that one must be cautious in generalizing from concepts generated in studies of electrophysiology or voltammetry (e.g., that DA release acts as a “teaching signal”) to the behavioral functions that are impaired when drugs or DA depletions are used to disrupt DA transmission. Furthermore, they indicate that studies of fast phasic activity of mesolimbic DA neurons may explicate the conditions that rapidly increase or decrease DA activity or provide a discrete DA signal, but do not strictly inform us as to the broad array of functions performed by DA transmission across multiple timescales, or those impaired by disruption of DA transmission.

Involvement of Mesolimbic and Neostriatal Mechanisms in Appetitive Instrumental Learning

Although one can define motivation in terms that make it distinct from other constructs, it should be recognized that, in fully discussing either the behavioral characteristics or neural basis of motivation, one also should consider related functions. The brain does not have box-and- arrow diagrams or demarcations that neatly separate core psychological functions into discrete, non-overlapping neural systems. Thus, it is important to understand the relation between motivational processes and other functions such as homeostasis, allostasis, emotion, cognition, learning, reinforcement, sensation, and motor function (Salamone, 2010). For example, Panksepp (2011) emphasized how emotional networks in the brain are intricately interwoven with motivational systems involved in processes such as seeking, rage or panic. In addition, seeking/instrumental behavior is not only influenced by the emotional or motivational properties of stimuli, but also, of course, learning processes. Animals learn to engage in specific instrumental responses that are associated with particular reinforcing outcomes. As a critical part of the associative structure of instrumental conditioning, organisms must learn which actions lead to which stimuli (i.e., action-outcome associations). Thus, motivational functions are intertwined with motor, cognitive, emotional, and other functions (Mogenson et al., 1980). Though the present review is focused upon the involvement of mesolimbic DA in motivation for natural reinforcers, it also is useful to have a brief discussion of the putative involvement of mesolimbic DA in instrumental learning.

One could think that it would be relatively straightforward to demonstrate that nucleus accumbens DA mediates reinforcement learning, or is critically involved in the synaptic plasticity processes underlying the association of an operant response with delivery of a reinforcer (i.e., action-outcome associations). But this area of research is as difficult and complicated to interpret as the motivational research reviewed above. For example, Smith-Roe and Kelley (2000) showed that simultaneous blockade of DA D1 and NMDA receptors in nucleus accumbens core retarded the acquisition of instrumental lever pressing. In addition, post-session manipulations that affect memory consolidation also affected the acquisition of instrumental lever pressing (Hernandez et al., 2002). Nevertheless, in reviewing the literature on nucleus accumbens and instrumental learning, Yin et al. (2008) concluded that “the accumbens is neither necessary nor sufficient for instrumental learning” (p 1439). Similarly, Belin et al. (2009) noted that lesion and drug manipulations of the nucleus accumbens core can affect the acquisition of instrumental behavior reinforced by natural stimuli, but stated that the “precise psychological contributions” of the accumbens and other brain structures remain unclear (p 91). Although there are many studies showing that cell body lesions, DA antagonists, or DA depletions can affect the learning related outcomes in procedures such as place preference, acquisition of lever pressing, or other procedures, this does not in itself demonstrate that nucleus accumbens neurons or mesolimbic DA transmission are essential for the specific associations that underlie instrumental learning (Yin et al., 2008). Specific effects related to instrumental learning can be demonstrated by assessments of the effects of reinforcer devaluation or contingency degradation, which often are not conducted in pharmacology or lesion studies. With this in mind, it is important to note that cell body lesions in either core or shell of the accumbens did not alter sensitivity to contingency degradation (Corbit et al., 2001). Lex and Hauber (2010) found that rats with nucleus accumbens DA depletions were still sensitive to reinforcer devaluation, and suggested that accumbens core DA might therefore not be crucial for encoding action-outcome associations. Although it is unclear if accumbens DA is critical for associations between the response and the reinforcer, considerable evidence indicates that nucleus accumbens DA is important for Pavlovian approach and Pavlovian to instrumental transfer (Parkinson et al., 2002; Wyvell and Berridge, 2000; Dalley et al., 2005; Lex and Hauber, 2008, 2010; Yin et al., 2008). Such effects could provide a mechanisms by which conditioned stimuli can exert activating effects upon instrumental responding (Robbins and Everitt, 2007; Salamone et al., 2007), as discussed above. The activating or arousing effects of conditioned stimuli can be a factor in amplifying an already acquired instrumental response, but also could act to promote acquisition by increasing response output and the variability of behavior, thereby setting the occasion for more opportunities to pair a response with reinforcement. A recent paper showed that optogenetic stimulation of ventral tegmental DA neurons did not provide positive reinforcement of instrumental lever pressing on its own, and did not affect food intake, but did amplify the emergence of food-reinforced lever pressing on an active lever during acquisition, and enhance output of previously extinguished instrumental responses (Adamantidis et al., 2011).

Interestingly, even though knockout of DA D1 receptors blunted the acquisition of Pavlovian approach behavior, knockout of NMDA receptors, which resulted in a 3-fold decrease in the fast phasic DA release instigated by presentation of food-associated cues, did not retard the acquisition of Pavlovian approach behavior (Parker et al., 2011). This indicates that the relation between fast phasic DA release and learning remains uncertain. Future studies should examine the effects of manipulations that affect fast phasic DA signaling using procedures that directly assess reinforcement learning (i.e., reinforcer devaluation and contingency degradations). Moreover, genetic and pharmacological methods that lead to the suppression of fast phasic DA activity should be assessed further for their actions on behavioral activation and effort related aspects of motivation.

Involvement of Mesolimbic DA in Aversive Motivation and Learning: Dynamic Activity of DA Systems

A cursory review of some articles in the DA literature could leave one with the impression that mesolimbic DA is selectively involved in hedonic processes, appetitive motivation and reinforcement-related learning, to the exclusion of aversive aspects of learning and motivation. However, such a view would be at variance with the literature. As described above, considerable evidence indicates that accumbens DA transmission does not directly mediate hedonic reactions to stimuli. Moreover, there is a very large literature indicating that mesolimbic DA is involved in aversive motivation, and can affect behavior in aversive learning procedures. A number of different aversive conditions (e.g. shock, tail pinch, restraint stress, aversive conditioned stimuli, aversive drugs, social defeat) can increase DA release as measured by microdialysis methods (McCullough et al., 1993; Salamone, 1994; Tidey and Miczek, 1996; Young, 2004). For many years, it was thought that ventral tegmental DA neuron activity was not increased by aversive stimuli, however, recent studies have demonstrated that the electrophysiological activity of putative or identified DA neurons is increased by aversive or stressful conditions (Anstrom and Woodward, 2005; Brischoux et al., 2009; Matsumoto and Hikosaka, 2009; Bromberg-Martin et al., 2010; Schulz, 2010; Lammel et al., 2011). Although Roitman et al. (2008) reported that an aversive taste stimulus (quinine) decreased DA transients in nucleus accumbens, Anstrom et al. (2009) observed that social defeat stress was accompanied by increases in fast phasic DA activity as measured by both electrophysiology and voltammetry. Uncertainty remains about whether there are separate DA neurons that respond differentially to appetitive and aversive stimuli, and what proportion of neurons respond to each, but there seems to be little doubt that mesolimbic DA activity can be enhanced by at least some aversive conditions, and therefore is not specifically tied to hedonia or positive reinforcement.

A substantial body of evidence going back several decades (Salamone, 1994) and continuing to the recent literature (Faure et al., 2008; Zweifel et al., 2011) demonstrates that interference with DA transmission can impair the acquisition or performance of aversively motivated behavior. In fact, for many years, DA antagonists underwent preclinical screening for antipsychotic activity based partly upon their ability to blunt avoidance behavior (Salamone, 1994). Accumbens DA depletions impair shock avoidance lever pressing (McCullough et al., 1993). Systemic or intra-accumbens injections of DA antagonists also disrupt the acquisition of place aversion and taste aversion (Acquas and Di Chiara, 1994; Fenu et al., 2001), as well as fear conditioning (Inoue et al., 2000; Pezze and Feldon, 2004). Zweifel et al., (2011) reported that knockout of NMDA receptors, which acts to reduce fast phasic DA release, impaired the acquisition of cue-dependent fear conditioning.

Human studies also have demonstrated a role for ventral striatum in aspects of aversive motivation and learning. War veterans with post-traumatic stress disorder showed increased blood flow in ventral striatum/nucleus accumbens in response to the presentation of aversive stimuli (i.e., combat sounds; Liberzon et al. 1999). Human imaging studies indicate that ventral striatal BOLD responses, as measured by fMRI, are increased in response to prediction errors regardless of whether the stimulus predicted rewarding or aversive events (Jensen et al., 2007), and that aversive prediction errors were blocked by the DA antagonist haloperidol (Menon et al., 2007). Baliki et al. (2010) reported that in normal subjects, phasic BOLD responses occurred both to the onset and the offset of a painful thermal stimulus. Delgado et al. (2011) demonstrated that ventral striatal BOLD responses were increased during aversive conditioning to a primary aversive stimulus (shock) as well as monetary loss. A PET study that obtained measurements of in vivo raclopride displacement to assess DA release in humans reported that exposure to psychosocial stress increased markers of extracellular DA in the ventral striatum in a manner that was correlated with increased cortisol release (Pruessner et al., 2004). Thus, human imaging studies also show that ventral striatum and its mesolimbic DA innervation is responsive to aversive as well as appetitive stimuli.

Summary and Conclusions

In summary, traditional ideas about DA as a mediator of “hedonia”, and the tendency to equate DA transmission with “reward” (and “reward” with “hedonia”), is giving way to an emphasis on dopaminergic involvement in specific aspects of motivation and learning-related processes (Figure 3), including behavioral activation, exertion of effort, cue instigated approach, event prediction, and Pavlovian processes. DA transmission in nucleus accumbens does not exert a powerful influence over the hedonic reactivity to tastes, nor does it appear to mediate primary food motivation or appetite (Berridge and Robinson, 1998; Salamone and Correa, 2002; Kelley et al., 2005; Barbano et al., 2009). Moreover, though dopaminergic manipulations can affect behavioral outcomes in animals trained on learning tasks, there is not strong evidence that accumbens DA is critical for the specific aspect of instrumental learning that involves the association between the instrumental action and the reinforcing outcome (Yin et al., 2008). Nevertheless, accumbens DA clearly is important for aspects of appetitive as well as aversive motivation (Salamone et al., 2007; Cabib and Puglisi-Allegra, 2012), and participates in learning processes, at least in part through processes that involve Pavlovian approach and Pavlovian to instrumental transfer (Yin et al., 2008; Belin et al., 2009). Interference with accumbens DA transmission blunts the acquisition of Pavlovian approach responses that are instigated by cues that predict food delivery, and impairs avoidance responses elicited by cues that predict aversive stimuli. Accumbens DA depletions or antagonism reduce the activating effects of conditioned stimuli, and make animals very sensitive to work-related instrumental response costs (e.g. output of ratio schedules with large ratio requirements, barrier climbing; Salamone et al. 2007, 2012; Barbano et al., 2009). Thus, nucleus accumbens DA is clearly involved in the aspects of motivation, and the regulation of goal-directed actions, but in a rather specific and complex way that is not conveyed by the simple word “reward”. Some instrumental tasks tap into the functions subserved by mesolimbic DA (e.g., activational aspects of motivation, exertion of effort), and thus impairment of mesolimbic DA readily affects performance on these tasks, while responding on other positively reinforced tasks, or measures of primary food motivation, are left intact.

Figure 3.

This figure illustrates that there are multiple dimensions that are relevant for understanding the impact of DAergic manipulations on motivated behavior. Interference with accumbens DA transmission is very likely to disrupt vigorous or effortful instrumental behaviors that are instigated or supported by conditioned stimuli. In contrast, consummatory behaviors such as food intake, which involves direct interaction with a primary motivational stimulus, as well as aversive behaviors induced by a primary aversive stimulus (e.g., escape), tend to be less easily disrupted by DAergic manipulations (see references in text). Although these factors are depicted as distinct dimensions, they also interact. For example, instrumental behaviors are generally instigated by conditioned stimuli, and conditioned stimuli also have activating properties.

In the last few years, the picture that has emerged is that neostriatum (i.e., dorsal striatum) and its DA innervation appears to have a clearer link to the processing of instrumental associations than does the nucleus accumbens (Yin et al., 2008). Lesions of the dorsomedial neostriatum made animals insensitive to both reinforcer devaluation and contingency degradation (Yin et al., 2005). Both cell body lesions and DA depletions in dorsolateral striatum have been shown to impair habit formation (Yin et al., 2004; Faure et al., 2005). The involvement of neostriatum in habit formation could be related to the hypothesized role of the basal ganglia in promoting the development of action sequences or “chunking” of components of instrumental behavior (Graybiel, 1998; Matsumoto et al., 1999). The idea that there is a transition from ventral striatal regulation of instrumental responding to neostriatal mechanisms that regulate habit formation has been employed extensively to provide an explanation of several features of drug addiction (see review by Belin et al., 2009), and also is relevant for understanding the effects of natural reinforcers (Segovia et al., 2012). However, in this context, it is useful to emphasize that the involvement of nucleus accumbens DA in aspects of instrumental learning or performance, or the involvement of neostriatal DA in regulating the encoding of action-outcome associations or habit formation, does not mean that these effects are mediated by actions on primary motivation or appetite for natural reinforcers such as food. For example, Smith-Roe and Kelley (2000) showed that combined injection of a D1 antagonist and an NMDA antagonist at doses that impaired acquisition of food-reinforced lever pressing did not affect food intake, and interpreted this result as demonstrating a lack of a general motivational effect of this manipulation. Moreover, interference with DA transmission in dorsolateral neostriatum was shown to impair habit formation, but leave goal-directed (i.e., motivationally driven) responding intact (Faure et al., 2005). Thus, the involvement of neostriatal DA in habit formation does not provide evidence for the dopaminergic mediation of primary food motivation or appetite. In fact, food intake is most greatly affected by DA depletions in ventrolateral neostriatum, and these impairments are related to motoric dysfunctions affecting feeding rate and forepaw usage during feeding, and occur in parallel with the induction of oral tremor that has the characteristics of Parkinsonian resting tremor (Jicha and Salamone, 1991; Salamone et al., 1993; Collins-Praino et al., 2011).

Although it is not a simple marker of hedonia or primary food motivation and appetite, DA in nucleus accumbens does appear to regulate multiple channels of information passing through this nucleus, and thus participates in a variety of behavioral processes related to aspects of motivation. For decades, researchers have suggested that basal ganglia structures act as regulators of sensorimotor function, which does not mean that interference with the basal ganglia produces a simple paralysis or motor incapacity, but instead refers to the idea that these structures, including the accumbens, participate in the gating (i.e., the thresholding) of the impact of sensory input on behavioral output. Similarly, Mogenson et al. (1980) and colleagues suggested years ago that nucleus accumbens acts as a “limbic-motor” interface, providing a link between limbic areas involved in emotion and cognition and neural circuits regulating behavioral output. Considerable evidence from multiple sources indicates that nucleus accumbens acts as a gate, a filter, or an amplifier, of information passing through from various cortical or limbic areas on its way to various motor areas of the brain (e.g., Roesch et al., 2009). Electrophysiological and voltammetry studies indicate that nucleus accumbens is organized into ensembles and microcircuits of task-specific neurons that are modulated by DA (O’Donnell, 2003; Carelli and Wodonoski, 2003; Cacciapaglia et al., 2011). Roesch et al. (2009) reported that nucleus accumbens neurons integrate information about the value of an expected reward with features of the motor output (i.e., response speed or choice) that occur during decision making. DA release may set a threshold for worthwhile cost expenditures, and under some circumstances may provide an opportunistic drive for exploitation of resources (Fields et al., 2007; Gan et al., 2010; Beeler et al., 2012). This suggestion is consistent with the proposed involvement of accumbens DA in the behavioral economics of instrumental behavior, particularly in terms of cost/benefit decision making (Salamone et al., 2007, 2009).

As stated above, organisms typically are separated from primary motivational stimuli or goals by obstacles or constraints. Another way of saying this is that the process of engaging in motivated behavior requires that organisms overcome the “psychological distance” between themselves and motivationally relevant stimuli. The concept of psychological distance is an old idea in psychology (e.g., Lewin, 1935; Shepard, 1957; Liberman and Forster, 2008) and has taken on many different theoretical connotations in different areas of psychology (e.g. experimental, social, personality, etc.). In the present context, it is simply used as a general reference to the idea that objects or events are often not directly present or experienced, and therefore organisms are separated along multiple dimensions (e.g. physical distance, time, probability, instrumental requirements) from these objects or events. In various ways, mesolimbic DA serves as a bridge that enables animals to traverse the psychological distance that separates them from goal objects or events. Multiple investigators have phrased this in diverse ways or emphasized different aspects of the process (Everitt and Robbins, 2005; Kelley et al., 2005; Salamone et al., 2005, 2007, 2009; Phillips et al., 2007; Nicola, 2010; Lex and Hauber, 2010; Panksepp, 2011; Beeler et al., 2012; see Figure 3), but many of the functions in which accumbens DA has been implicated, including behavioral activation, exertion of effort during instrumental behavior, Pavlovian to instrumental transfer, responsiveness to conditioned stimuli, event prediction, flexible approach behavior, seeking, and energy expenditure and regulation, are all important for facilitating the ability of animals to overcome obstacles and, in a sense, transcend psychological distance. Overall, nucleus accumbens DA is important for performing active instrumental responses that are elicited or maintained by conditioned stimuli (Salamone 1992), for maintaining effort in instrumental responding over time in the absence of primary reinforcement (Salamone et al., 2001; Salamone and Correa, 2002), and for regulating the allocation of behavioral resources by setting constraints on the instrumental responses that are selected for procuring reinforcement based upon cost/benefit analyses (Salamone et al., 2007, 2012; Hernandez et al., 2010).

Translational and Clinical Implications

In parallel with the animal research reviewed above, experimental and clinical studies with humans also have begun to elucidate some of the motivational functions of ventral and dorsal striatal DA, and point towards their potential clinical significance. This emerging research on humans, using imaging as well as pharmacological methods, has generated results consistent with the idea that striatal systems in general, and DA in particular, are involved in aspects of instrumental behavior, anticipation of reinforcement, behavioral activation and effort-related processes. Knutson et al. (2001) reported that accumbens fMRI activation was evident in people performing a gambling task, but that the increased activity was associated with reward prediction or anticipation rather than the actual presentation of the monetary reward. O’Doherty et al. (2002) observed that anticipation of glucose delivery was associated with increased fMRI activation in midbrain and striatal DA areas, but that these areas did not respond to glucose delivery. Recent imaging studies have implicated ventral striatum in cost/benefit decision making (Croxson et al., 2009; Botvinick et al., 2009; Kurniawan et al., 2011). Treadway et al. (2012) found that individual differences in exertion of effort in humans were associated with an imaging marker of striatal DA transmission. In addition, Wardle et al. (2011) showed that amphetamine enhanced willingness of people to exert effort to obtain rewards, particularly when reward probability was low, but did not alter the effects of reward magnitude on willingness to exert effort. A recent imaging paper showed that doses of L-DOPA that enhanced the striatal representation of appetitively motivated actions did not affect the neural representation of reinforcement value (Guitart-Masip et al., 2012). Another recent report described the ability of catecholamine manipulations to dissociate between different aspects of motivation and emotion in humans (Venugopalan et al., 2011). In this study, access to cigarette smoking was used as the reinforcer, and the investigators manipulated DA transmission by transiently inhibiting catecholamine synthesis with phenylalanine/tyrosine depletion. Inhibition of catecholamine synthesis did not blunt self-reported craving for cigarettes, or smoking-induced hedonic responses. Nevertheless, it did lower progressive ratio break points for cigarette reinforcement, indicating that people with reduced DA synthesis showed a reduced willingness to work for cigarettes. Furthermore, imaging research has demonstrated that the human nucleus accumbens/ventral striatum is not only responsive to appetivite stimuli, but also responds to stress, aversion and hyperarousal/irritability (Liberzon et al., 1999; Pavic, 2003; Phan et al., 2004; Pruessner et al., 2004; Levita et al., 2009; Delgado et al., 2011). Taken together, these studies suggest that there are many similarities between findings generated from animal models and those obtained from human research, in terms of many of the motivational functions of mesostriatal DA systems.

As concepts about DA continue to evolve, research on the behavioral functions of DA will have profound implications for clinical investigations of motivational dysfunctions seen in people with depression, schizophrenia, substance abuse and other disorders. In humans, pathological aspects of behavioral activation processes have considerable clinical significance. Fatigue, apathy, anergia (i.e., self-reported lack of energy) and psychomotor retardation are common symptoms of depression (Marin et al., 1993; Stahl, 2002; Demyttenaere et al., 2005; Salamone et al., 2006), and similar motivational symptoms also can be present in other psychiatric or neurological disorders such as schizophrenia (i.e., “avolition”), stimulant withdrawal (Volkow et al., 2001), Parkinsonism (Friedman et al., 2007; Shore et al., 2011), multiple sclerosis (Lapierre and Hum, 2008), and infectious or inflammatory disease (Danzer et al., 2009; Miller, 2009). Considerable evidence from both animal and human studies indicates that mesolimbic and striatal DA is involved in these pathological aspects of motivation (Schmidt et al., 2001; Volkow et al., 2001; Salamone et al., 2006, 2007, 2012; Miller, 2009; Treadway and Zald, 2011). A recent trend in mental health research has been to reduce the emphasis on traditional diagnostic categories, and instead focus on the neural circuits mediating specific pathological symptoms (i.e., the Research Domain Criteria approach; Morris and Cuthbert, 2012). It is possible that continued research on the motivational functions of DA will shed light on the neural circuits underlying some of the motivational symptoms in psychopathology, and will promote the development of novel treatments for these symptoms that are useful across multiple disorders.