Dissecting components of reward: ‘liking’, ‘wanting’, and learning

Abstract

In recent years significant progress has been made delineating the psychological components of reward and their underlying neural mechanisms. Here we briefly highlight findings on three dissociable psychological components of reward: ‘liking’ (hedonic impact), ‘wanting’ (incentive salience), and learning (predictive associations and cognitions). A better understanding of the components of reward, and their neurobiological substrates, may help in devising improved treatments for disorders of mood and motivation, ranging from depression to eating disorders, drug addiction, and related compulsive pursuits of rewards.

Introduction

Liking

For most people a ‘reward’ is something desired because it produces a conscious experience of pleasure — and thus the term may be used to refer to the psychological and neurobiological events that produce subjective pleasure. But evidence suggests that subjective pleasure is but one component of reward, and that rewards may influence behavior even in the absence of being consciously aware of them. Indeed, introspection can actually sometimes lead to confusion about the extent to which rewards are liked, whereas immediate reactions may be more accurate [1]. In the extreme, even unconscious or implicit ‘liking’ reactions to hedonic stimuli can be measured in behavior or physiology without conscious feelings of pleasure (e.g. after a subliminally brief display of a happy facial expression or a very low dose of intravenous cocaine) [2,3]. Thus, though perhaps surprising, objective measures of ‘liking’ reactions to rewards may sometimes provide more direct access to hedonic systems than subjective reports.

A major goal for affective neuroscience is to identify which brain substrates cause pleasure, whether subjective or objective. Neuroimaging and neural recording studies of have found that rewards ranging from sweet taste to intravenous cocaine, winning money or a smiling face activate many brain structures, including orbitofrontal cortex, anterior cingulate and insula, and subcortical structures such as nucleus accumbens, ventral pallidum, ventral tegmentum, and mesolimbic dopamine projections, amygdala, etc. [4•,5,6,7••,8,9•,10•,11–13]. But which of those brain systems actually cause the pleasure of the reward? And which activations instead are merely correlates (e.g. because of spreading network activation) or consequences of pleasure (mediating instead other cognitive, motivational, motor, etc. functions related to the reward)? We and others have searched for pleasure causation in animal studies by identifying brain manipulations that amplify hedonic impact [6,14••,15,16,17•,18–22].

To study neural systems responsible for the hedonic impact of rewards, we and others have exploited objective ‘liking’ reactions to sweet taste rewards, such as affective facial expressions of newborn human infants and the homologous facial reactions of orangutans, chimpanzees, monkeys, and even rats and mice [4•,18,23,24]. Sweets elicit positive facial ‘liking’ expressions in all of these (lip licking, rhythmic tongue protrusions, etc.), whereas bitter tastes instead elicit negative ‘disliking’ expressions (gapes, etc.; Figure 1; Supplemental movie 1). Such ‘liking’–‘disliking’ reactions to taste are controlled by a hierarchy of brain systems for hedonic impact in the forebrain and brainstem, and are influenced by many factors that alter pleasantness, such as hunger/satiety and learned taste preferences or aversions.

Figure 1.

Example behavioral ‘liking’ reactions and brain hedonic hotspots for a sensory pleasure. Top: Positive hedonic ‘liking’ reactions are elicited by sucrose taste from human infant and adult rat (e.g. rhythmic tongue protrusion). By contrast, negative aversive ‘disliking’ reactions are elicited by bitter quinine taste. Below: Forebrain hedonic hotspots in nucleus accumbens shell and in ventral pallidum where mu opioid agonist microinjections cause amplification of ‘liking’ reactions to sweetness. Red/yellow indicates greatest amplification of ‘liking’ for the sensory pleasure. Modified based on data from [14••,17•,28].

Only a few neurochemical systems have been found so far to enhance ‘liking’ reactions to a sweet taste in rats, and only within a few circumscribed brain locations. Opioid, endocannabinoid, and GABA-benzodiazepine neurotransmitter systems are important for generating pleasurable reactions [14••,15,16,17•,25,26], particularly at specific sites in limbic structures (Figure 1 and Figure 2) [15,16,17•,21,27]. We have called these sites ‘hedonic hotspots’ because they are capable of generating increases in ‘liking’ reactions, and by inference, pleasure. One hedonic hotspot for opioid enhancement of sensory pleasure is located in the nucleus accumbens within the rostrodorsal quadrant of its medial shell, about a cubic millimeter in volume [14••,15,28]. That is, the hotspot comprises only 30% of medial shell volume, and less than 10% of the entire nucleus accumbens. Within that hedonic hotspot, microinjection of the mu opioid agonist, DAMGO, doubles or triples the number of ‘liking’ reactions elicited by sucrose taste [14••,28]. Another hedonic hotspot is found in the posterior half of the ventral pallidum, where again DAMGO potently increases ‘liking’ reactions to sweetness [17•,21,28]. In both hotspots, the same microinjection also doubles ‘wanting’ for food in the sense of stimulating eating behavior and food intake.

Figure 2.

Expansion of mu opioid hotspot in nucleus accumbens with delineation of ‘liking’ versus ‘wanting’ zones. Green: the entire medial shell mediates opioid-stimulated increases in ‘wanting’ for food reward. Orange-red: circumscribed cubic-millimeter sized hedonic hotspot generates increases in ‘liking’ after the same opioid stimulation. Blue: a small hedonic ‘coldspot’ suppresses ‘liking’ reactions to sucrose, whereas a larger purple zone suppresses ‘disliking’ reactions to quinine. Reprinted with permission from [27], based on data from [14••].

Outside of those hotspots, even in the same structure, opioid stimulations produce very different effects. For example, in NAc at virtually all other locations DAMGO microinjections still stimulate ‘wanting’ for food as much as in the hotspot, but do not enhance ‘liking’ (and even suppress ‘liking’ in a more posterior coldspot in the medial shell while still stimulating food intake; Figure 2). Thus, comparing the effects of mu opioid activity in or outside the hotspot in NAc medial shell indicates that opioid sites responsible for ‘liking’ are anatomically dissociable from those that influence ‘wanting’ [14••,16].

Endocannabinoids enhance ‘liking’ reactions in a NAc hotspot that overlaps the mu opioid site [16,27]. Microinjection of anandamide in the endocannabinoid hotspot, acting perhaps by stimulating CB1 receptors there, more than doubles the level of ‘liking’ reactions to sucrose taste (and more than doubles food intake). This hedonic endocannabinoid substrate may relate to medication effects of endocannabinoid antagonists when used as potential treatments for obesity or addiction [16,29,30].

The ventral pallidum is a chief target for nucleus accumbens outputs, and its posterior half contains a second opioid hotspot [17•,21]. In the pallidum hotspot, microinjections of DAMGO double ‘liking’ for sucrose and ‘wanting’ for food (measured as intake). By contrast, microinjection of DAMGO anterior to the hotspot suppresses ‘liking’ and ‘wanting’. Quite independently, ‘wanting’ is stimulated separately at all locations in ventral pallidum by blockade of GABA_A receptors via bicuculline microinjection, without altering ‘liking’ at any location [17•,31].

The role of ventral pallidum in ‘liking’ and ‘wanting’ makes it of special interest for studies of neural activation induced by reward. In humans, cocaine, sex, food, or money rewards all activate the ventral pallidum, including the posterior subregion that corresponds to the hedonic hotspot in rats [9•,10•,11,21]. In more detailed electrophysiological studies of how neurons in the posterior ventral pallidum encode hedonic signals in rats, we have found that hotspot neurons fire more vigorously to the sweet taste of sucrose than to an unpleasant salty taste (triple the concentration of seawater) [7••]. However, by itself a difference in evoked firing between sucrose and salt does not prove that the neurons encode their relative hedonic impact (‘liking’ versus ‘disliking’) rather than, say, merely a basic sensory feature of the stimulus (sweet versus salty). However, we additionally found that neuronal activity tracked a change in the relative hedonic value of these stimuli when the pleasantness of NaCl taste was selectively manipulated by inducing a physiological salt appetite. When rats were sodium depleted (by mineralocorticoid hormone and diuretic administration), the intense salty taste became behaviorally ‘liked’ as much as sucrose, and neurons in ventral pallidum began to fire as vigorously to salt as to sucrose [7••] (Figure 3). We think such observations indicate that, indeed, the firing patterns of these ventral pallidal neurons encode hedonic ‘liking’ for the pleasant sensation, rather than simpler sensory features [21,32].

Figure 3.

Neuronal coding of ‘liking’ for the sensory pleasure of sweet and salty tastes. Neuronal firing responses are shown from a ventral pallidum recording electrode to tastes of sucrose and intense salt infused into the mouth of a rat. Two conditions were tested for both tastes: a baseline condition of normal physiological balance (in which intense salt is ‘disliked’ and sugar is ‘liked’), and a depletion condition of sodium deficit and salt appetite (in which both tastes are ‘liked’). Time = 0 is when each taste infusion began. Modified from [7••].

Hedonic hotspots distributed across the brain may be functionally linked together into an integrated hierarchical circuit that combines multiple forebrain and brainstem, akin to multiple islands of an archipelago that trade together [21,24,27]. At the relatively high level of limbic structures in ventral forebrain, the enhancement of ‘liking’ by hotspots in accumbens and ventral pallidum may act together as a single cooperative heterarchy, needing unanimous ‘votes’ by both hotspots [28]. For example, hedonic amplification by opioid stimulation of one hotspot can be disrupted by opioid receptor blockade at the other hotspot although ‘wanting’ amplification by the NAc hotspot was more robust, and persisted after VP hotspot blockade [28]. A similar interaction underlying ‘liking’ has been seen following opioid and benzodiazepine manipulations (probably involving the parabrachial nucleus of the brainstem pons) [27]. The ‘liking’ enhancement produced by benzodiazepine administration seems to require the obligatory recruitment of endogenous opioids, because it is prevented by naloxone administration [33]. Thus a single hedonic circuit may combine together multiple neuroanatomical and neuro-chemical mechanisms to potentiate ‘liking’ reactions and pleasure.

‘Wanting’

Usually a brain ‘likes’ the rewards that it ‘wants’. But sometimes it may just ‘want’ them. Research has established that ‘liking’ and ‘wanting’ rewards are dissociable both psychologically and neurobiologically. By ‘wanting’, we mean incentive salience, a type of incentive motivation that promotes approach toward and consumption of rewards, and which has distinct psychological and neurobiological features. For example, incentive salience is distinguishable from more cognitive forms of desire meant by the ordinary word, wanting, that involve declarative goals or explicit expectations of future outcomes, and which are largely mediated by cortical circuits [34–37]. By comparison, incentive salience is mediated by more subcortically weighted neural systems that include mesolimbic dopamine projections, does not require elaborate cognitive expectations and is focused more directly on reward-related stimuli [34,35,38]. In cases such as addiction, involving incentive-sensitization, the difference between incentive salience and more cognitive desires can sometimes lead to what could be called irrational ‘wanting’: that is, a ‘want’ for what is not cognitively wanted, caused by excessive incentive salience [39•,40•,41].

‘Wanting’ can apply to innate incentive stimuli (unconditioned stimuli, UCSs) or to learned stimuli that were originally neutral but now predict the availability of reward UCSs (Pavlovian conditioned stimuli, CSs) [38,40•]. That is, CSs acquire incentive motivational properties when a CS is paired with receipt of an innate or ‘natural’ reward via Pavlovian stimulus–stimulus associations (S–S learning). Incentive salience becomes attributed to those CSs by limbic mechanisms that draw upon those associations at the moment of ‘wanting’, making a CS attractive, and energizing and guiding motivated behavior toward the reward [35].

When a CS is attributed with incentive salience it typically acquires distinct and measurable ‘wanting’ properties [35,42], which can be triggered when the CS is physically re-encountered (although vivid imagery of reward cues may also suffice, especially in humans). The ‘wanting’ properties triggered by such reward cues include the following:

1. Motivational magnet feature of incentive salience. A CS attributed with incentive salience becomes motivationally fascinating, a kind of ‘motivational magnet’, which is approached and sometimes even consumed (Supplemental Movie 1) [43,44•,45]. The motivational magnet feature of CS incentives can become so powerful that a CS may even evoke compulsive approach [46]. Crack cocaine addicts, for example, sometimes frantically ‘chase ghosts’ or scrabble after white granules they know are not cocaine.

2. Cue-triggered US ‘wanting’ feature. An encounter with a CS for a reward also triggers ‘wanting’ for its own associated UCS, presumably via transfer of incentive salience to associatively linked representations of the absent reward [34,47,48]. In animal laboratory tests, this is manifest as a phasic peak of cue-triggered increases in working for the absent reward (mostly specifically assessed in tests called PIT or Pavlovian-Instrumental Transfer conducted under extinction conditions; Figure 4). The cue-triggered ‘wanting’ can be quite specific for the associated reward, or sometimes spill over in a more general way to spur ‘wanting’ for other rewards too (as perhaps when sensitized addicts or dopamine-dysregulation patients exhibit compulsive gambling, sexual behavior, etc., in addition to compulsive drug-taking behavior) [49,50]. Thus, encounters with incentive stimuli can dynamically increase motivation to seek out rewards, and increase the vigor with which they are sought, a phenomenon that may be especially important when cues trigger relapse in addiction.

3. Conditioned reinforcer feature. Incentive salience also makes a CS attractive and ‘wanted’ in the sense that an individual will work to obtain the CS itself, even in the absence of the US reward. This is often called instrumental conditioned reinforcement. Similarly, adding a CS to what is earned when an animal works for a US reward such as cocaine or nicotine, increases how avidly they work, perhaps because the CS adds an additional ‘wanted’ target [51]. However, we note that conditioned reinforcement is broader than ‘wanting’, needing additional associative mechanisms to acquire the instrumental task. Also, alternative S-R mechanisms might mediate conditioned reinforcement in certain situations without incentive salience at all. This makes the motivational magnet and cue-triggered ‘wanting’ properties especially important for the identification of excessive incentive salience.

Figure 4.

NAc amphetamine amplification of cue-triggered ‘wanting.’ Transient peaks of ‘wanting’ for sucrose reward are triggered by 30-s appearances of a Pavlovian sucrose cue in a Pavlovian-Instrumental Transfer test (CS+; right). Amphetamine microinjection in nucleus accumbens magnifies ‘wanting’ for sugar reward — only in the presence of the reward cue (CS+), indicating magnification of the cue’s incentive salience. Only cue-triggered ‘wanting’ was enhanced by this dopamine-related stimulation. By contrast, ‘liking’ reactions to sucrose were not amplified by amphetamine microinjections in NAc (not shown). Drug-induced sensitization of NAc-related systems produces a similar pattern of effects that lasts much longer. Modified from [47].

Extensions of incentive salience

1. Action salience? Before we leave the psychological features of ‘wanting’, we are tempted to speculate that some behavioral actions or motor programs may also become ‘wanted’, almost like incentive stimuli, through a form of incentive salience applied to brain representations of internal movements rather than representations of external stimuli. We call this idea ‘action salience’ or ‘wanting’ to act. Action salience we suggest may be a motor equivalent to stimulus incentive salience, and mediated by overlapping brain systems (e.g. dorsal nigrostriatal dopamine systems that overlap with ventral mesolimbic ones). Generation of urges to act, perhaps involving blended motor and motivational functions within the neostriatum (a structure also known to participate in movement) seems consistent with several emerging lines of thought about basal ganglia function [52,53,54•,55].

2. Can desire be related to dread? Finally, we note that incentive salience may also share perhaps surprising underpinnings in mesocorticolimbic mechanisms with fearful salience [56,57•,58,59]. For example, dopamine and glutamate interactions in nucleus accumbens circuits generate not only desire, but also dread, organized anatomically as an affective keyboard, in which disruption of sequentially localized keys generates incremental mixtures of appetitive versus fearful behaviors [57•]. Further, some local ‘keys’ in the nucleus accumbens can be flipped from generating one motivation to the opposite by psychologically changing external affective ambience (e.g. change from a comfortable home environment to a stressful one brightly lit and filled with raucous rock music) [56]. Such recent findings indicate that neurochemical specializations or anatomical localizations of ‘liking’ or ‘wanting’ functions described above may not necessarily reflect permanently dedicated ‘labeled line’ mechanisms where ‘one substrate = one function’. Rather they may reflect specialized affective capabilities (e.g. hedonic hotspots) or motivation-valence biases (e.g. desire-dread keyboard) of their particular neurobiological substrates. Some of those substrates may be capable of multiple functional modes, depending on other simultaneous factors, so that they are able to switch between generating functions as opposite as desire versus dread.

Neurobiological substrates for ‘wanting’

Contrasting the neurobiology of ‘wanting’ to ‘liking’, we note that brain substrates for ‘wanting’ are more widely distributed and more easily activated than substrates for ‘liking’ [38,53,60,61•,62–65]. Neurochemical ‘wanting’ mechanisms are more numerous and diverse in both neurochemical and neuroanatomical domains, which is perhaps the basis for the phenomenon of ‘wanting’ a reward without equally ‘liking’ the same reward. In addition to opioid systems, dopamine and dopamine interactions with corticolimbic glutamate and other neurochemical systems activate incentive salience ‘wanting’. Pharmacological manipulations of some of those systems can readily alter ‘wanting’ without changing ‘liking’. For example, suppression of endogenous dopamine neurotransmission reduces ‘wanting’ but not ‘liking’ [38,64]. Conversely, amplification of ‘wanting’ without ‘liking’ has been produced by the activation of dopamine systems by amphetamine or similar catecholamine-activating drugs given systemically or microinjected directly into the nucleus accumbens, or by genetic mutation that raises extracellular levels of dopamine (via knockdown of dopamine transporters in the synapse) in mesocorticolimbic circuits, and by the near-permanent sensitization of mesocorticolimbic-dopamine-related systems by repeated administration of high-doses of addictive drugs (Figure 3–Figure 5) [39•,40•,61•,66]. We have proposed that in susceptible individuals the neural sensitization of incentive salience by drugs of abuse may generate compulsive ‘wanting’ to take more drugs, whether or not the same drugs are correspondingly ‘liked’, and thus contribute to addiction [39•,40•,42] (Figure 5).

Figure 5.

Incentive-sensitization model of addiction. Schematic model of how ‘wanting’ to take drugs may grow over time independently of ‘liking’ for drug pleasure as an individual becomes an addict. The transition from casual drug use to compulsive addiction is posited to be owing to drug-induced sensitization of mesocorticolimbic mechanisms of incentive salience. Modified from [42].

Dissecting learning from ‘wanting’: the predictive versus incentive properties of reward-related cues

Once reward-related cues are learned, those cues predict their associated rewards and in addition trigger motivational ‘wanting’ to obtain the rewards. Are prediction and ‘wanting’ one and the same? Or do they involve different mechanisms? Our view is that learned prediction and incentive salience can be parsed apart, just as ‘liking’ and ‘wanting’ can [37,38,39•,41,46,61•]. Parsing psychological functions and their neurobiological substrates is important for experimental models of reward learning and motivation, and has implications for pathologies, including addiction. We will briefly describe three lines of evidence from our laboratories that suggest the predictive and incentive motivational properties of reward-related cues are dissociable.

The first example comes from experiments demonstrating that CSs can elicit approach — that is, they act as a ‘motivational magnet’, drawing the individual to them. Many experiments have established that when a cue or ‘sign’ (CS), such as insertion of a lever through the wall, is paired with presentation of a rewarding US, such as food, animals tend to approach and engage the cue [43,44•]. The key to distinguishing prediction from motivation lies partly in the nature of an individual’s conditioned response (CR) [43]. Some rats will approach the lever more and more rapidly upon each presentation and come to avidly engage the lever by sniffing, nibbling, and even biting it — seemingly attempting to ‘eat’ the lever (Supplemental Movie 1) [45]. A cue that predicts cocaine reward is similarly approached and engaged with its own pattern of excited sniffing behavior [44•], which may account for the ability of drug-associated cues to become maladaptive, attracting addicts to them. Such CRs directed toward the CS itself are called ‘sign-tracking’.

However, not all rats develop a sign-tracking CR. Even in the same experimental situation some rats develop a different CR — they learn to approach the ‘goal’ (the food tray), not the lever, when the lever-CS is presented. This CR is called ‘goal-tracking’. Thus, with experience goal-trackers come to approach the goal more and more rapidly upon each presentation of the lever-CS, and they begin to engage the food tray avidly, nibbling, and even biting it [43,44•,45]. For all rats, the CS (lever insertion) carries equal predictive significance: it triggers both the sign-tracking CRs and the goal-tracking CRs. The only difference is where the CR is directed. This suggests that in sign-trackers the lever-CS is attributed with incentive salience because for them it is attractive, and that is supported by observations that sign-trackers specifically also will learn to perform a new response to get the CS (i.e. instrumental conditioned reinforcement) [46]. For goal-trackers the CS predicts food, and leads to the development of a CR, but the CS itself does not seem to be attributed with incentive salience in these ways (instead if anything, the goal is ‘wanted’) [43,46]. Such findings are consistent with our proposition that the reward-predicting or associative value of a learned CS may be dissociated from its motivational value, depending on whether it is actively attributed with incentive salience [46].

A second line of evidence to parse prediction from incentive salience comes from studies of ‘wanting’ neural codes, especially after dopamine-related brain activations (by amphetamine or prior sensitization). Dopamine elevation appears to specifically enhance limbic neural firing to signals that encode maximal incentive salience (Figure 6) [61•]. By contrast, dopamine activation did not enhance neural signals that code maximal prediction [61•].

Figure 6.

Separation of CS incentive value (wanting) from CS predictive value (learning) by mesolimbic activation (induced by sensitization or acute amphetamine administration). This profile analysis of neuronal firing patterns in ventral pallidum shows shifts toward CS incentive coding by either form of mesolimbic activation, and additive interaction when sensitization and amphetamine after were combined. Ordinarily neurons maximally signal predictive value (firing maximally to CS + 1 of a series of three stimuli: CS + 1 sound, CS + 2 sound, UCS sugar). Sensitization and amphetamine administration each shift neuronal coding preference toward incentive signaling (firing maximally to the CS + 2), and away from predictive signaling (without altering signals for the hedonic impact of the sugar UCS). Modified from [61•].

A third line of evidence comes from dynamically reversing ‘wanting’ of a CS while holding its learned prediction constant. For example, a cue that predicts intense saltiness is normally ‘not wanted’ but can be reversed into a ‘wanted’ cue when a physiological salt appetite is induced. No new learning, and thus no change in learned predictions, needs to occur for this motivation reversal to happen. Further, the unusual appetite state need never have been experienced before, and the CS does not need to have ever been associated with a ‘liked’ taste before. Yet still, the previously negative CS suddenly becomes ‘wanted’ in the new state and able to elicit firing patterns that are typical of incentive salience. On the very first trials in the salt appetite state, the CS suddenly evokes neural firing signals that encode positive ‘wanting’, even before the salt UCS has ever been tasted as ‘liked’ [67]. Such observations indicate that a cue’s predictive value is distinct from its ability to elicit ‘wanting’, as the latter requires engaging additional neural systems to generate incentive salience and attribute ‘wanting’ to a motivational target.

More research will be required to determine how ‘wanting’ versus learning and prediction are parsed within the brain. Nevertheless, the evidence so far indicates that these components have distinct psychological identities and distinguishable neural substrates.

Conclusion

Affective neuroscience studies of ‘liking’, ‘wanting’, and learning components of rewards have revealed that these psychological processes map onto distinct neuroanatomical and neurochemical brain reward systems to a marked degree. This insight can lead to a better understanding of how brain systems generate normal reward, and into clinical dysfunctions of motivation and mood. Such applications include especially how sensitization of mesolimbic systems may produce compulsive pursuit of rewards in drug addiction and related motivation disorders by specifically distorting ‘wanting’ for a reward.

Appendix A. Supplementary data

Supplementary data associated with this article can be found, in the online version, at doi:10.1016/j.coph. 2008.12.014.