Disentangling pleasure from incentive salience and learning signals in brain reward circuitry

Abstract

Multiple signals for reward—hedonic impact, motivation, and learned associative prediction—are funneled through brain mesocorticolimbic circuits involving the nucleus accumbens and ventral pallidum. Here, we show how the hedonic “liking” and motivation “wanting” signals for a sweet reward are distinctly modulated and tracked in this circuit separately from signals for Pavlovian predictions (learning). Animals first learned to associate a fixed sequence of Pavlovian cues with sucrose reward. Subsequent intraaccumbens microinjections of an opioid-stimulating drug increased the hedonic liking impact of sucrose in behavior and firing signals of ventral pallidum neurons, and likewise, they increased incentive salience signals in firing to the reward-proximal incentive cue (but did not alter firing signals to the learned prediction value of a reward-distal cue). Microinjection of a dopamine-stimulating drug instead enhanced only the motivation component but did not alter hedonic impact or learned prediction signals. Different dedicated neuronal subpopulations in the ventral pallidum tracked signal enhancements for hedonic impact vs. incentive salience, and a faster firing pattern also distinguished incentive signals from slower hedonic signals, even for a third overlapping population. These results reveal separate neural representations of wanting, liking, and prediction components of the same reward within the nucleus accumbens to ventral pallidum segment of mesocorticolimbic circuitry.

Pleasant rewards are often predicted and wanted as well as liked, especially when preceded by learned cues. Attraction to food in the refrigerator when hungry, for example, involves learned predictions of tasty treats, motivation to eat, and finally, pleasure enjoyed upon eating. How do neural circuits separately control and track these components of Pavlovian reward as distinct signals? An answer to this question is crucial to unraveling addictions, binge eating, depression, and related motivation or hedonic disorders.

Diverse information about rewards is funneled through pathways from the nucleus accumbens (NAc) to the ventral pallidum (VP) embedded in larger mesocorticolimbic circuitry (1–13). Food, sex, drugs, winning money, and other rewards or predictive cues all activate these pathways in humans and other animals (11, 14–29). Recent evidence indicates a surprising multiplicity of control systems within the NAc and VP, including multiple parallel and segregated loops that carry point to point signals through restricted subregions of prefrontal cortex-NAc-VP-thalamus and back again (7). Modulation by diverse neurochemicals also contributes to mediating separable hedonic and motivation functions (1, 10, 25, 30, 31).

Once thought of mainly as a limbic motor translation zone (13), it is now clear that the VP as well as the NAc controls reward motivation and hedonics (1, 3, 6, 31–34). Both structures contain subregional hotspots where μ-opioid agonist microinjections induce a doubling of the hedonic impact or palatability of a sweet taste of sucrose (35, 36). Hedonic enhancement is reflected in positive patterns of affective orofacial expressions that are homologous between humans and rodents, which can be used to operationalize palatability (referred to in shorthand here as “liking” reactions). Such microinjections simultaneously induce increases in incentive salience or Pavlovian-triggered motivation (here called “wanting” in shorthand) (1, 31, 35–37). A neurochemical distinction has been made between opioid stimulation of the NAc hotspot, which amplifies hedonic impact (liking) as well as motivation (wanting) for reward, and mesolimbic dopamine stimulation, which only amplifies motivation (1, 38–44).

Firing patterns of neurons within the VP encode Pavlovian conditioned stimulus (CS) features, such as prediction and incentive salience, and unconditioned stimulus (UCS) reward features, such as hedonic impact (45, 46). Problematically, however, it has never been clear how NAc-VP circuitry could distinguish various reward signals from each other. Existence of multiple anatomical loops connecting the NAc and VP provides a way to segregate the mixture of reward components, but some NAc and VP subregions and single neurons may also need to represent more than one signal (6, 35, 47). Here, we find that disentangling of signals can still be faithfully achieved. They are revealed by distinct effects of activating NAc neuromodulatory transmitters on different VP reward-related firing patterns in a manner that preserves separate wanting, liking, and Pavlovian prediction information for the same reward.

Results

Serial Separation of Maximal Reward Signals.

Firing of VP neurons was recorded in rats exposed to a learned Pavlovian reward sequence after microinjection in the NAc of vehicle (control condition), [D-Ala2, N-MePhe4, Gly-ol]-enkephalin (DAMGO) (μ-opioid stimulation), or amphetamine (dopamine stimulation; n = 115 neurons, mean = 12.8 neurons recorded per test session, n = 8 rats for neural recording, n = 10 rats for Fos functional localization analysis). In this sequence, an initial CS+1 (auditory tone; 1–5 s) preceded a second CS+2 (different tone; 10–15 s) and finally, a pleasant UCS reward (10-s infusion of sucrose solution into the mouth by oral cannulae; 13–23 s), all in a fixed interval (Fig. 1). The use of a sequence of distinct Pavlovian cues allowed us to temporally separate moments of maximum intensity for each reward component (48, 49). Concerning prediction signals, in information theory (50), the predictive contingency between CS+1, CS+2, and UCS in these time intervals meant that CS+1 presentation potentially added 3.8 log units of information, designated by the term H, to create a nearly 100% certainty (i.e., minimal entropy) that CS+2 and reward UCS would both follow. Given that CS+1 temporarily reduces uncertainty about impending reward to near zero, because it predicts the entire remaining sequence of CS+2 and UCS events, the CS+2 presentation adds little or no further information value or reduction in uncertainty about UCS occurrence (quantitative explanation in Materials and Methods) (48, 50, 51). When CS+1 and CS+2 are separated by several seconds, a gradual rise in incentive salience begins soon after the CS+1 onset, motivating anticipatory behavioral approach (e.g., head entries into a sucrose dish) which reaches maximum levels around the moment of CS+2 presentation and just before the UCS (48, 49). The final event in the sequence, the UCS sucrose infusion (i.e., the reward), carried the strongest hedonic impact, which was confirmed by taste reactivity results (45). Separately from this CS+1 → CS+2 → UCS trial sequence, a distinct CS− tone was randomly interleaved during the session as a control stimulus that predicted nothing and thus, carried little predictive, motivation, or hedonic value. For additional comparison of reward modulation of VP activity during appetitive and consummatory behaviors, the rats were also exposed on the same day to learned traditional Pavlovian trials in which a single CS+ (feeder click; combining predictive and incentive value) predicted sucrose pellet delivery into a dish and evoked approach and consumption.

Fig. 1.

Design of experiments within a test day. In the serial Pavlovian design, two CS+ sounds (first CS+1 and then CS+2) predicted a sucrose UCS. In the simple Pavlovian design, a single CS+ sound predicted sucrose pellet delivery as a UCS. On test days conducted after rats had learned the Pavlovian associations, microinjections of DAMGO, amphetamine, or vehicle were made in the hedonic hotspot of the NAc shell to activate NAc-VP circuits immediately before the test, and VP neural firing was recorded to all stimuli. For serial Pavlovian tests, serial CS+ cues were presented in extinction to prevent relearning about the UCS (20–45 min postinjection), and sucrose UCS was tested alone in a second block (45–60 min) to isolate hedonic signals. Reinforced Pavlovian approach trials with the single CS+ were presented in a third block (60–75 min). Consumption of M&Ms was then used to verify increases of behavioral food wanting (75–105 min).

Tests occurred over 3 d for within-subject comparison of vehicle, DAMGO, and amphetamine microinjection conditions in an order pseudorandomly assigned across rats. On each test day, VP neural activity was recorded after NAc microinjection first to serial CS+ cues presented in extinction (i.e., without sucrose to prevent new learning about UCS), then to sucrose UCS infusions, and finally, to the reinforced single CS+ Pavlovian approach trials (Fig. 1). Last, motivated food consumption was measured in a free intake test of M&M candies in the home cage. The order of testing was designed to avoid timing confounds in comparing across drug conditions by equating the time points for each stimulus after drug injections. For example, CS+1 vs. CS+2 data were always compared first each day (under vehicle, amphetamine, or DAMGO microinjection conditions), UCS data were always compared next, and chocolate consumption was always compared in the final test each day. The order design also isolates the crucial extinction tests of serial CS+1 and CS+2 signals in the very first test and provides data for motivation signals at early (CS+2) and late (single CS+) time points after injections. Previous studies have suggested that DAMGO/amphetamine microinjection effects persist long enough each day to influence all test blocks (52–54), which we show they do here.

NAc Opioids and Dopamine Accentuate VP Signals for CS+ Wanting Without Distorting Prediction Signals.

Two-thirds of recorded VP neurons (67%; 77/115) changed firing after presentation of the CS+1, CS+2, UCS, or single-approach CS+ stimuli in trained rats. These neurons were either integrative, responding to multiple stimuli (47%), or belonged to dedicated subpopulations that fired to only one stimulus (53%).

In the cue-only extinction block of trials, 45% of VP neurons (52/115) fired phasically at the onset of the CS+1 and/or CS+2 cues (23% of these fired to the CS+1, 21% of these fired to the CS+2, and 56% of these fired to both) (Figs. 2 and 4 and Fig. S1). The phasic response consisted of a rapid climb in firing rate to reach a peak within 200 ms of each CS+ tone onset. The peak firing duration was 200–300 ms, at which point firing either rapidly returned to baseline levels (34/52; 65%) or remained slightly elevated above baseline for the duration of the tone cues (18/52; 35%) (examples shown in Fig. 4 and Fig. S1). A few neurons exhibited more highly phasic responses with a rapid firing increase at cue onset that persisted for only 100 ms or less (example in Fig. S1). There were no recorded neurons with only tonic activity changes to these cues.

Fig. 2.

Incentive salience was amplified by opioid or dopamine stimulation of the NAc. (A) Neural firing in the VP was increased to the incentive CS+2 but never to the predictive CS+1 after either DAMGO (red line) or amphetamine (green) microinjections in the NAc compared with after vehicle (gray line). For each neuron, firing rate in sequential 0.5-s time windows was calculated as a percentage of its average rate 5 s before stimulus onset [baseline (BL) = 100%]. Lines represent mean normalized firing per drug, and lighter bands represent ± SEM. (B) Sequential 0.1-s windows at cue onset show that firing increases were driven by a fast and phasic burst, which tapered to baseline by 0.2–0.3 s after cue onset (note that the firing rise in A is averaged over the full 0.5 s, which dampens peak amplitude). (C) Behavioral intake of chocolate M&Ms was also increased by NAc amphetamine (A) or DAMGO (D) microinjection compared with vehicle (V) from ∼7 to 10 M&Ms eaten over 30 min. (D) VP firing to the control CS− that predicted nothing was never changed by dopamine or opioid stimulation of the NAc.

Fig. 4.

Distinct neural populations and firing signatures track liking vs. wanting enhancement. (A–D) Neuronal enhancements of firing to CS+2 vs. UCS induced by DAMGO. (A) Partly distinct DAMGO subpopulations encoded enhancement of firing above vehicle to the incentive CS+2 (but not to UCS), firing enhancement of hedonic UCS (but not of CS+2), or firing enhancements to both CS+2 and UCS. (B) Distinct temporal firing patterns tracked UCS liking vs. CS+2 wanting in the population that encoded both (patterns were also shared by the larger VP populations). Line graph shows firing to the CS+2 (green dashed) vs. UCS (red solid) during the first 5 s of stimulus presentation relative to prestimulus baseline levels. Time 0 equals tone onset for CS+2 and estimated arrival of sucrose in the mouth for UCS (0.5 s after pump onset because of infusion lag). DAMGO-induced elevation of CS+2-evoked firing was rapid and phasic, whereas DAMGO elevation of UCS firing was slower in latency to reach peak firing levels and was more sustained in duration. Shown also on the y axis are peak-firing levels to each stimulus in the vehicle condition (gray dash marks). A single example neuron is additionally shown below firing during the UCS and CS+2. (C) In the subpopulation of neurons in which DAMGO elevated firing to the incentive CS+2 only (and not to UCS), the elevated response was similarly rapid and phasic, although it was lower in magnitude and slightly more sustained compared with the dual-coding population in which DAMGO elevated both CS+2 and UCS firing. An example neuron is shown below. (D) In the subset of neurons in which DAMGO elevated firing to the liked UCS only (and not to CS+2), a gradual elevation of firing rate was observed to the UCS after opioid stimulation compared with vehicle levels. An example neuron is shown below. (E) Amphetamine-firing enhancements are neurons that fired at higher levels to the CS+2 compared with after vehicle, and they had a rapid and phasic peak elevation similar to DAMGO enhancement of CS+2 firing (but amphetamine never elevated UCS firing over vehicle).

On control tests after vehicle microinjections, the increase in phasic firing to the CS+1 was greater than any other stimulus: more than the CS+2, more than the UCS (in separate trials), and much more than the control cue, CS−. In other words, with this learned sequence of cues, the CS+1 prediction-related signal dominated VP firing under normal (vehicle) conditions (49) (compare peak amplitudes of CS+1 with CS+2 and CS− under vehicle in Fig. 2 A, B, and D). A computational profile analysis (49) and inspection of individual neurons confirmed the dominance of CS+1-related activity (Fig. S2).

By contrast, when either amphetamine or DAMGO microinjections were made in the NAc before testing to, respectively, increase either dopamine or opioid neurotransmission, VP firing still occurred to the CS+1, but peak dominance switched to the CS+2 (Fig. 2 and Fig. S2). This switch from CS+1 to CS+2 dominance was always caused by a selective elevation of VP firing to the CS+2 onset by >150% compared with vehicle levels (main drug effect = 0–0.5 s, F_2,382 = 6.72, P = 0.001; DAMGO/vehicle P < 0.05 and amphetamine/vehicle P < 0.01). The DAMGO- or amphetamine-enhanced CS+2 firing peak was rapid, phasic, and anchored to the CS+2 onset, appearing within the first 200 ms of CS+2 presentation and typically decaying back to vehicle control levels by 400 ms (0.1–0.2 s main drug, F_2,382 = 8.41, P < 0.001; DAMGO/vehicle P < 0.05 and amphetamine/vehicle P < 0.05). The amplitudes of CS+1-associated firing peaks were not altered by DAMGO or amphetamine microinjection [no main drug effect for either drug during 0–0.5 s bin: F_2,391 = 1.90, not significant (NS); 0.1–0.2 s: F_2,391 = 1.421, NS], which indicates that the incentive salience signal was selectively enhanced, whereas the CS+1 prediction signal was unchanged (Fig. 2 and Fig. S2). Firing to the control CS− tone, which predicted nothing, was never enhanced by either drug microinjection and instead, declined below its original low level (main drug effect = 0–0.5 s, F_2,181 = 4.74, P = 0.01; DAMGO/vehicle P < 0.05 and amphetamine/vehicle P < 0.05) (Fig. 2).

Crucially, the enhancements of CS+2 firing by NAc stimulation were dynamic and required no new learning about the UCS under drug, because they were detectable even on the five initial extinction presentations of CS+2 in firing rasters of individual neurons and in population activity (population firing above vehicle levels—DAMGO: F_1,116 = 3.95, P < 0.05; amphetamine: F_1,98 = 22.76, P < 0.001). Also, the CS+2 firing enhancement was not a result of baseline firing changes, which were opposite for the drugs. DAMGO microinjections suppressed, whereas amphetamine microinjections enhanced baseline firing (pre-CS baseline firing drug effect: F_2,774 = 19.90, P < 0.001; DAMGO/amphetamine: P < 0.001 and vehicle/amphetamine: P < 0.05) (Fig. S3), although both drugs produced the same enhancement of CS+2-evoked firing.

Enhancement of CS+ Incentive Salience Replicates During Approach Trials and Promotes Consumption.

We next confirmed that the enhanced representation of CS+ incentive salience in VP neuronal activity occurred similarly in the simpler Pavlovian active approach task later in the same day. These trials represented a more naturalistic situation where a single CS+ was paired with delivery of a sugar pellet UCS that rats must actively approach and consume (rather than receive sucrose passively by intraoral infusions). In this paradigm, the moment of the single CS+ sound represents a composite blend of maximal prediction and maximal incentive salience. As one might expect from this conjunction, phasic firing in the VP was enhanced by both DAMGO and amphetamine microinjections in the NAc. Both drugs amplified the ∼250-ms CS+ firing peak to levels 50–100% over normal peak levels in the vehicle condition and up to 200–350% over pre-CS baseline firing (main drug effect: F_2,128 = 15.29, P < 0.001; DAMGO/vehicle P < 0.001 and amphetamine/vehicle P < 0.05) (Fig. S4).

We also confirmed, with a behavioral index of wanting (30-min free intake test), that both DAMGO and amphetamine microinjections amplified actual voluntary consumption of chocolate candies (M&Ms). Rats consumed 33% more chocolate on average after they had received either DAMGO or amphetamine than when they received vehicle microinjections (equal to ∼2 g more chocolate after DAMGO or amphetamine microinjection; DAMGO: t = 2.60, P < 0.05; amphetamine: t = 2.91, P < 0.05 without outlier) (Fig. 2).

NAc Opioid Stimulation but Not Dopamine Accentuates Palatability and Hedonic Signals.

Intraoral UCS taste infusions evoked variable patterns of activity from VP neurons, but the most predominant response was a slow rise in spiking above baseline to peak during the first 1.5 s of sucrose infusion. The elevation typically persisted as a moderate and sustained plateau of firing throughout the remainder of the 10-s infusion (Figs. 3 and 4 and Fig. S1). Only DAMGO microinjection in the NAc magnified the elevation in VP firing to the UCS, raising sucrose-elicited firing to a peak that was >50% above firing recorded after vehicle control injections (first 1.5 s of drug main effect: F_1,381 = 8.90, P < 0.001; DAMGO/vehicle posthoc P < 0.05 during seconds 1.0–1.5 of UCS infusion) (Figs. 3 and 4 and Figs. S1 and S2). A simultaneous DAMGO-evoked enhancement in behavioral responses was observed as a >30% increase in the number of positive orofacial affective reactions elicited by the taste of sucrose infusion compared with vehicle levels (F_1,152 = 35.62, P < 0.001). This behavioral response pattern confirms enhanced sucrose palatability or liking (Fig. 3) with NAc DAMGO treatment. Negative disgust reactions (e.g., gapes) remained very low and unchanged.

Fig. 3.

Hedonic liking for UCS was increased by opioid stimulation of the NAc hotspot but not by dopamine stimulation. (A) DAMGO microinjection in the NAc increased VP firing to the intraoral UCS sucrose taste (red line; mean firing rate over baseline ± SEM band as in Fig. 2) compared with after vehicle microinjection (gray line). Amphetamine microinjection in the NAc failed to enhance VP firing to the UCS (green line). x axis is as it is in Fig. 2. (B) Behavioral confirmation that DAMGO uniquely enhanced hedonic impact of sucrose was seen in increased orofacial liking reactions to the sucrose taste after opioid stimulation of the NAc but never after dopamine stimulation.

In stark contrast, amphetamine microinjections in the NAc never elevated VP firing to the UCS (instead, it produced a trend to suppression; posthoc from above, NS) (Figs. 3 and 4 and Figs. S1 and S2) and likewise, never enhanced positive behavioral orofacial reactions to the palatability of sucrose (F_1,154 = 1.27, NS) (Fig. 3). Both negative results indicated that NAc dopamine failed to accentuate either hedonic-related UCS firing activity in the VP or behavioral indices of hedonic impact. Baseline firing rates before the UCS continued to be slightly suppressed by DAMGO and slightly enhanced by amphetamine (baseline of 5 s before UCS infusion; main drug effect: F_2,390 = 53.28, P < 0.001; DAMGO/vehicle P < 0.001 and amphetamine/vehicle P < 0.001) (Fig. S3). We note that the failure of amphetamine to enhance UCS firing was not likely because of the higher baseline rate, because (i) the highest baseline under amphetamine (20 spikes/s) never reached heights of stimulus-evoked firing (>60 spikes/s), (ii) even UCS-responsive neurons with low baselines under the amphetamine condition still failed to show enhanced UCS-evoked firing, and (iii) amphetamine baseline rates continued to rise in the single CS+ approach block of trials that were conducted after these UCS trials (below) when both amphetamine and DAMGO increased firing to that single CS+ (Fig. S3).

We assessed hedonic impact further by measuring orofacial reactions to the Pavlovian CS+ cues, which in principle, can acquire conditioned hedonic value as well as incentive value from their paired UCS (55–60). The CS+1 tone occasionally and the CS+2 more frequently elicited a few positive hedonic reactions (on average, 1 reaction per 10 s under vehicle condition; this weak hedonic conditioned response was only 12% of the UCS unconditioned level of 8.3 per 10 s, but tone-elicited reactions were still mildly elevated over near-zero pre-CS baseline levels; CS+2: F_1,157 = 8.30, P < 0.01; CS+1: F_1,157 = 3.82, P = 0.052) (Fig. S5). DAMGO microinjection in the NAc approximately doubled the number of positive hedonic conditioned reactions such as lip licking (to two reactions per tone) during both the CS+1 and CS+2 above vehicle numbers (CS+2: F_1,130 = 4.77, P < 0.05; CS+1: F_1,130 = 4.33, P < 0.05). By contrast, amphetamine microinjection never enhanced conditioned orofacial reactions elicited by tone CSs compared with vehicle (CS+2: F_1,145 = 1.05, NS; CS+1: F_1,145 = 0.67, NS) (Fig. S5) just as it had failed to enhance hedonic reactions elicited by the UCS. This illustrates that the UCS carried much higher hedonic impact than CSs and confirms our hypothesis that DAMGO but not amphetamine in the NAc amplifies hedonic impact of stimuli, whether conditioned or unconditioned.

Distinct VP Ensembles and Firing Signatures Potentially Disentangle Liking vs. Wanting Enhancements.

Can NAc-VP circuits distinctly parse reward components even when multiple ones are enhanced together in the same neurons? How conjoint increases in wanting vs. liking are distinguished cannot be fully resolved by current data, but indications were found both for a segregated subpopulation code that discriminated CS+2 from UCS enhancements by DAMGO and for a temporal pattern code that distinguished CS+2 vs. UCS signals even when both were enhanced in the same neurons (Fig. 4). In a subset of 14 VP neurons that fired to the CS+2 or UCS in the DAMGO condition at levels clearly exceeding vehicle (i.e., those that dominated the above ensemble firing enhancement to those stimuli), eight (57%) fired only to the CS+2 and not to UCS sucrose. Conversely, two other neurons (14%; 2/14) fired above normal vehicle levels to the UCS after DAMGO but not to the CS+2. Thus, most of these stimulus-preferring neurons (10/14; 71%) fired robustly to one stimulus but not at all to the other stimulus.

This population coding mechanism for separating CS+2 from UCS would not apply, however, to a third remaining subpopulation of four neurons (29%) that showed similarly elevated activations above vehicle levels after NAc DAMGO to both the hedonic UCS and the incentive CS+2. The enhanced peak responses to the CS+2, however, were phasic with shorter latencies and shorter durations compared with UCS peak responses, which in contrast, had slower latencies of rise (1 s or more to peak after sucrose entry into the mouth; estimated to occur 0.5 s after pump onset) and more sustained duration of activation during UCS infusion (Fig. 4). Thus, on average, DAMGO elevation of firing above vehicle levels took approximately more than three times longer to manifest after UCS compared with the rapid enhancement after CS+2 onset. Notably, this temporal pattern was shared over the entire population of responsive neurons after DAMGO (compare Fig. 2 with Fig. 3), indicating a potential temporal way of discriminating CS+2 and UCS events in addition to subpopulations.

Stimulus Focus of VP Populations.

A related potent decoding in responsive VP populations was observed in a sharpened stimulus focus after NAc microinjections of DAMGO or amphetamine. Compared with vehicle, both drugs caused a rise in the proportion of responsive neurons that were activated uniquely (that is, responsive to only CS+1, only CS+2, only UCS, or only the single CS+). Specifically, under normal vehicle conditions, 68.4% of VP neurons fired to at least two stimuli in the task, but after dopamine or opioid stimulation of the NAc, there was a significant increase in the proportion of neurons responsive to only one stimulus (each ∼50% of responsive neurons; χ², P = 0.0001). Alongside this was a consequent fall in the proportion that responded to two or more stimuli. Thus, after NAc stimulation, VP neurons became more selective for a particular type of reward stimulus, conceivably adding clarity to the discrimination among reward signals for downstream targets (Fig. S6).

Test Requirements Modulate VP Bursting and Baseline Activity.

VP bursting patterns and baseline firing levels were modulated by the behavioral requirements of the tests. Temporal bursting patterns increased by about 70% as the test paradigm was switched from the pure Pavlovian test (where CS and intraoral UCS stimuli were presented no matter what the rat did; 20–60 min postdrug injection) to the active Pavlovian approach test (where rats had to approach the dish and actively retrieve to consume a sucrose pellet UCS; 60–75 min postinjection; ANOVA: F_{4, 91} = 14.979, P < 0.001) (Fig. S7). Higher bursting during the active approach task was observed during both tonic baseline firing and phasic CS and UCS peaks (F_2,233 = 17.95, P < 0.001). We also observed a concomitant increase in baseline rate in prestimulus firing at the task switch compared with earlier test blocks (Fig. S3). This was most pronounced in the vehicle-control condition but also present after amphetamine or DAMGO microinjections (all conditions overall: F_2,233 = 17.95, P < 0.001; vehicle/amphetamine P < 0.05 and vehicle/DAMGO P < 0.001). The rise in VP baseline (and bursting) activity could reflect the switch to active-approach conditions, contextual conditioning to the chamber, or a relative clamping of baseline firing by NAc drugs that contributed to the steady stimulus-evoked firing increases across test epochs.

Anatomical Distribution of NAc Manipulations.

Finally, we mapped the NAc microinjection effects on VP firing based on anatomical identification of microinjection sites and additional Fos plume data on the spread of drug impact around a microinjection (35, 36) (Fig. S1). As outlined in Materials and Methods, the local spread of NAc neuronal activation caused by DAMGO or amphetamine microinjection was measured under conditions of maximal spread (i.e., first drug microinjection; n = 10 separate rats). This measured diameter was used to construct symbol size in Fig. S1 plume maps representing drug spread (plume center volumes [i.e., 10× normal or 5× vehicle Fos elevation] = 0.0037–0.10 mm³; surrounding outer halo volumes [i.e., 5× normal or 3× vehicle elevation] = 0.84–1.40mm³). It is important to note that all other data depicted in the maps show functional consequences of microinjections and were obtained from rats tested for VP firing and behavior (integration of Fos/function data discussed in Materials and Methods). Microinjection locations in the NAc from the VP recording experiment were plotted based on histologically confirmed locations, and functional maps were constructed representing the effect of each drug microinjection at a particular NAc location on firing in all VP neurons recorded from that particular animal. These maps revealed that the enhancements of incentive salience and hedonic impact were anatomically consistent across our microinjection sites focused on the previously identified hotspot in the rostrodorsal quadrant of medial NAc shell. At most microinjection sites in the NAc medial shell, DAMGO and amphetamine raised phasic VP firing to the CS+2. By contrast, only DAMGO placements in and around the hotspot in midrostral medial shell additionally enhanced phasic VP firing to the UCS (35, 37).

Discussion

Presentation of cues for food, sex, drugs, or money to humans can evoke an increase in VP blood oxygen level dependence (BOLD) signals (14, 18, 19, 61), and in rodents, robust firing in VP neurons is evoked by cues for sensory rewards (46, 62). It has usually been difficult to determine which component of Pavlovian reward (prediction, incentive salience, or hedonic impact) is represented by a neural activation in VP and larger circuitry, because all these reward components covary together in most cases. However, they can be dissociated experimentally under special conditions, such as here, and also may dissociate clinically to cause particular disorders, such as addiction (42, 44). Functional dissociations imply that brain circuits are able to disentangle and track the reward component signals, but how that task is accomplished has remained unclear. Our results help solve this logical puzzle. We showed that dopamine or opioid neuromodulators in the NAc differentially fine-tuned the ratio of reward components represented in VP firing in a manner that may serve to separately track even jointly enhanced incentive salience vs. hedonic impact signals separately.

Associative Prediction of Future Reward.

Incentive- and hedonic-related firing patterns were also distinguishable from prediction signals that, in contrast, did not change during dopamine or opioid stimulation of the NAc. Prediction signal firing rate profiles normally dominated (i.e., under vehicle-control conditions), with highest firing occurring as a consistently rapid and phasic burst that peaked at the onset of CS+1. Prediction dominance by CS+1 here may be related to previous reports of maximal firing to the first predictor in a sequence of several cues by midbrain dopamine neurons and neurons in target structures (49, 51, 63–65) in studies not incorporating drugs or other neural manipulations. It is consistent with computational temporal difference algorithms that increment a cached value of expected future reward, and with learning, that move the prediction forward in time to the initial cue (at least when mesocorticolimbic systems are not in a stimulated state) (50, 51, 66).

Incentive Salience or Wanting Component.

Dopamine stimulation of the NAc enhanced only incentive salience-related firing that peaked at the CS+2. Opioid stimulation similarly enhancedCS+2 firing (and additionally, enhanced hedonic UCS firing as discussed below). These incentive amplifications made CS+2-evoked firing consistently greater than CS+1-evoked firing after both drugs, shifting dominant representations in VP activity from prediction to incentive motivation. Accentuated firing related to the incentive cue was similar across variations in testing procedure (serial cues vs. single-approach cue) and independent of baseline firing rates. Behavioral responses were consistent with this shift reflecting higher wanting for sweet reward: rats ate more chocolate after receiving either amphetamine or DAMGO microinjections in the NAc shell. Enhancement of eating by NAc opioid stimulation is well-known (1, 31, 35, 53), and eating enhancement by NAc amphetamine, although more fragile, is also consistent with prior reports (67, 68) as well as our hypothesis that dopamine mediates incentive salience in a manner sufficient to drive motivated behavior like eating (1, 42, 69) (note that NAc amphetamine-stimulated eating can be contrasted to anorectic effects of systemic amphetamine that occur primarily through norepinephrine or dopamine in medial hypothalamus [and in other structures at high doses]) (70–73).

To distinguish prediction and incentive salience computationally, Zhang et al. (48) proposed a model that quantitatively disentangles the two components in a way that fits our results. In that model, incentive salience, , is computed as (Eq. 1)

Inputs to incentive salience on the right side of the equation distinguish a stable, well-learned prediction value (r_t) that is equivalent to predictions of future reward derived from contemporary temporal difference models of reinforcement learning and accumulated based on previous pairings of the Pavlovian CS with reward UCS (51, 66, 74, 75) from a more dynamic gain-control factor (k) that can shift quickly with changes in physiological state. This gain-control factor, we suggest, evolved to allow natural appetites to modulate wanting independently from associative predictions, and here, it was dynamically elevated by NAc dopamine or opioid stimulation. Consequently, CS+1 firing reflecting prediction of future reward based on the associative memory cache (r_t) was unchanged here. By contrast, only firing evoked by the CS+2 (or CS+ in the approach task) was enhanced by DAMGO and amphetamine microinjections that amplified the NAc-VP gain factor (k).

Hedonic Impact or Liking Component.

Identifying the hedonic component of reward required a different approach, which was aided by effects of NAc opioid stimulation. DAMGO microinjection in the NAc was the only manipulation to enhance liking reactions to the sucrose UCS, which was assessed behaviorally by orofacial taste reactivity (and to enhance the much weaker conditioned hedonic orofacial reactions evoked by CS tones). Amphetamine microinjection failed to enhance liking reactions to any stimulus. Accordingly, only opioid stimulation enhanced the slow-latency rise in VP firing to the UCS that seemed to track the hedonic impact of sucrose taste. These findings show that μ-opioid stimulation of the NAc hotspot magnifies neural signals evoked by a sweet sensation in a manner that may inform neural target structures about the enhancement of hedonic impact.

Reward Component Separation by Population Segregation and Firing Pattern.

All of the neural responses discussed above occur closely together in time, making it a potential challenge for downstream neural circuits to disentangle reward components. Our results, however, suggest that liking, wanting, and learned prediction of reward are still distinguishable in NAc-VP circuits by neuronal population and firing pattern codes. First, somewhat different VP neuronal subpopulations seemed to carry incentive salience (CS+2) vs. hedonic impact (UCS) signals, even when both components were enhanced by the same NAc opioid stimulation. Neuronal segregation seems compatible with anatomical reports that multiple segregated paths exist in parallel within NAc-VP circuits, which are embedded within larger corticolimbic circuits and pallidal-thalamocortical reentry loops (7). Second, even for VP neurons that represented combined wanting and liking enhancements, CS+2 effects were potentially distinguishable by a faster-latency rise in activation than sucrose UCS effects. These population and temporal firing mechanisms would allow liking, wanting, and prediction signals to be told apart within ventral striatopallidal circuits during natural appetite, drug intoxication or withdrawal, or stress states known to elevate reward measures (1, 2, 4, 22, 25, 27, 30, 31, 46, 76).

We suggest that these VP neuronal signals for reward components could be related to reports of activity in human posterior VP positively correlated with pleasant food images (17) and mechanisms by which opioid and related stimulation in a VP hotspot can modulate the hedonic impact of sensory rewards (33, 36, 77). Conversely, one could speculate that impairment of hedonic signals in NAc-VP pathways could contribute to clinical manifestations of anhedonia or incentive motivation impairment in depression and related disorders (78) or to dysphoria after lesions encroaching on the VP hotspot (79–81).

NAc stimulation also focused firing activation on a particular stimulus, individually tailored for particular neurons so that more VP neurons responded to only one stimulus and not any of the others (e.g., CS+1 only, CS+2 only, or UCS only) rather than to multiple stimuli. The net effect of this was a relative increase in the size of the activated neural population with labeled line-like qualities (i.e., responses to a single stimulus entity). More focused channeling could facilitate tracking of separate reward components and improve clarity for downstream neural recipients. This idea is similar to the notion of dynamic focused activation in basal ganglia as a mechanism used to select appropriate actions (82–86).

Concerning NAc-VP interaction, many have noted that reciprocal inhibitory GABAergic projections tend to enforce opposite polarity of events within NAc vs. VP so that hyperpolarization in neurons of the NAc accompanies depolarization in neurons of the VP and vice versa (5, 87, 88). An expectation from that reciprocal inhibition view would be that excitatory peaks of VP firing observed here might have corresponded to inhibitory pauses of NAc neurons, which have been suggested to signal reward (20, 24, 88, 89). However, simultaneous excitations in NAc and VP (or simultaneous inhibitions) may also be possible (29, 37, 62, 87, 90), perhaps enabled by corelease of peptides such as dynorphin, enkephalin, or substance P to modulate the impact of GABA on postsynaptic neurons (5, 6, 32, 87). Additionally, neurons of the NAc hotspot in the rostrodorsal medial shell may not project directly to neurons in the posterior VP hotspot recorded here but rather, to a more anterior site in VP and to lateral hypothalamus (7) from where interneurons might convey signals to posterior VP to contribute to functional interactions (37). A series of multiple GABAergic synapses would open the possibility for disinhibition to create simultaneous excitations in NAc-VP hotspots. Future studies would be needed to assess these issues. Furthermore, regarding circuitry, we emphasize that VP neurons also are likely to encode many other signals beyond reward, including ones related to the originally hypothesized function of limbic to motor translation. Also, of course, the NAc-VP path is only one segment within larger mesocorticolimbic circuits involving cortical reentry loops, parallel segregation, and other features important for both reward impact and translation into behavior (1, 5–7, 11, 27, 34, 91, 92).

Finally, we suggest that our results have relevance to addiction and other compulsive disorders. For example, addiction-related cues can often trigger relapse and consumption, but their ability to do so fluctuates. We suggest that the motivation power of cues is amplified by states of mesolimbic reactivity that magnify incentive salience as observed here, such as drug intoxication or stress states (41, 42, 49, 93). Likewise, the UCS proximity of peak incentive salience signals may be related to why it is easier to resist cues that are temporally distant from rewards (e.g., sight of a crack house; here comparable with encountering CS+1) than to resist other cues that are temporally closer to reward (e.g., sight of crack in one's own hand; comparable with CS+2). In short, the observations described above suggest a possible role for NAc-VP motivational signals in controlling the ebb and flow of vulnerability to cues as triggers of addictive behaviors.

Materials and Methods

Male Sprague-Dawley rats were housed individually in tub cages on a reverse light–dark cycle (total n = 8 rats, 115 neurons from nine test sessions). Additional rats (n = 10) were used to measure Fos reactivity at NAc microinjection sites to estimate drug functional spread (see below). Experiments were conducted during late morning to afternoon hours, which coincided with the rats’ active (dark) period after acclimating to housing conditions for at least 1 wk. Food and water were available ad libitum throughout testing, except for food restriction (20 g/d) during habituation only for the main recording experiment. All procedures were approved by the University Committee on the Use and Care of Animals at the University of Michigan.

Habituation and Surgery.

Rats were initially habituated to the testing chamber for 3 d, given preparatory magazine training for the Pavlovian approach task (free sucrose pellets in the magazine chamber), and given M&Ms in their home cage overnight for familiarization. For surgery, rats were anesthetized with ketamine (100 mg/kg) and xylazine (10 mg/kg) and placed in a stereotaxic apparatus. Bilateral oral cannulae (PE-100 tubing) entered the mouth in the upper cheek lateral to the first maxillary molar, were threaded beneath the zygomatic arch, and exited the dorsal head near the skull (94). Rats also received intracranial cannulae and electrode implantation (on the same day for five rats or after the fourth training day for three other rats). Bilateral stainless steel guide cannulae (23 gauge) were implanted 2.5 mm above the rostral and dorsal quadrant of the NAc medial shell (histologically identified placements spanned anteroposterior (AP) = 0.9–2.2 mm, mediolateral (ML) ± 1.0 mm, dorsoventral (DV) = −6.6 to 7.7 mm) (Fig. S1). A stainless steel obturator was inserted in the NAc cannulae to prevent occlusion. A recording electrode was implanted unilaterally in the posterior VP (histologically shown to span ML = 2.4–3.0 mm, AP = −0.5 to 1.2 mm, and DV = −7.6 to 8.0 mm) (Fig. S1). Each electrode consisted of eight wires (50-μm tungsten). One-half of the electrodes had a screw-driven brass microdrive for lowering before testing, and one-half were lowered during surgery and permanently fixed. On test days, one wire with no spike activity was selected during testing sessions to serve as a reference channel for differential recording. The implant was anchored to the skull with bone screws and acrylic cement. Animals were allowed to recover for at least 7 d.

Training and Serial Cue Design.

A serial Pavlovian reward task was used to separate moments of maximal occurrence for predictive, incentive, and hedonic signals: a CS+1 followed by a CS+2 and an intraoral sucrose UCS, all at fixed intervals. In information theory (50) in the trial overall, the sucrose UCS has an objective probability of occurrence P = 0.07 (e.g., 1 of every 14.7 bins of 10-s duration). That is, in an information theory sense, the surprisal value (h) of a UCS event generally occurring is h = log₂ (1/p). For UCS, h = 3.88 [h = log₂ (14.7)] if sequential dependencies are not considered (50). This general probability might also correspond psychologically to the strength of general contextual association between the training chamber and sucrose if contextual learning matches the objective probability of reward. However, during the CS+1 → CS+2 → UCS sequence, the conditional probability of reward after CS+1 temporarily rises to 100% or P = 1.0 (conditional probability of CS+2 likewise becomes certain; P = 1.0). After it is learned, when a CS+1 is encountered, the momentary surprisal value of the 100% certain UCS that follows reduces to h = 0 [log2 (1/1.0)] as does the surprisal value of the intervening CS+2. That is, the surprisal value of the CS+1 remains h = 3.88, but after it occurs, the CS+1 fully predicts (i.e., reduces uncertainty) both the CS+2 and UCS. The subsequent CS+2 within the sequence adds no additional predictive information or uncertainty reduction concerning the already-predicted UCS. This is seen by noting that, after CS+1, CS+2 h = 0 [log2 (1/1.0)]. The h = 0 zero value of CS+2 and UCS means that the CS+2 occurrence cannot reduce sequential uncertainty about UCS any more (i.e., h cannot go below zero). In other words, the 100% stereotypy of the CS+1 → CS+2 → UCS sequence transfers all information surprise and prediction value forward to the CS+1 that initiates the sequence (similar redundancy is often used to compact data in encryption systems) (48, 50, 51). After a sequence is over, the general probability of reward reverts to 0.07 for some time until the next CS+1 is encountered when the conditional probabilities of the sequence again take over.

However, the CS+2 still carries a high degree of incentive salience, potentially even more than the CS+1 (48, 49). For example, CS+2 presentations are temporally associated with higher levels of appetitive Pavlovian conditioned approach responses than CS+1 presentations (49, 95, 96). Similarly, late-phase behaviors occurring near reward may be more potentiated by a CS+ than earlier-phase behaviors in Pavlovian-instrumental transfer and related procedures to assess cue-triggered incentive motivation (97, 98), which is consistent with a temporal distinction between prediction and incentive salience. Also, hyperbolic discounting or impulsive choice for more immediate rewards in humans is related to dopamine activation (99). Most relevant, systemic amphetamine administration or drug sensitization, which amplifies incentive salience through enhanced dopamine-related activation (93), specifically amplify VP neuron firing to a CS+2 but not to an earlier CS+1 (49). This phenomenon was exploited here to identify firing signals in the VP related to incentive salience that occurred at CS+2 presentation, which is distinct from prediction signals maximal to the CS+1.

Finally, the UCS sucrose taste itself is the only stimulus to carry sensory sweetness, and it elicits maximal facial affective reactions (even when sucrose is predicted by CS+1 and CS+2). Thus, in this stimulus sequence, the UCS is assumed to carry the strongest hedonic impact (45), and this assumption was confirmed by taste reactivity results in which UCS evoked 10 times higher levels of positive orofacial reactions than CS+1 or CS+2.

Training took place on each of 5 consecutive d (Fig. 1) and consisted of two blocks conducted sequentially on each day. Rats were placed in a transparent Plexiglas chamber (28 × 35 cm) with an open top. During the serial cue trials, rats were confined to a transparent cylinder (25-cm diameter) placed within the chamber. A mirror under the glass bottom of the cylinder allowed video recording of bodily and orofacial movements. A computer program controlled stimulus presentations and oral sucrose infusions (Mtask; J. Wayne Aldridge, University of Michigan, Ann Arbor, MI). A separate program controlled neural recording (Recorder; Plexon Inc). The time-stamped clocks were synchronized for taste infusions, stimulus events, neural recordings, and videotape recordings. In the first trial block, rats were presented with 10 serial CS+ trials consisting of the two sequential tones (5-s CS+1 followed by 5-s intertrial interval (ITI) and 5-s CS+2) that predicted an intraoral 0.1-mL infusion of 9% sucrose solution (0.26 M). The infusion was 10 s in duration and began 3 s after CS+2 onset. Ten CS− trials, consisting of a distinct tone predicting nothing, were interspersed randomly with serial CS+ trials using a variable 1-min ITI. CS+1 and CS+2 tone identities were counterbalanced between rats and were a high- (3,800 Hz) or low-frequency continuous tone (400 Hz). CS+ tone assignment did not affect the rate of VP firing to stimuli, and it was not a factor in the effects of NAc microinjection on VP firing (two-way ANOVA on drug X tone assignment; tone assignment: F_2,775 = 2.05, NS; drug and tone interaction: F_2,775 = 1.79, NS); therefore, groups were combined for analyses. The CS− was a low-frequency 0.75-s on/off pulsed tone (400 Hz). A white noise generator was used to mask the noise of the sucrose infusion pump located exterior to the testing chamber. In the second training block conducted immediately after the first block on the same days, the Plexiglas cylinder was removed, and animals were given 10 Pavlovian approach trials in which a single CS+ (feeder click) was followed immediately by the delivery of a sucrose pellet into a hopper (variable 1-min ITI).

Testing.

Tests were then conducted on 3 repeated d spaced 48 h apart and separated by 1 d retraining (conducted as above) (Fig. 1). On each test day, rats first received an intra-NAc microinjection. Drugs were 0.05 μg/0.2 μL DAMGO and 10 μg/0.2 μL d-amphetamine sulfate (Sigma) dissolved in artificial cerebral spinal fluid (ACSF) as vehicle (Harvard Apparatus). Microinjections were made using a stainless steel injector cannula (29 gauge) that extended 2.5 mm beyond the ventral end of the guide cannulae and connected to PE-20 tubing and a syringe pump. Drug microinjections were made bilaterally at a rate of 0.030 μL/min. Microinjector tips were left in place for an additional 1 min after each infusion, and then, obturators were reinserted. Animals received a mock injection of ACSF vehicle after the fifth day of training for habituation. Each rat received each of the three drugs (amphetamine, DAMGO, and ACSF) pseudorandomly assigned across test days.

Twenty minutes after drug microinjection, rats were presented with 10 serial CS+ trials (CS+1 followed by CS+2) randomized with 10 CS− trials in an extinction setting without sucrose reward delivery (although the empty syringe pump was activated as usual). This extinction condition isolated direct shifts in CS signals and prevented CS assessment from being confounded by new learning about the UCS during the test under the influence of drug (e.g., preventing positive UCS prediction error or hedonic signals that might secondarily elevate CS signals). Immediately after the test, hedonic impact of sucrose infusions was tested in subsequent UCS-only trials to isolate sucrose hedonic signals and protect UCS impact from being dampened by prediction. Animals were presented with 10 free oral infusions of the sucrose UCS (10-s infusions and variable 1-min ITI) (Fig. 1). Afterward, the Plexiglas cylinder was removed, and animals were given 10 Pavlovian approach trials that were identical to training in which a feeder click preceded the delivery of one sucrose pellet into a hopper. Finally, animals were then removed from the testing chamber and placed in their home cage with M&Ms. Intake (grams and number consumed) was recorded at 30 and 60 min.

Neural Analysis.

Single neurons were identified using principle components or peak-width analysis of waveforms using Offline Sorter (Plexon Inc). Neurons were verified by distinct spike waveforms and clear refractory periods in an autocorrelation histogram, and cross-correlation analysis was performed to ensure that neurons were counted only one time (NeuroExplorer; Nex Techologies).

A normalized firing response to a stimulus event for each neuron was obtained by dividing the neuron's absolute firing response in a predetermined 500-ms time window at stimulus onset by its baseline during a 5-s period before each trial's onset. A neuron was considered responsive if its absolute firing rate was different from the preceding baseline period for a stimulus event of interest (determined by Tukey-corrected t tests). A few slow-firing neurons that showed phasic excitatory responses to stimuli and crossed a 90% confidence interval but did not reach significance in t tests were also considered responsive. Stimuli events were also compared for neuronal firing at successive 100- or 500-ms epochs to characterize response properties. For comparison of NAc microinjection effects on VP firing, normalized firing rate to each stimulus event in the three testing blocks was compared across drug conditions using ANOVA with trial as a covariate, and individual drug comparisons were made using Tukey posthoc tests. Except where noted, all tests of normalized firing responses to events were conducted on cells firing differently from baseline during the stimulus of interest (exceptions were profile analysis and Fos plume maps that compared across all recorded neurons). Drug-evoked changes in basal firing rate of VP neurons were also compared across microinjection conditions using one-way ANOVA for each of the three testing periods (normalized firing magnitude in a −5- to 0-s window before CS+1, UCS, and feeder click). Additional profile and bursting analysis methods are described in SI Materials and Methods and SI Results.

Behavioral Analyses.

Hedonic, aversive, and neutral taste reactivity patterns during the sucrose infusion in the UCS-only test (stimulus duration plus 5 s = 15 s) (Fig. 3) and during presentation of CS tones in the CS-only test (stimulus duration plus 5 s = 10 s) (Fig. S5) were videotaped and scored off-line in slow-motion using established procedures (94). Hedonic responses included rhythmic midline tongue protrusions, lateral tongue protrusions, and paw licks. Aversive responses included gapes, head shakes, face washes, forelimb flails, and chin rubs. Neutral responses included passive dripping of solution out of the mouth, ordinary grooming, and rhythmic mouth movements. Individual reaction totals were calculated for hedonic vs. aversive categories by adding all response scores within an affective category for that rat (hedonic, aversive, and neutral). These scores were statistically examined for drug vs. vehicle effects using ANOVA with trial as a covariate. Additionally, alerting reactions to cue presentation (head turn, step, or rear) were videoscored offline during the first 2 s of CS tones and were compared statistically between drug and vehicle (orient vs. not orient; one-way ANOVA per CS stimulus and trial covariate) (SI Materials and Methods and SI Results). M&M intake under drug was analyzed against intake under vehicle for statistical testing (one-way ANOVA).

Histology.

Cannulae tracks were marked with ink, and rats were overdosed with sodium pentobarbital at the end of the experiment. Brains were removed, fixed in 10% paraformaldehyde, cryoprotected with buffered 20% sucrose solution, sectioned coronally (60 μm), and stained for nissl substance. Maps illustrating the location of microinjection sites and electrode recording sites (Fig. S1) were constructed by identifying the spread of ink from the center of the microinjector tip placement on tissue sections and by identifying the electrode track and tip across sections (100). Separate animals (n = 10) were used to calculate spread of microinjection using Fos plume measurements used to guide functional mapping (SI Materials and Methods and SI Results).