Instant transformation of learned repulsion into motivational ‘wanting’

Summary

Background

Learned cues for pleasant rewards often elicit desire, which in addicts may become compulsive. According to the dominant view in addiction neuroscience and reinforcement modeling, such desires are the simple products of learning, coming from past association with reward outcome.

Results

We demonstrate that cravings are more than merely the product of accumulated pleasure memories: even a repulsive learned cue for unpleasantness can become suddenly desired via activation of mesocorticolimbic circuitry. Rats learned repulsion toward a Pavlovian cue (briefly-inserted metal lever) that always predicted an unpleasant Dead Sea saltiness sensation. Yet upon first re-encounter in a novel sodium depletion state to promote mesocorticolimbic reactivity (reflected by elevated Fos activation in ventral tegmentum, nucleus accumbens, ventral pallidum, and orbitofrontal prefrontal cortex), the learned cue was instantly transformed into an attractive and powerful motivational magnet. Rats jumped and gnawed on the suddenly attractive Pavlovian lever cue, despite having never yet tasted intense saltiness itself as anything other than disgusting.

Conclusions

Instant desire transformation of a learned cue contradicts views that Pavlovian desires are based essentially on previously learned values (e.g., prediction error or temporal difference models). Instead desire is re-computed at re-encounter by integrating Pavlovian information with current brain/physiological state. This powerful brain transformation reversed strong learned revulsion into avid attraction. Applied to addiction, related mesocorticolimbic transformations (e.g., drugs, neural sensitization) of cues for already pleasant drug experiences could create even more intense cravings. This cue/state transformation helps define what it means to say that addiction hijacks brain limbic circuits of natural reward.

Introduction

Learned cues for rewards (Pavlovian conditioned stimuli; CS) often trigger pulses of intense motivation to consume their associated rewards (unconditioned stimulus; UCS). The smell of food may make you suddenly feel hungry when you weren’t a minute before, and drug cues may trigger relapse in addicts trying to quit. Attribution of incentive salience to a Pavlovian reward cue can make the CS ‘wanted’ or become a tempting and attractive ‘motivational magnet’: hard to ignore, eagerly approached and sometimes ‘consumed’ similar to real reward [1–3]. Desires triggered by such Pavlovian cues seem almost entirely learned. But the purely learned appearance may be largely an illusion, at least according to incentive salience theory, because learned Pavlovian associations contribute only part of the input to computations that make the CS ‘wanted’ [4, 5] (Fig. 1). The other ‘wanting’ input comes from relevant states of brain mesocorticolimbic systems at the moment of cue re-encounter. Brain state can be modulated by many physiological factors such as natural appetite or satiety, stress, drugs, etc. A relevant change in brain state can powerfully transform incentive salience elicited by a CS.

Figure 1. Theoretical model of the synergy between learned value and mesocorticolimbic activation.

The diagram displays the impact of a sudden change in internal/mesocorticolimbic state (novel salt appetite) on the value of a Pavlovian CS according to the predictions made by incentive salience or learning prediction theory [4]. Incentive salience theory predicts that a change in internal mesocorticolimbic state would be sufficient to drastically change the reward value of a CS from negative to positive without requiring new learning (presentation of the CS alone). In contrast, learning prediction theory suggests that the change in reward value would be progressive and would require successive experiences of the CS paired with the now positive UCS.

Perhaps the strongest proof of principle for incentive salience transformation would be to demonstrate that even a repulsive Pavlovian CS, always previously associated with unpleasantness, can suddenly become a ‘wanted’ motivational magnet if re-encountered in an appropriate new state. Ideally, the transformation should come from a first reencounter in a completely novel physiological/brain state never experienced before in an individual’s life. Novelty rules out any learning-based explanations for consequent changes in motivation, via precluding opportunity to learn about values in that state.

Salt deficiency is a useful state because it is totally novel for most modern humans and laboratory rats (though frequently encountered by wild animals)[6]. In human history, the value deficiency gave to salt is signified by the word ‘salary’, which derives from Latin ‘sal’ for salt, based on the salarium paid to Roman soldiers for its purchase [7]. In states of sodium deficiency, intense saltiness becomes pleasant and associated cues become valuable [8–11]. However, it is unknown whether a CS for saltiness actually becomes transformed as suggested for ‘wanting’ computations. If so, the CS could become instantly imbued with incentive salience on first deficiency re-encounter, and so be instantly attractive and ‘wanted’ – despite always being repulsive before, and despite the salty UCS itself never having yet been tasted in the new deficiency state.

Saltiness at seawater concentration is generally unpleasant. Tastes saltier than seawater are even more unpleasant, such as three-times saltier Dead Sea concentrations of sodium chloride (Dead Sea = 9%/1.5 M NaCl plus 20% other salts). Can a cue for such intensely unpleasant saltiness ever become instantly desired? Here we used 9% Dead Sea concentration of NaCl as an unpleasant UCS (1.5 M/9% NaCl; reliably elicited disgust gapes from normal rats). In our novel autoshaping/sign-tracking paradigm, each salty UCS was infused as a pulse into a rat’s mouth via implanted cannula (because rats usually will not voluntarily drink such high NaCl concentrations). The NaCl was always predicted by a distinctive Pavlovian CS+ (referred to as CS_Salt: sudden appearance of a metal lever accompanied by an identifying sound, such as a tone). A second CS+ for sweetness (referred to as CS_Sucrose) was insertion of a different lever that emerged from the opposite wall, accompanied by a different sound (e.g., white noise), predicting infusion of palatable sucrose UCS (0.5 M/17%; reliably elicited positive hedonic reactions of lateral tongue protrusions and paw licking). A third lever served as a control CS, and predicted nothing. In order to ascertain if incentive salience transformations occur for both sign-tracking and goal-tracking phenotypes known for autoshaping, 75% of rats previously had been pre-screened in a standard autoshaping procedure (where a UCS sucrose pellet was delivered to a dish, and its predictive CS was a 4^th distinctive lever).

In their normal CS-UCS training state, all rats quickly learned to turn away and retreat from the CS_Salt cue that predicted the disgustingly salty NaCl (Video 1). Conversely, all rats rapidly learned to sign-track the CS_Sucrose (i.e., approached and nibbled the sucrose lever).

After training with at least 50 discriminative CS-UCS pairings, rats were suddenly one night put into a novel state of salt appetite via injections (deoxycorticosterone and furosemide to mimic sodium deficiency brain signals normally triggered by angiotensin II and aldosterone), which produced avid salt appetite the next day for a crucial test [12–14]. The question was: how would the rats respond toward their always-previously-nasty lever/sound CS_Salt on first re-encounter, when they had never yet re-tasted the NaCl UCS as pleasantly nice in the new state?

Results

Sodium depletion converts CS_Salt into instant motivational magnet

In a decisive behavioral test for instant motivational transformation of CS_Salt, rats were presented first with the Pavlovian CS levers alone (in extinction; with no UCS infusion occurring) in the novel salt appetite state. Rats’ behavior toward the very first presentation of the CS_Salt lever in the new state was immediately transformed into avid approach, nibbles and sniffs (F_(1,8) = 29.350, p = 0.001; Fig 2; Video 1). Rats immediately approached the CS_Salt on first appearance (Wilcoxon test, z = −2.079, p = 0.038), intensely grasped, sniffed and nibbled the metal lever within a few seconds (Wilcoxon test, z = −2.666, p = 0.008), and depressed the lever over 1000% more than on any previous day (Wilcoxon test, z = −2.524, p = 0.012). The sudden transformation was specifically triggered by insertion of the CS_Salt lever for most rats, eliciting immediate approach even when they had been distant moments before (t₍₁₁₎ = 5.354, p < 0.001). One rat approached the location even before lever insertion (Video 1; location and wall-slot are also partial CSs), though others waited until first lever presentation, and all rats remained within 8 cm after the first appearance (in contrast to avoidance on previous training days) (Wilcoxon test, z = −2.521, p = 0.012). The instant attraction occurred for essentially all rats, although none had yet tasted the NaCl UCS as positive in the new state, and all had previously avoided the location on all earlier days (during training, CS_Salt lever reliably evoked repulsion: turning away and sometimes keeping pressed against the opposite wall; Video 1) (F_(1,8) = 58.542, p < 0.0001). Subsequent presentations of the CS_Salt lever on the appetite day elicited the same ‘wanting’ pattern, sometimes even more strongly.

Figure 2. Instant transformation of the Salt CS from disgusting and avoided into an attractive motivational magnet.

The overall intensity of motivated behaviors is shown on each trial (total number of appetitive-consumatory behaviors (eg: approach, sniff, nibbles) minus aversive behaviors (avoidance)) per CS_Salt presentation (red circles) or CS_Sucrose presentation (grey circles). Effects of transition are shown across different internal physiological/mesocorticolimbic conditions (Homeostasis [Day11], Sodium depletion [Day13], Sodium re-repletion [Day18]). On the very first presentation of the CS_Salt in extinction (CS1 - CS ONLY Phase), at a time when the triple seawater UCS has never been experienced as anything other than highly disgusting, CS_Salt suddenly becomes a powerful motivational magnet. In contrast, motivated behaviors towards CS_Sucrose remain unchanged. In a subsequent test (CS-UCS Phase), following each CS_Salt presentation by triple seawater solution that has now become strongly ‘liked’ does not further increase the motivational value of the cue. After then returning to normal physiological sodium levels (Sodium Re-repleted), the value of the CS_Salt in extinction instantly decreases to levels similar to those prior to the induction of novel salt appetite. Data are represented as mean +/− SEM. (See also Video S1).

Regarding autoshaping phenotypes, the instant transformation of CS_Salt lever into motivationally-attractive magnet occurred equally for all rats in the group, regardless of whether they had previously been ascertained to be sign-trackers or goal trackers when pre-screened in a traditional autoshaping procedure (i.e., with sucrose pellet UCS that required voluntary approach and ingestion at a goal location different from the CS). Therefore, we conclude that instant CS transformation of incentive salience may occur in traditional goal-trackers as well as in sign-trackers (at least when discriminative CS-UCS associations are formed in a pure Pavlovian procedure such as ours, where UCS solutions arrived automatically in the mouth without needing any instrumental action or active goal approach).

The CS_Sucrose lever by comparison always evoked high levels of approach and consummatory nibble-and-sniffs regardless of normal training vs appetite test states (Wilcoxon test, Appetitive: z = −1.599, p = 0.110; Aversive: z = −0.690, p = 0.490; Fig. 2; Video 1). No increase in approach to CS_Sucrose lever was induced by the new sodium depletion state (F_(1,10) = 0.520, p = 0.487). A third CS_Control lever that predicted nothing elicited nearly zero approaches on all days, with no enhancement by new depletion state (Wilcoxon test, z = −1.116, p = 0.265). It remains possible that the motivational transformation of the CS_Salt lever was aided by previous autoshaping to the CS_Sucrose lever. For example, psychological attribution of incentive salience that allows a metal lever to be perceived as attractive may have been facilitated, opening the way to similar attributions to a new lever later. However the sudden transformation of CS_Salt lever was still quite specific. For example, no enhancement was transferred onto the 3^rd control lever that predicted nothing. Thus there was clearly a special synergy between CS_Salt and sodium appetite state that controlled the direction of the motivational transformation, and created a specific motivational magnet.

A conditioned alliesthesia reaction (state/learning generation of hedonic palatability) was also evoked by the CS_Salt in over 80% of rats, reflected in elicitation of positive hedonic or ‘liking’ orofacial reactions near the end of CS_Salt presentations in the novel state (t₍₁₁₎ = 3.208, p = 0.008; Fig. 3)

Figure 3. CS_Salt in a novel salt appetite produces conditioned hedonic taste reactivity and becomes a nibbled and sniffed motivational magnet.

A, Conditioned Taste reactivity to CS_Salt showing hedonic (tongue protrusions, paw licking [blue circles/’liking’]) and aversive (gapes [red circles/’disgust’]) reactions to the presentation of the CS_Salt cue in extinction (no UCS salt solution) in normal homeostatic physiological state and in novel sodium appetite. Grey triangles represent hedonic responses to CS_Sucrose. B, Unconditioned Taste reactivity to 9% Dead Sea Salt UCS infusions showing hedonic (tongue protrusions, paw licking [blue circles/’liking’]) and aversive (gapes [red circles/’disgust’]) reactions in subsequent CS-UCS reinforced trials in normal homeostatic physiological state and in novel sodium appetite. Grey triangles represent hedonic ‘liking’ responses to 17% Sucrose UCS. C, Appetitive (sniffs, nibbles) reactions towards the CS_Salt and CS_Sucrose in extinction (no UCS) under normal homeostatic physiological state and in novel sodium appetite. Data are represented as mean +/− SEM.

In summary, an intense and immediate transformation of CS incentive salience was induced by the first combination of external Pavlovian lever and internal depletion state. New ‘wanting’ was specifically targeted to the CS_Salt, and the cue transformation occurred in advance of any re-valuation experience with the UCS. Thus clearly no re-learning about the improved hedonic value of NaCl taste was required to make its CS suddenly ‘wanted’.

Subsequent hedonic reactions to UCS confirm alliesthesia flip

Later on the same day of novel depletion state we confirmed that palatability of the intensely salty UCS flipped to positively hedonic or ‘liked’ (e.g., eliciting lateral tongue protrusions; Fig. 3), in a round of reinforced CS-UCS trials subsequent to the extinction CS tests. Infusions of 1.5 M/9% NaCl solution into the rat’s mouth elicited mostly positive hedonic reactions, at levels 40 times higher than any previous day (t₍₁₁₎ = 6.050, p = 0.000), and 6 times higher than to the CS_Salt alone in extinction on the same day. At the same time, aversive disgust reactions to NaCl were cut to less than half of previous levels (t₍₁₁₎ = 5.358, p = 0.0001; Fig. 3).

We also independently confirmed induction of salt appetite later that night, using a traditional test of voluntary intake beginning 24 hours after injections (3% NaCl solution; overnight access plus water and food). A 775% increase in the amount of voluntary NaCl consumed in home cage was induced by the salt appetite treatment (t₍₁₁₎ = 6.745, p = 0.000; 20.67 ml NaCl sodium deficient vs. 2.67 ml NaCl normal state). NaCl intake gradually declined back to initial baseline levels over the next 2–5 days as bodily sodium homeostasis was restored.

Finally, another CS-only or extinction test, similar to the novel state test, was performed after several days recovery of sodium homeostasis. Results confirmed that the motivation value of the CS_Salt lever partly flipped back to negatively repulsive again when sodium homeostasis was regained (Depleted to Re-repleted: Wilcoxon test, z = −2.549, p = 0.011; Fig. 2). The flip back to repulsion occurred even though the rats had not re-tasted a 9%/1.5 M concentration of NaCl since their sodium depletion test day. This flip back confirmed again that the re-computation of CS_Salt incentive salience was state-dependent. In other words, to make the CS_Salt positively ‘wanted’ required the synergistic combined presence of both external Pavlovian stimulus (CS_Salt) and internal physiological stimulus (depletion state).

Mesocorticolimbic Fos expression to cue plus novel state

To identify brain systems recruited by the instant transformation of CS_Salt value, the expression of Fos protein in the brain was assessed in separate rats under four conditions matched to the procedures above: 1) CS_Salt presentations in extinction during a novel state of salt appetite, 2) novel salt appetite alone (no CS or UCS), 3) UCS retasting of NaCl during novel salt appetite, or 4) normal homeostatic physiological state (control baseline group).

Dramatic increases in neuronal Fos expression within mesocorticolimbic structures were recruited specifically by the synergistic combination of CS_Salt plus simultaneous salt appetite state (Fig. 4). Highest 1700% increases in neuronal Fos to this combination were seen in the nucleus accumbens, especially in the rostral half of its medial shell component (compared to normal control baseline levels) (t₍₅₎ = 5.163, p = 0.004; Fig. 4). The rostral half of medial shell is the same region that contains a ‘hedonic hotspot’ capable of neurochemically magnifying hedonic impact of pleasant taste [15–17]. Intense increases in Fos were also observed throughout most of the core of nucleus accumbens (t₍₅₎ = 2.880, p = 0.035). Less intense tripling of Fos was seen in the caudal half of the medial shell (t₍₁₀₎ = 2.365, p = 0.039). Outside the nucleus accumbens, tripling or higher increases in Fos were observed in limbic regions of prefrontal cortex, especially in orbitofrontal (>333%; t₍₅₎ = 1.930 p = 0.111) and infralimbic regions (homologous to deeply ventral anterior cingulate cortex in humans; >550%; t₍₅₎ = 3.318, p = 0.021). Subcortically, >600% elevation was also observed in the rostral half of ventral pallidum (t₍₅₎ = 4.501, p = 0.006), and >450% elevation in the midbrain ventral tegmentum area that contains dopamine neurons (t₍₅₎ = 2.981, p = 0.033; Fig. 4).

Figure 4. Presentation of CS_Salt in a novel salt appetite increases mesocorticolimbic Fos activation.

Fos activation in mesocorticolimbic circuit following either 1) presentation of CS_Salt cue in a novel salt appetite in extinction [Sodium Deplete + CS_Salt], 2) retasting of NaCl UCS during novel salt appetite [Sodium Deplete + Salt], 3) novel salt appetite alone (no CS or UCS)[Sodium Deplete], or 4) normal homeostatic physiological state (control baseline group) [Control]. Colors represent the percentage increase in Fos activation in Sodium Deplete + CS_Salt condition for each brain region in comparison to control baseline group. Arrows inside each bar graph represent the percent increase in Fos activation from Sodium Deplete to Sodium Deplete + CS_Salt condition. Data are represented as mean +/− SEM.

Sodium depletion state alone (without the external Pavlovian CS_Salt) produced intermediate increases in Fos expression, lower than those above, and in fewer structures. We observed >500% increase in the nucleus accumbens core (t₍₁₀₎ = 2.657, p = 0.025), >250% increase in the infralimbic region of prefrontal cortex (t₍₁₀₎ = 3.175, p = 0.010) and >300% increase in lateral hypothalamus (Sodium Depletion: t₍₁₀₎ = 1.512, p = 0.162) during the salt appetite state alone (no Pavlovian CS_Salt).

Adding the UCS of NaCl re-tasting and ingestion to the appetite state actually produced a suppressive trend toward reducing Fos in lateral hypothalamus (>35% suppression of depletion alone; t₍₁₅₎ = 0.877, p = 0.394), similar to the pattern reported by Liedtke et al. [12], and in the orbitofrontal cortex (>60% suppression; t₍₁₅₎ = 1.769, p = 0.097). Conversely, after retasting NaCl UCS in the deficient state, moderate increases were seen in the nucleus accumbens: rostral medial shell (>550%; t₍₁₇₎ = 2.375, p = 0.043), caudal medial shell (>325%; t₍₁₈₎ = 2.365, p = 0.039) and core (>525%; t₍₁₇₎ = 2.657, p = 0.025), in both rostral and caudal ventral pallidum (rostral: >262%; t₍₉₎ = 2.197, p = 0.058; caudal: >318%; t₍₁₁₎ = 2.216, p = 0.050; ) and in infralimbic region of medial prefrontal cortex (>195%; t ₍₁₈₎ = 1.909, p = 0.089; see Fig. 4).

Discussion

The instant transformation of incentive salience for the CS_Salt highlights the critical role played by moment-to-moment internal states in generating motivation for Pavlovian cues (Fig. 1). The transformation occurred on the very first re-encounter of the metal lever cue for saltiness, despite its previous association with purely disgusting experiences. It occurred even though rats had never yet tasted the intense Dead Sea saltiness UCS of NaCl itself as positively ‘liked’, and without requiring any new re-learning of CS-UCS values in the new state. Mesocorticolimbic brain circuitry recruited at the same moment by the synergistic re-encounter provides a potential neurobiological mechanism to explain the psychological transformation of motivation.

We note that our motivation transformation contrasts to some previous reports that rats have failed to cognitively infer a higher value for salt or to instrumentally pursue actions that would obtain NaCl when tested in novel salt appetite state, failures that correspond to model-based reinforcement computations when models lack any experience-gained knowledge about new salt value [18, 19]. Such failures to transform occur especially when decisions are guided primarily by memories of previous act-outcome reward values [18, 19], in accordance with the logical assumption that past displeasure predicts future low value. Such value-based reinforcement computation and decisions are switched only by allowing re-tasting of NaCl in the appetite state to gain knowledge about the new positive value [19, 20].

We suggest a crucial feature of the instant desire transformation demonstrated here, which did not require re-learning, is the presence of a distinctive Pavlovian cue (CS_Salt) that can be transformed in incentive salience by mesocorticolimbic systems to serve as a motivational magnet. More similar to our demonstration are reports that sodium depletion can directly increase rats’ pressing on a lever distinctively paired with NaCl (which combines Pavlovian and instrumental associations to the lever) [10], increase consumption of an almond or banana solution previously paired with NaCl (flavor as Pavlovian CS) [8], or increase immediate return to a place or environment previously associated with NaCl as UCS (location/context as Pavlovian CS) [9, 21]. Still, it was never clear whether those CSs actually became positively ‘wanted’ incentives with instant motivational magnetic properties, or instead simply signaled a possible route to alleviate distress. It also was not clear until now whether an instant transformation is powerful enough to reverse intense learned repulsion (such as to a CS for Dead Sea concentrations of 9% NaCl) into instant strong desire. Our results show that both do happen: a CS instantly gains positive incentive salience, and the transformation is powerful enough to reverse cue value from strongly negative to strongly positive.

Biological mechanisms underlying transformation of CS ‘wanting’

Natural physiological transformations of incentive salience are evolutionarily adaptive in the wild. For example, Kenyan elephants after previously chewing NaCl-containing rocks in a volcanic cave (UCS) are reported to follow the wafting odor of smoke (Pavlovian CS) from that erupting volcano back to the same mountain to find salt again [22]. Natural sodium deficiency produces elevations in blood-born aldosterone and angiotensin II [6]. In the brain, aldosterone stimulates hormone receptors of neurons in extended amygdala structures, such as the amygdale central nucleus and bed nucleus of stria terminalis, and in the hindbrain nucleus of the solitary tract [6, 23, 24]. Angiotensin II stimulates thirst-related receptors of neurons in the subfornical organ and ventral forebrain [12, 25]. The generation of appetite motivation requires mesocorticolimbic participation, such as elevation of dopamine (reduced dopamine transporter binding) and opioid (enkephalin mRNA) signals in nucleus accumbens and striatum, and enhanced neuronal reactivity to relevant cues in ventral pallidum [13, 26]. Much of this brain reward circuitry was also recruited here by the CS_Salt re-encounter in the novel salt appetite state, reflected by up to ten-fold increases in Fos expression in nucleus accumbens, ventral pallidum, ventral tegmentum and limbic prefrontal cortex.

Psychological processes mediating transformation

Psychologically, the transformation of incentive salience afresh on CS_Salt re-encounter requires model-based information, but involving a Pavlovian sensory memory of saltiness that is quite distinct from model-based information about prior values (the only value memory here was previous unpleasantness) [27, 28]. This sensory-model makes the incentive salience transformation quite different from most model-based reinforcement computations that require the model to hold experience-gained information about positive reward value in some previously-experienced state [29]. Here, only the sensory association between CS_Salt and UCS gustatory saltiness could be used to freshly generate incentive salience upon cue re-encounter.

The generation of CS value was based on the new positive value that UCS saltiness sensation would have in the appetite state, even though the actual NaCl had not yet been re-tasted as positive. This sensory-memory transformation into positive value was probably responsible also for the conditioned alliesthesia or positive taste ‘liking’ hedonic orofacial reactions that were elicited by CS_Salt in the new appetite state before the NaCl ever came [14, 30, 31].

Computationally, this synergistic transformation of CS_Salt motivational value can be described by the incentive salience model of Zhang and colleagues [4]. In that computational model, the incentive salience of CS_Salt is called Ṽ(S_t) (S denotes the Pavlovian CS stimulus; the moment of cue re-encounter is denoted as t for time). Ṽ(S_t) is computed as: Ṽ(S_t) = r̃(r_t+logκ)+γV(s_t+1). The current mesocorticolimbic brain state reflecting sodium appetite state is represented in the model by a gain-control factor κ (kappa), which transforms the current incentive salience from previously learned values. The previously learned Pavlovian association is (r_t) derived from a temporal difference model, where γ is a discounting parameter for events more distant in future.

Incentive salience (Ṽ(S_t)) (on the left side of equation) is generated dynamically at the moment of cue-reencounter by logarithmically combining the previously established (r_t) memory and the current κ state factor. If current state remained similar to training state, then κ = 1, which preserves the learned value of CS_Salt as negative. But in the new salt appetite state the kappa factor grows: κ≫1. Consequently, in the first CS_Salt re-encounter in the novel κ state, the incentive salience is logarithmically transformed to a positive value of Ṽ(S_t)(Fig. 1). In that novel state, the previously repulsive and disgust-associated CS_Salt is suddenly attractive, approached, sniffed and nibbled as a ‘wanted’ salty Pavlovian incentive.

Relevance to Addiction

A dominant view in addiction neuroscience and reinforcement learning models of the past decade has been that the motivating value of a learned cue comes solely from its past association with rewarding outcomes [29, 32–34]. For example, Wise nicely expressed that view: "It is only after the sight of food or a response lever has been associated with the reinforcing effects of that food or an addictive drug that the food or lever becomes an incentive motivational stimulus that can itself stimulate craving and elicit approach. The argument here is that it is yesterday’s reinforcing effects of a particular food or drug that establishes today’s cravings for that food or drug”(p 5)[33]. More computationally, Schultz concurs: "In learning situations governed only by experienced rewards, consecutive unrewarded trials lead to progressively decreasing reward prediction”(p4)[34].

By contrast, the argument here is that cravings today (for a salty cue) can far exceed the level of reinforcing effects on all previous yesterdays (salty disgust). Our results show that consecutive unrewarding trials (or even punishing trials) with a CS can still lead to that cue triggering intensely high levels of ’wanting’ in a new state, no matter how low (or even negative) the Pavlovian prediction of previously learned value. Instant transformation in motivation value of a learned Pavlovian cue is powerful and real, even if transformation contradicts views of reinforcement based on experientially-learned values, which are the centerpiece of addiction-learning neuroscience approaches today [29, 32–34].

We suggest that the lesson to be drawn for addiction is: if brain mesocorticolimbic activation can transform learned negative revulsion into strong positive ‘wanting’ triggered by a cue for disgusting saltiness, how much more intense could mesocorticolimbic-amplified ‘wanting’ become when triggered by cues for drugs, food, sex, gambling, and related already-pleasant experiences? As posited by the incentive sensitization hypothesis of addiction, such mesocorticolimbic amplifications of incentive salience create compulsively intense levels of motivation in drug addiction [2]. Drugs could even become ‘wanted’ under conditions where their experience is known to be unpleasant, (similar to the salty cue). Incentive salience transformation as seen here helps define what it means to say that addiction hijacks brain limbic circuits of natural reward [2, 3, 35, 36].

Experimental Procedures

The University Committee on the Use and Care of Animals of the University of Michigan approved all experimental methods performed in this research. These studies were conducted with female Sprague-Dawley rats (250–325g, behavior: N = 12, immunoreactivity: N = 21). To permit oral solution infusions, rats were anesthetized and implanted with oral cannula following methods described in detail elsewhere [37]. Pavlovian conditioning was carried out in standard Med Associates® operant chambers as described in detail elsewhere [3]. Most rats (75%) were initially prescreened on a standard autoshaping task using voluntary intake of sucrose pellet UCS to determine whether they were goal-trackers or sign-trackers [3]. Pre-screening did not alter subsequent behavior to CS_Sucrose or CS_Salt in the oral-delivery autoshaping tests, so results from all rats were combined (F_(1,10) = 1.826, p = 0.206 for CS-UCS reinforced Baseline/Homeostasis test). Behavioral procedures consisted of blocked training of CS-UCS presentations, where CS_salt and CS_sucrose levers were diagonally located on opposite walls of the chamber and respectively predicted infusions of hypertonic NaCl (1.5 M; 9% NaCl) or sucrose (0.5 M; 17.1 %) solution as UCS. Test days (Baseline Homeostasis, Sodium Depleted, Sodium Re-repleted) consisted of CS+ only extinction tests and CS-UCS reinforced tests. All behaviors during tests, including UCS elicited taste reactivity behaviors were video recorded and subsequently scored in slow motion in a manner previously described [3, 37]. Salt appetite was induced within 24 hours by injection of the diuretic furosemide (7.5 mg/kg, sc; Hospira) and deoxycorticosterone (DOCA, 1 mg/kg in propylene glycol, sc; Sigma Aldrich) [11]. Fos immunofluorescence was assessed in separate animals under four separate conditions (1- CS_Salt + novel salt appetite; 2- UCS retasting of 0.5 M/3% NaCl + novel salt appetite; 3- novel salt appetite alone; 4- normal homeostatic physiological state) following procedures described elsewhere [38, 39]. For more details, please refer to Supplemental Experimental Procedures.

Highlights.

• -Motivation is the product of past experience and current mesocorticolimbic state
• -Novel appetite state transforms a repulsive salt cue into a motivational magnet
• -The cue becomes avidly ‘wanted’ despite knowledge the salt always tasted disgusting
• -This dynamic transformation recruits brain mesocorticolimbic circuitry