Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Feb 1.
Published in final edited form as: Curr Opin Behav Sci. 2017 Jan 5;13:178–185. doi: 10.1016/j.cobeha.2016.12.004

Neurobiological Basis of Individual Variation in Stimulus-Reward Learning

Shelly B Flagel a, Terry E Robinson b
PMCID: PMC5486979  NIHMSID: NIHMS837241  PMID: 28670608

Abstract

Cues in the environment can guide behavior in adaptive ways, leading one towards valuable resources such as food, water, or a potential mate. However, cues in the environment may also serve as powerful motivators that lead to maladaptive patterns of behavior, such as addiction. Importantly, and central to this article, there is considerable individual variation in the extent to which reward cues gain motivational control over behavior. Here we describe an animal model that captures this individual variation, allowing us to better understand the psychological and neurobiological processes that contribute to cue-evoked behaviors. When a discrete cue is paired with a food reward in a Pavlovian manner it acquires greater control over motivated behavior in some rats (“sign-trackers, STs) than in others (“goal-trackers”, GTs). We review studies that have exploited this animal model to parse the neurobiological mechanisms involved in learning associations between stimuli vs. those involved in attributing incentive salience to those same stimuli. The latter seems to be dependent on dopamine and subcortical circuits, whereas the former may engage more cortical “top-down” mechanisms.

Introduction

Cues (conditional stimuli, CSs) that predict the impending delivery of biologically significant events (unconditional stimuli, USs), such as a food reward, acquire the ability to control behavior, or produce a conditioned response (CR), via Pavlovian learning mechanisms [1]. The same is true for stimuli associated with aversive events, but here we will focus only on cues associated with rewards. The ability of a CS to evoke simple reflexive CRs, such as salivation in the case of Pavlov’s dogs, is well known. It is less well appreciated, however, that CSs can also acquire the ability to evoke complex emotional and motivational states [2–5]. This latter transformation is thought to occur if a CS is attributed with incentive salience and thus acquires the properties of an incentive stimulus [2–4, 6]. Incentive stimuli: 1) bias attention towards them and can elicit approach into close proximity with them; 2) become desirable themselves, in the sense that an animal will work for access to the stimulus alone (i.e., they act as conditioned reinforcers); and 3) can instigate or invigorate reward-seeking behavior (as in Pavlovian-to-instrumental transfer effects, PIT). Incentive stimuli can guide behavior in adaptive ways, leading one towards valuable resources such as food, water, or a potential mate. However, such cues may also serve as powerful motivators that lead to maladaptive patterns of behavior, as in over-eating and addiction. Importantly, there is considerable individual variation in the extent to which CSs act as incentive stimuli and gain motivational control over behavior [7].

In the laboratory, if a discrete and localizable Pavlovian CS, such as presentation of a lever, is reliably paired with presentation of a food reward, some rats come to approach the CS (Figure 1), even though no response is required for delivery of the reward [8]. This is called “sign-tracking” [9, 10]. In contrast, upon presentation of the lever-CS, other rats go to the location of impending reward delivery (Figure 1) [8]. This CR is called “goal-tracking” [11]. It is important to note that the lever-CS is an equally effective predictive stimulus (CS) in both sign-trackers (STs) and goal-trackers (GTs) – they learn their respective CRs at the same rate – but only in STs does the lever-CS acquire the properties of an incentive stimulus [12]. That is, for STs, the CS is more attractive and elicits approach towards it, is a more effective conditioned reinforcer, and is more effective at instigating reward-seeking behavior relative to GTs [12–14]. Importantly, this variation in the ability of a CS to acquire incentive salience is captured best by a localizable CS (i.e. lever or light) [15], and is not apparent when the CS is a tone [16]. Furthermore, tone stimuli paired with a food reward are attributed with less incentive value than a lever-CS [16, 17].

Figure 1. Characterization of a “sign-tracker” and “goal-tracker”.

Figure 1

Open in a new tab

A) Sign-trackers (STs) approach and manipulate the lever (conditional stimulus, CS), reflective of incentive salience attribution. B) Goaltrackers (GTs) approach the location of food reward (unconditional stimulus, US) delivery upon presentation of the lever-CS. Images adapted from [58]. C) Mean +/− SEM Pavlovian conditioned Approach Index, a composite score used to assess the propensity of an individual rat to approach the lever-CS vs. the food cup (see [59]), is shown across 5 conditioning sessions for GTs (n=1867), those in the intermediate group (IG, n=2296) that vacillate between the two responses, and STs (n=1934). An Approach Index of −1 indicates behavior directly solely towards the food cup, whereas that of +1 indicates that behavior is directly solely towards the lever-CS. D) A histogram illustrating the population distribution of the propensity to attribute incentive salience to a reward cue in 6097 rats (the same rats used for panel C). Phenotype classification is based on an Approach Index of −1 to −0.5 for GTs, −0.5 to 0.5 for IG and 0.5 to 1 for STs. The large population of rats used for C and D has come from a database generated using Sprague-Dawley rats that have been screened for Pavlovian conditioned approach behavior in the labs of Drs. Shelly Flagel, Jonathan Morrow and Terry Robinson at the University of Michigan.

Sign-trackers are also more resistant to extinction of their CR than GTs [18] and will continue to approach the CS even if contact with it results in omission of the reward [19], indicating that approach behavior is not contingent upon subsequent food delivery or maintained via response reinforcement processes [20]. It has also been shown that sign-tracking behavior becomes more pronounced when the relationship between the CS and the reward is uncertain, such that the probability of the reward following CS presentation changes [21]. These findings provide further evidence for dissociation between the predictive vs. incentive value of the CS (see also [22]). Thus, a lever-CS acquires all of the properties of an incentive stimulus in some individuals (STs), but not others (GTs). Importantly, this individual variation in the propensity to attribute incentive salience to a discrete or localizable CS has been described not only for food predictive cues, but also for cues that predict drug rewards [7, 15]}. The current article will focus on what we have learned about the neurobiological mechanisms of stimulus-reward learning by exploiting this individual variation.

Elucidating the Neurobiological Mechanisms Underlying Individual Differences in Stimulus-Reward Learning

Dopamine

There has been considerable research on the role of dopamine (DA) in stimulus-reward learning, and one popular hypothesis is that phasic DA signals serve as a prediction error signal necessary for learning associations (for recent review see [22]). Given that STs and GTs learn CS-US associations equally well, but differ in the degree to which they attribute incentive salience to the CS [12], we have used this animal model to parse the role of dopamine in stimulus-reward learning [23]. The classic evidence that DA provides a prediction error signal is the observation that a phasic DA signal transfers from the US to the CS over the course of learning [24]. We used in vivo voltammetry in the core of the nucleus accumbens to see if this shift in DA occurred as STs and GTs learned their respective CRs [23]. We found that the transfer of DA occurs only in rats that learned a sign-tracking CR, suggesting that the role of DA is to encode the incentive properties of reward cues, not the predictive properties [23] (for additional discussion see [22]). In agreement, Parker et al. [25] found that mice with disrupted phasic dopamine signaling were perfectly capable of learning a goal-tracking CR, even though there was no apparent transfer in dopamine signaling from the US to the CS.

Furthermore, a series of pharmacological studies support the notion that dopamine signaling is important for sign-tracking, but not goal-tracking behavior. Systemic injections of flupenthixol, a non-specific dopamine receptor antagonist, blocked the acquisition of a ST CR, but had no effect on the acquisition of a goal-tracking CR [23]. Similarly, flupenthixol administered directly into the core of the nucleus accumbens [26] attenuated the performance of an already learned sign-tracking CR, but not a goal-tracking CR. This effect appears to be due to DA antagonism degrading the incentive value of the cue, which is necessary for it to elicit approach behavior, because the effect was evident on the very first trial and therefore could not be due to new learning [26]. In support, DA antagonism had no effect on the performance of a different CR, conditioned orienting behavior, in either STs or GTs [27]. This latter finding suggests that DA blockade does not degrade the CS-US association, even in STs. Lastly, flupenthixol administered in the core of the accumbens also attenuates cue-evoked reinstatement of cocaine-seeking and does so preferentially in STs [28]; while enhancing DA activity by local injection of amphetamine increases both cue-evoked reinstatement of cocaine-seeking [28], and approach to a lever-CS [29] in STs. Taken together, these findings demonstrate that dopamine signaling, specifically in the core of the nucleus accumbens (Figure 2), is critical for the attribution of incentive salience to reward cues, and not for encoding the predictive value.

Figure 2. Sagittal schematic of the brain areas that are involved in reward processing for incentive stimuli.

Figure 2

Open in a new tab

Brain regions highlighted in gray are those that show enhanced cue-induced activation in response to either a food- or drug-cue that has been attributed with incentive salience, but not both; those highlighted in yellow show enhanced activation to both food- and drug-associated cues that have been attributed with incentive salience; and those highlighted in red show enhanced activation in response to a food- or drug-cue, and have also been implicated in incentive salience attribution using other methods (i.e., lesions (PVT, BLA, lateral habenula), chemogenetics (VP), electrophysiology (VP)). Regions that are identified, but not highlighted are those that are believed to be involved in incentive salience attribution, but published data is currently lacking. Abbreviations: BLA, basolateral nucleus of the amygdala; CeA, central nucleus of the amygdala; CeM, central medial nucleus of the thalamus; HC, hippocampus; IMD, intermediodorsal nucleus of the thalamus; LH, lateral hypothalamus; MeA, medial nucleus of the amygdala; PVT, paraventricular nucleus of the thalamus; VP, ventral pallidum Image adapted from [27].

One source of variation in DA signaling in STs and GTs may be that these phenotypes differ in DA transporter (DAT) and/or dopamine receptor expression and function [29–31]. Indeed, STs show greater surface DAT expression in the ventral striatum, and this is associated with faster DA uptake in the core of the accumbens. Also, systemic amphetamine administration inhibits DA uptake to a greater extent in STs than GTs [29], which could contribute to its selective effects on approach and cue-induced reinstatement described above [28]. There are also data to suggest that STs have lower levels of dopamine D2 receptor mRNA in the nucleus accumbens [30, 31], and pharmacological studies suggest that antagonism of this receptor disrupts sign-tracking, without affecting goal-tracking behavior [32]. However, the expression of both sign- and goal-tracking CRs were attenuated with systemic administration of flupenthixol [23] or antagonists acting with greater selectivity at D2/D3 receptors [33, 34], and agonists of these receptors have similar effects [33]. Thus, while dopamine signaling clearly plays a role in the propensity to attribute incentive salience to reward cues, it is not yet clear which receptors are mediating these signals; although recent evidence suggests that D3 and D4 receptors are not selectively involved [33, 35]. Further investigation is also needed to determine whether or not inherent differences in expression patterns or efficacy of the dopamine system are playing a role. Moreover, the role of dopamine and/or which dopamine receptors are involved changes over the course of training [35], such that effects on acquisition may differ from those on the expression of the conditioned response [36].

Circuit Level Differences

There have been numerous studies examining the neural systems involved in cue-motivated behaviors, most of which support the notion that there is a widespread and overlapping network of “reward circuits” mediating the response to many different classes of rewards (e.g., food, sex, drugs) and reward cues (e.g., discrete cues, contexts) (for review see [4, 37–39]). This “motive circuit” includes cortico-striato-pallido-thalamic loops with cortical and subcortical networks converging on the nucleus accumbens [37–40]. What has been less clear, however, is exactly which properties of a reward cue are responsible for activating this circuit. In most learning studies the predictive and incentive value of cues are confounded, and they tend to change together, making it difficult to separate the neurobiological processes involved in learning associations vs. those involved in attributing incentive salience to a cue. However, the sign-tracker/goal-tracker model allows us to do this [12], and here we describe initial studies in which we have exploited this individual variation to better understand the circuitry underlying these different forms of stimulus-reward learning.

One of our first efforts was to determine which brain regions are engaged by presentation of a food cue in STs and GTs, using c-fos as a marker of neuronal activity [41]. Interestingly, a food cue induced c-fos expression throughout the so-called motive circuit in STs but not GTs, including the dorsal (caudate-putamen) and ventral striatum (accumbens), lateral habenula and midline thalamus, and similar effects have been described upon presentation of a drug (opioid) cue [27] (Figure 2). These findings demonstrate that the predictive value of a cue is not sufficient to engage these brain regions; it must also be attributed with incentive salience.

Furthermore, an examination of interregional cue-induced c-fos mRNA levels within each phenotype revealed distinct patterns of correlated neural activity, with subcortical patterns of activation in STs, and evidence for cortical engagement in GTs [41]. In particular, STs showed correlated cue-induced activity between the paraventricular nucleus of the thalamus (PVT) and the ventral striatum; whereas GTs showed a strong pattern of correlated activity between the prefrontal cortex (PFC) and paraventricular nucleus of the thalamus (PVT) [41, 42]. These findings, and the fact that the PVT shows pronounced differences in cue-induced c-fos activity between STs and GTs [27, 41] (Figure 2), has prompted further investigation into the role of the PVT and related circuitry in sign- and goal-tracking behaviors [42, 43].

When the PVT is lesioned prior to the acquisition of a conditioned response, the initial learning of the CS-US association is not affected, but the differences in the expressed CR are amplified, with an increase in sign-tracking behavior in STs and decrease in goal-tracking behavior in GTs [43]. When lesions occur after rats have acquired a CR, effects are only evident in GTs who, following PVT lesions, show increased sign-tracking behavior [43]. These results suggest that the PVT may act to suppress the attribution of incentive salience to reward cues, at least in GTs, as disruption of the functional activity of this nucleus enhances the tendency to sign-track.

To better understand the circuitry surrounding the PVT in these cue-motivated behaviors, we conducted dual-labelling (c-fos and flourogold) studies to identify which cells communicating with the PVT are activated in response to a food cue [44]. The findings suggest that the pathway from the prelimbic cortex (PrL) to the PVT may play a primary role in encoding the predictive properties of reward cues; whereas subcortical activity – both inputs to the PVT from amygdalar and hypothalamic subregions (Figure 2), and outputs from the PVT to the ventral striatum – are mediating the attribution of incentive salience to reward cues.

Using chemogenetics, we (SBF) are now targeting some of these circuits to further explore their role in sign- and goal-tracking behaviors. We are especially interested in the PrL-PVT circuit, as depletion of serotonin in the mPFC enhances sign-tracking behavior in C57 mice [45]; and deficits in mPFC cholinergic activity have been associated with poor attentional control in STs [46]. STs are also more impulsive than GTs [30, 47], another trait inherent to these phenotypes that may be mediated by aberrant top-down executive control. Thus, we hypothesize that the PrL-PVT circuit may act to inhibit incentive salience attribution, and while this is effective in GTs, in STs the subcortical drive may override the cortical control, contributing to maladaptive tendencies.

Other Regions of Interest

The findings reviewed above which are, admittedly, primarily from our own work, provide evidence for the involvement of dopamine and specific nodes of the “motive circuit” in the attribution of incentive salience to reward cues. Support for these findings comes from other studies, such as that by DiFeliceantonio and Berridge [48], demonstrating that the dorsolateral striatum can act to enhance the incentive salience of a Pavlovian reward cue. It is important to note that other brain regions and neurotransmitter systems, known to interact with the dopamine system (for review see [40]), have also been studied within this realm.

One such region is the lateral habenula, an area that showed enhanced cue-induced c-fos activity in STs relative to GTs [27, 41]. DA neuron firing is strongly modulated by inputs from the lateral habenula (LHb), such that increasing LHb activity decreases DA activity, whereas decreasing LHb activity does the opposite (for review see [49]). In an interesting study, Danna and colleagues [50] showed that lesions of the LHb outputs (increased DA) increased sign-tracking; whereas stimulation of the same outputs (decreased DA) attenuated sign-tracking. Neither manipulation had any influence on goal-tracking. These data further support the idea that DA is important for the performance of sign-tracking, but not goal-tracking behavior, and demonstrate a role for the lateral habenula in incentive salience attribution (Figure 2).

Another region that showed robust Fos expression in STs relative to GTs in response to both a food and opioid cue is the basolateral amygdala (BLA; [27], Figure 2). Consistent with this finding, it has been shown that lesions of the BLA attenuate sign-tracking behavior after the CR is acquired [51]. Interestingly, in the same study, it was shown that lesions of the nucleus accumbens impaired the initial acquisition of sign-tracking behavior, and when communication between the BLA and nucleus accumbens was disrupted with disconnection lesions, there were deficits in both the initial acquisition and expression of sign-tracking behavior [51]. These findings suggest that different brain regions within a circuit may mediate different aspects (i.e. acquisition vs. performance) of stimulus-reward learning.

In recent years, the ventral pallidum has become a focus of great interest as a mechanism of incentive motivation [52, 53]. In support, Chang and colleagues [54], using a chemogenetic approach, showed that transient disruption of neurons in the ventral pallidum impaired the acquisition of sign-tracking, but did not affect goal-tracking. Furthermore, using single-unit in vivo electrophysiology, Ahrens et al. [55], recently demonstrated that, during CS presentation, there are a greater number of responsive neurons in the ventral pallidum, and they show greater changes in activity in STs than GTs. Furthermore, this activity is strongly correlated with the degree of attraction to the reward cue, demonstrating that neural activity in the ventral pallidum largely reflects the degree to which the reward cue is attributed with incentive salience (Figure 2).

Conclusions

We have provided a review of converging data that implicate several brain regions and possible circuits in mediating the attribution of incentive salience to reward cues (Figure 2). Using the sign-tracker/goal-tracker animal model, we, and others, have demonstrated that the role of dopamine is to encode the incentive—and not the predictive—properties of reward cues. Furthermore, the corticostriato-pallido-thalamic loops of the “motive circuit” are engaged only when a reward cue is attributed with incentive salience. Based on the existing data, we postulate that the ability of reward cues to gain motivational control over behavior (as in STs) is regulated by subcortical networks; whereas “top-down” cortical circuits act to inhibit this process (as in GTs). This animal model has been invaluable in allowing us to parse the neurobiological mechanisms underlying stimulus-reward learning and incentive motivation, providing a better understanding of the neural processes that go awry in psychopathology. It is hoped that this model will continue to be utilized across disciplines [56, 57] to further advance our understanding and potentially lead to novel therapeutic targets for the treatment of addiction and related disorders.

Highlights.

  • There is considerable individual variation in conditioned responses to reward cues

  • Cues can evoke simple conditioned responses and/or complex motivational states

  • When a reward cue becomes an incentive stimulus it can elicit maladaptive behavior

  • We can exploit variation to elucidate the neurobiology of cue-elicited behavior

  • Distinct neural circuits are engaged when a cue has incentive vs. predictive value

Acknowledgments

We would like to thank the former and present members of the Robinson and Flagel laboratories who contributed to some of the studies reviewed here and who prompted insightful discussions surrounding the topic.

FUNDING SOURCES

Support for the authors and the studies reviewed in this manuscript is provided by grants from the National Institute on Drug Abuse (NIDA): P01 DA031656 (SBF, TER), P50 DA037844 (SBF, TER) and R01 DA038599 (SBF).

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflict of interest

The authors have no conflicts of interest to declare.

REFERENCES

  • 1.Pavlov I. Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex. London: Oxford University Press; 1927. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Berridge KC. Reward learning: reinforcement, incentives and expectations. In: Medin D, editor. Psychology of learning and motivation. Academic Press; 2001. pp. 223–278. [Google Scholar]
  • 3.Bindra D. How adaptive behavior is produced: a perceptual-motivation alternative to response reinforcement. Behavioral and Brain Sciences. 1978;1:41–91. [Google Scholar]
  • 4.Cardinal RN, Parkinson JA, Hall J, Everitt BJ. Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev. 2002;26(3):321–352. doi: 10.1016/s0149-7634(02)00007-6. [DOI] [PubMed] [Google Scholar]
  • 5.Rescorla RA. Pavlovian conditioning. It's not what you think it is. Am Psychol. 1988;43(3):151–160. doi: 10.1037//0003-066x.43.3.151. [DOI] [PubMed] [Google Scholar]
  • 6.Milton AL, Everitt BJ. The psychological and neurochemical mechanisms of drug memory reconsolidation: implications for the treatment of addiction. Eur J Neurosci. 2010;31(12):2308–2319. doi: 10.1111/j.1460-9568.2010.07249.x. [DOI] [PubMed] [Google Scholar]
  • 7.Robinson TE, Yager LM, Cogan ES, Saunders BT. On the motivational properties of reward cues: Individual differences. Neuropharmacology. 2014;76(Pt B):450–459. doi: 10.1016/j.neuropharm.2013.05.040. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Flagel SB, Akil H, Robinson TE. Individual differences in the attribution of incentive salience to reward-related cues: Implications for addiction. Neuropharmacology. 2009;(56 Suppl 1):139–148. doi: 10.1016/j.neuropharm.2008.06.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Brown PL, Jenkins HM. Auto-shaping of the pigeon's key-peck. J Exp Anal Behav. 1968;11(1):1–8. doi: 10.1901/jeab.1968.11-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Hearst E, Jenkins H. Monograph of the Psychonomic Society. Austin; 1974. Sign-tracking: the stimulus-reinforcer relation and directed action. [Google Scholar]
  • 11.Boakes RA, Poli M, Lockwood MJ, Goodall G. A study of misbehavior: token reinforcement in the rat. J Exp Anal Behav. 1978;29(1):115–134. doi: 10.1901/jeab.1978.29-115. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Robinson TE, Flagel SB. Dissociating the predictive and incentive motivational properties of reward-related cues through the study of individual differences. Biol Psychiatry. 2009;65(10):869–873. doi: 10.1016/j.biopsych.2008.09.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Saunders BT, Robinson TE. A cocaine cue acts as an incentive stimulus in some but not others: implications for addiction. Biol Psychiatry. 2010;67(8):730–736. doi: 10.1016/j.biopsych.2009.11.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Yager LM, Robinson TE. Cue-induced reinstatement of food seeking in rats that differ in their propensity to attribute incentive salience to food cues. Behav Brain Res. 2010;214(1):30–34. doi: 10.1016/j.bbr.2010.04.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Yager LM, Robinson TE. A classically conditioned cocaine cue acquires greater control over motivated behavior in rats prone to attribute incentive salience to a food cue. Psychopharmacology (Berl) 2013;226(2):217–228. doi: 10.1007/s00213-012-2890-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Meyer PJ, Cogan ES, Robinson TE. The form of a conditioned stimulus can influence the degree to which it acquires incentive motivational properties. PLoS One. 2014;9(6):e98163. doi: 10.1371/journal.pone.0098163. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Beckmann JS, Chow JJ. Isolating the incentive salience of reward-associated stimuli: value, choice, and persistence. Learn Mem. 2015;22(2):116–127. doi: 10.1101/lm.037382.114. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Ahrens AM, Singer BF, Fitzpatrick CJ, Morrow JD, Robinson TE. Rats that sign-track are resistant to Pavlovian but not instrumental extinction. Behav Brain Res. 2016;296:418–430. doi: 10.1016/j.bbr.2015.07.055. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Chang SE, Smith KS. An omission procedure reorganizes the microstructure of sign-tracking while preserving incentive salience. Learn Mem. 2016;23(4):151–155. doi: 10.1101/lm.041574.115. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Skinner BF. "Superstition" in the pigeon. Journal of Experimental Pscyhology. 1948;38:168–172. doi: 10.1037/h0055873. [DOI] [PubMed] [Google Scholar]
  • 21.Anselme P, Robinson MJ, Berridge KC. Reward uncertainty enhances incentive salience attribution as sign-tracking. Behav Brain Res. 2013;238:53–61. doi: 10.1016/j.bbr.2012.10.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Schultz W. Neuronal Reward and Decision Signals: From Theories to Data. Physiol Rev. 2015;95(3):853–951. doi: 10.1152/physrev.00023.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Flagel SB, Clark JJ, Robinson TE, Mayo L, Czuj A, Willuhn I, Akers CA, Clinton SM, Phillips PE, Akil H. A selective role for dopamine in stimulus-reward learning. Nature. 2011;469(7328):53–57. doi: 10.1038/nature09588. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Schultz W. Dopamine reward prediction-error signalling: a two-component response. Nat Rev Neurosci. 2016;17(3):183–195. doi: 10.1038/nrn.2015.26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Parker JG, Zweifel LS, Clark JJ, Evans SB, Phillips PE, Palmiter RD. Absence of NMDA receptors in dopamine neurons attenuates dopamine release but not conditioned approach during Pavlovian conditioning. Proc Natl Acad Sci U S A. 2010;107(30):13491–13496. doi: 10.1073/pnas.1007827107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Saunders BT, Robinson TE. The role of dopamine in the accumbens core in the expression of Pavlovian-conditioned responses. Eur J Neurosci. 2012;36(4):2521–2532. doi: 10.1111/j.1460-9568.2012.08217.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27. Yager LM, Pitchers KK, Flagel SB, Robinson TE. Individual variation in the motivational and neurobiological effects of an opioid cue. Neuropsychopharmacology. 2015;40(5):1269–1277. doi: 10.1038/npp.2014.314. The series of studies presented in this manuscript demonstrate that, similar to food- and cocaine-associated cues: 1) a discrete opioid-cue attains greater incentive motivational value in STs than GTs; 2) the attribution of incentive motivational value to an opioid cue is dopamine dependent, and 3) an opioid cue engages the “motive circuit” only if it is imbued with incentive salience. That is, both food- and opioid- associated cues engage similar circuitry if attributed with incentive salience. This work is important, as little has been done to examine individual differences in conditioned responses to cues associated with rewards of different classes, especially on a neurobiological level.
  • 28.Saunders BT, Yager LM, Robinson TE. Cue-evoked cocaine "craving": role of dopamine in the accumbens core. J Neurosci. 2013;33(35):13989–14000. doi: 10.1523/JNEUROSCI.0450-13.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Singer BF, Guptaroy B, Austin CJ, Wohl I, Lovic V, Seiler JL, Vaughan RA, Gnegy ME, Robinson TE, Aragona BJ. Individual variation in incentive salience attribution and accumbens dopamine transporter expression and function. Eur J Neurosci. 2016;43(5):662–670. doi: 10.1111/ejn.13134. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Flagel SB, Robinson TE, Clark JJ, Clinton SM, Watson SJ, Seeman P, Phillips PE, Akil H. An animal model of genetic vulnerability to behavioral disinhibition and responsiveness to reward-related cues: implications for addiction. Neuropsychopharmacology. 2010;35(2):388–400. doi: 10.1038/npp.2009.142. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Flagel SB, Watson SJ, Robinson TE, Akil H. Individual differences in the propensity to approach signals vs goals promote different adaptations in the dopamine system of rats. Psychopharmacology (Berl) 2007;191(3):599–607. doi: 10.1007/s00213-006-0535-8. [DOI] [PubMed] [Google Scholar]
  • 32.Danna CL, Elmer GI. Disruption of conditioned reward association by typical and atypical antipsychotics. Pharmacol Biochem Behav. 2010;96(1):40–47. doi: 10.1016/j.pbb.2010.04.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Fraser KM, Haight JL, Gardner EL, Flagel SB. Examining the role of dopamine D2 and D3 receptors in Pavlovian conditioned approach behaviors. Behav Brain Res. 2016;305:87–99. doi: 10.1016/j.bbr.2016.02.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Lopez JC, Karlsson RM, O'Donnell P. Dopamine D2 Modulation of Sign and Goal Tracking in Rats. Neuropsychopharmacology. 2015;40(9):2096–2102. doi: 10.1038/npp.2015.68. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Cocker PJ, Haar CV, Winstanley CA. Elucidating the role of D receptors in mediating attributions of salience to incentive stimuli on Pavlovian conditioned approach and conditioned reinforcement paradigms. Behav Brain Res. 2016;312:55–63. doi: 10.1016/j.bbr.2016.06.007. [DOI] [PubMed] [Google Scholar]
  • 36.Clark JJ, Collins AL, Sanford CA, Phillips PE. Dopamine encoding of Pavlovian incentive stimuli diminishes with extended training. J Neurosci. 2013;33(8):3526–3532. doi: 10.1523/JNEUROSCI.5119-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Belin D, Jonkman S, Dickinson A, Robbins TW, Everitt BJ. Parallel and interactive learning processes within the basal ganglia: relevance for the understanding of addiction. Behav Brain Res. 2009;199(1):89–102. doi: 10.1016/j.bbr.2008.09.027. [DOI] [PubMed] [Google Scholar]
  • 38.Kalivas PW, Volkow ND. The neural basis of addiction: a pathology of motivation and choice. Am J Psychiatry. 2005;162(8):1403–1413. doi: 10.1176/appi.ajp.162.8.1403. [DOI] [PubMed] [Google Scholar]
  • 39.Kelley AE, Baldo BA, Pratt WE. A proposed hypothalamic-thalamic-striatal axis for the integration of energy balance, arousal, and food reward. J Comp Neurol. 2005;493(1):72–85. doi: 10.1002/cne.20769. [DOI] [PubMed] [Google Scholar]
  • 40.Haber SN. The place of dopamine in the cortico-basal ganglia circuit. Neuroscience. 2014;282C:248–257. doi: 10.1016/j.neuroscience.2014.10.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Flagel SB, Cameron CM, Pickup KN, Watson SJ, Akil H, Robinson TE. A food predictive cue must be attributed with incentive salience for it to induce c-fos mRNA expression in cortico-striatal-thalamic brain regions. Neuroscience. 2011;196:80–96. doi: 10.1016/j.neuroscience.2011.09.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Haight JL, Flagel SB. A potential role for the paraventricular nucleus of the thalamus in mediating individual variation in Pavlovian conditioned responses. Front Behav Neurosci. 2014;8:79. doi: 10.3389/fnbeh.2014.00079. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43. Haight JL, Fraser KM, Akil H, Flagel SB. Lesions of the paraventricular nucleus of the thalamus differentially affect sign- and goal-tracking conditioned responses. Eur J Neurosci. 2015;42(7):2478–2488. doi: 10.1111/ejn.13031. This is one of the first studies to examine the functional relevance of a structure outside of the classic reward circuitry in incentive salience attribution. Moreover, it was demonstrated that this structure, the paraventricular nucleus of the thalamus (PVT), is important in regulating both sign- and goal-tracking conditioned responses, as lesions of this nucleus enhance the expression of sign-tracking, and attenuate goal-tracking. These findings suggest that the PVT may act to suppress the attribution of incentive salience to reward cues, as disruption of the functional activity of this structure enhances the tendency to sign-track. This work lays the foundation for future studies to investigate the role of the neural circuitry surrounding the PVT in incentive salience attribution.
  • 44. Haight JL, Fuller ZL, Fraser KM, Flagel SB. A food-predictive cue attributed with incentive salience engages subcortical afferents and efferents of the paraventricular nucleus of the thalamus. Neuroscience. 2017;340:135–152. doi: 10.1016/j.neuroscience.2016.10.043. This manuscript delineates specific afferent and efferent pathways of the paraventricular nucleus of the thalamus (PVT) that are engaged in response to a predictive vs. an incentive reward cue. Cue-induced neuronal activity was enhanced in prelimbic afferents to the PVT in both sign-trackers and goal-trackers. In contrast, in subcortical regions such as the medial nucleus of the amygdala and lateral hypothalamus, sign-trackers showed a greater percentage of cells that were active in afferents to the PVT in response to cue presentation, and the same was true in efferents of the nucleus accumbens. These findings suggest that the prelimbic cortex may play a critical role in processing the predictive qualities of a reward-paired sitmuli; whereas the hypothalamic-thalamic-striatal axis, as well as the medial amygdala, play a primary role in incentive motivational processes.
  • 45.Campus P, Accoto A, Maiolati M, Latagliata C, Orsini C. Role of prefrontal 5-HT in the strain-dependent variation in sign-tracking behavior of C57BL/6 and DBA/2 mice. Psychopharmacology (Berl) 2016;233(7):1157–1169. doi: 10.1007/s00213-015-4192-7. [DOI] [PubMed] [Google Scholar]
  • 46.Paolone G, Angelakos CC, Meyer PJ, Robinson TE, Sarter M. Cholinergic control over attention in rats prone to attribute incentive salience to reward cues. J Neurosci. 2013;33(19):8321–8335. doi: 10.1523/JNEUROSCI.0709-13.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Lovic V, Saunders BT, Yager LM, Robinson TE. Rats prone to attribute incentive salience to reward cues are also prone to impulsive action. Behav Brain Res. 2011;223(2):255–261. doi: 10.1016/j.bbr.2011.04.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.DiFeliceantonio AG, Berridge KC. Dorsolateral neostriatum contribution to incentive salience: opioid or dopamine stimulation makes one reward cue more motivationally attractive than another. Eur J Neurosci. 2016;43(9):1203–1218. doi: 10.1111/ejn.13220. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Hikosaka O, Sesack SR, Lecourtier L, Shepard PD. Habenula: crossroad between the basal ganglia and the limbic system. J Neurosci. 2008;28(46):11825–11829. doi: 10.1523/JNEUROSCI.3463-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Danna CL, Shepard PD, Elmer GI. The habenula governs the attribution of incentive salience to reward predictive cues. Front Human Neurosci. 2013;7(781):1–7. doi: 10.3389/fnhum.2013.00781. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Chang SE, Wheeler DS, Holland PC. Effects of lesions of the amygdala central nucleus on autoshaped lever pressing. Brain Res. 2012;1450:49–56. doi: 10.1016/j.brainres.2012.02.029. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Root DH, Melendez RI, Zaborszky L, Napier TC. The ventral pallidum: Subregion-specific functional anatomy and roles in motivated behaviors. Prog Neurobiol. 2015;130:29–70. doi: 10.1016/j.pneurobio.2015.03.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Smith KS, Tindell AJ, Aldridge JW, Berridge KC. Ventral pallidum roles in reward and motivation. Behav Brain Res. 2009;196(2):155–167. doi: 10.1016/j.bbr.2008.09.038. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Chang SE, Todd TP, Bucci DJ, Smith KS. Chemogenetic manipulation of ventral pallidal neurons impairs acquisition of sign-tracking in rats. Eur J Neurosci. 2015;42(12):3105–3116. doi: 10.1111/ejn.13103. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55. Ahrens AM, Meyer P, Ferguson L, Robinson TE, Aldridge JW. Neural activity in the ventral pallidum encodes variation in the incentive value of a reward cue. Journal of Neuroscience. 2016;36(30):7957–7970. doi: 10.1523/JNEUROSCI.0736-16.2016. Single-unit electrophysiology was used to examine neural activity associated with individual diffrences in incentive salience attribution. The findings revealed that cue-evoked neural firing in the ventral pallidum reflects the strength of incentive motivation, with the greatest neural responses occurring in individuals that demonstrated the strongest attraction to the cue. These findings demonstrate in important role for the ventral pallidum in the process by which cues gain inordinate control over behavior.
  • 56. Garofalo S, di Pellegrino G. Individual differences in the influence of task-irrelevant Pavlovian cues on human behavior. Front Behav Neurosci. 2015;9:163. doi: 10.3389/fnbeh.2015.00163. This study is amongst the first to successfully demonstrate individual differences in response to Pavlovian reward cues in humans. Individuals were classified as sign-trackers or goal-trackers based on the tendency to direct eye-movements towards the “sign” or the “goal” during a Pavlovian conditioning task. It was then shown that the individuals who had a greater tendency to sign-track were also more responsive the reward cue on a Pavlovian-to-instrumental transfer task. Human STs were also more impulsive than GTs based on a self-report questionnaire. This work establishes a “human model” of STs and GTs that can be used for translational and neuroimaging studies.
  • 57. Huys QJ, Tobler PN, Hasler G, Flagel SB. The role of learning-related dopamine signals in addiction vulnerability. Prog Brain Res. 2014;211:31–77. doi: 10.1016/B978-0-444-63425-2.00003-9. This manuscript provides a comprehensive overview of the theoretical, mathematical and experimental accounts of the role of dopamine in stimulus-reward learning, with a focus on addiction vulnerability. It includes a description of the most popular computational models that have been used to describe reward-related dopamine signaling, including those proposed for the sign-tracker/goal-tracker animal model. The contents of this article represents the interest the sign-tracker/goal-tracker has garnered across disciplines, and its relevance to addiction and related disorders.
  • 58.Saunders BT, Robinson TE. Individual variation in resisting temptation: Implications for addiction. Neurosci Biobehav Rev. 2013;37(9 Pt A):1955–1975. doi: 10.1016/j.neubiorev.2013.02.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Meyer PJ, Lovic V, Saunders BT, Yager LM, Flagel SB, Morrow JD, Robinson TE. Quantifying individual variation in the propensity to attribute incentive salience to reward cues. PLoS One. 2012;7(6):e38987. doi: 10.1371/journal.pone.0038987. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES