Leveraging individual differences in cue-reward learning to investigate the psychological and neural basis of shared psychiatric symptomatology: The sign-tracker/goal-tracker model

Abstract

In our modern environment, we are bombarded with stimuli, or cues, that exert significant influence over our actions. The extent to which such cues attain control over or disrupt goal-directed behavior is dependent on several factors, including one’s inherent tendencies. Using a rodent model, we have shown that individuals vary in the value they place on stimuli associated with reward. Some individuals, termed “goal-trackers”, primarily attribute predictive value to reward cues, whereas others, termed “sign-trackers”, attribute predictive and incentive value. Thus, for sign-trackers, the reward cue is transformed into an incentive stimulus that is capable of eliciting maladaptive behaviors. The sign-tracker/goal-tracker animal model has allowed us to refine our understanding of behavioral and computational theories related to reward learning and to parse the underlying neural processes. Further, the neurobehavioral profile of sign-trackers is relevant to several psychiatric disorders, including substance abuse disorder (SUD), impulse control disorders (ICD), obsessive compulsive disorder (OCD), attention-deficit/hyperactivity disorder (ADHD), and post-traumatic stress disorder (PTSD). This model, therefore, can advance our understanding of the psychological and neurobiological mechanisms that contribute to individual differences in vulnerability to psychopathology. Notably, initial attempts at translation - capturing individual variability in the propensity to sign-track in humans - have been promising and in line with what we have learned from the animal model. In this review, we highlight the pivotal role played by the sign-tracker/goal-tracker animal model in enriching our understanding of the psychological and neural basis of motivated behavior and psychiatric symptomatology.

Introduction

“I do not understand what I do. For what I want to do I do not do, but what I hate I do.” (Romans 7:15; New International Version Holy Bible, 1978/2017).

Our persistent tendency to engage in behaviors that directly undermine our own goals and aspirations has been a source of consternation for philosophers, religious thinkers, and ordinary people alike throughout human existence. One example of this in modern times is responding to the sound of a notification on your smart phone, even if it interrupts your work. Undoubtedly, some individuals struggle with this cognitive-behavioral clash more than others. Research aimed at uncovering the psychological and neural processes driving the ability of environmental stimuli to control behavior against an individual’s wishes can advance our understanding of psychiatric disorders and the maladaptive behaviors that characterize them. Determining how and why these neurobehavioral processes differ amongst individuals remains an exciting area of research with great potential to advance the field and lead to more effective treatments in psychiatry.

Our behavior is shaped by stimuli, or cues, in the environment. Sensitivity to environmental cues is an adaptive process that promotes survival by facilitating the acquisition of valuable resources and avoidance of potential threats. However, such cues can also elicit a behavioral response that contradicts one’s goals. Extreme examples of this are characteristic of the lives of individuals who suffer from compulsive urges, such as those with substance use disorder (SUD). For individuals with SUD, cues (e.g., people, places, paraphernalia) previously associated with the drug-taking experience gain excessive control over their behavior and attain the ability to elicit drug-seeking or relapse (T. E. Robinson & Berridge, 1993; Tomie, 1996). This can occur even when one is fully aware of adverse consequences and despite the desire to remain abstinent (Childress et al., 1992, 1993).

Environmental cues come to control behavior via associative learning processes. Upon learning the relationship between a cue and outcome, value is placed upon the cue. However, we know that individuals vary in how they respond to environmental cues and this variability is driven largely by differences in the propensity to attribute incentive value to a reward cue (Flagel et al., 2009). When a reward cue is attributed with incentive motivational value, or incentive salience, the cue itself becomes attractive, desirable, and able to trigger behaviors beyond one’s control (T. E. Robinson & Berridge, 1993). Perhaps it is not surprising, therefore, that individuals with an increased propensity to attribute incentive value to reward cues exhibit a neurobehavioral profile of relevance to multiple psychiatric disorders, including SUD (Saunders & Robinson, 2010, 2013), impulse control disorders (ICD; Lovic et al., 2011), obsessive compulsive disorder (OCD; Eagle et al., 2020, 2020), attention-deficit/hyperactivity disorder (ADHD; Paolone et al., 2013), and post-traumatic stress disorder (PTSD; Morrow et al., 2011, 2015).

Here we will describe an animal model that captures individual variability in the propensity to attribute incentive salience to reward cues. We will begin with a historical perspective and then highlight the utility of this model in parsing the psychological, neurobiological, and computational substrates of motivated behavior. In addition, we will describe initial attempts at translation with great promise for advancing our understanding of individual differences in vulnerability to psychiatric disorders.

Pavlovian Cue-reward Learning

Reward learning is an intricate process that enables cues that predict favorable outcomes to elicit motivated behavior. A neutral cue becomes a conditioned stimulus (CS) after it consistently precedes the delivery of a reward or unconditioned stimulus (UCS; Pavlov, 1927). Through repeated exposure to the CS-UCS pairing, an association is formed between the cue and the reward. In essence, reward learning occurs, resulting in our perception of the cue as a predictor of the reward. Thus, through associative learning, the cue gains predictive value and, consequently, may guide our actions toward obtaining the reward. However, Pavlovian conditioning experiments have demonstrated that, in addition to predictive value, a cue can also acquire incentive motivational value.

Autoshaping: The attribution of incentive motivational value to reward cues

In the late 1960s, Pavlovian autoshaping experiments demonstrated that cues associated with rewards can evoke complex emotional and motivational states reflected as seemingly irrelevant and maladaptive behaviors (Brown & Jenkins, 1968). Autoshaping, in this case, refers to the Pavlovian procedure that results in conditioned approach towards a stimulus that has been paired with a reward. Importantly, in this paradigm, an operant response is not required for the animal to receive a reward; thus, there is no response reinforcement. Yet, Brown and Jenkins (1968) showed that when a key-light (CS) is repeatedly and non-contingently paired with a food reward (UCS), pigeons will come to vigorously peck at the key-light. Remarkably, they do so in a consummatory manner, treating it as the reward itself. Thus, if the reward (UCS) is grain, the pigeons peck at the key-light with an open beak, and if the reward (UCS) is water, the beak remains closed and the pecking is coupled with swallowing movements (Jenkins & Moore, 1973). This cue-elicited consummatory behavior was thought to reflect “stimulus-substitution” (Pavlov, 1927; Staddon & Simmelhag, 1971) - wherein the cue (CS) appears to act as a proxy for the reward (US). However, we now know that such a response reflects the attribution of incentive motivational value, or incentive salience, to the reward cue (Berridge, 2000; Cardinal et al., 2002; Flagel et al., 2009). In support, pigeons will continue to approach and peck at the key-light (CS) even if doing so prevents them from retrieving the food reward at the other end of the testing chamber, and even if they are hungry (Hearst & Jenkins, 1974). Further, under conditions of omission, wherein contact with the cue (CS) prevents the reward from being delivered, animals will continue to interact with (D. R. Williams & Williams, 1969) and/or approach the cue (CS), but with a different topography (Chang & Smith, 2016; Locurto et al., 1976; María-Ríos et al., 2023;); suggesting that the cue maintains its incentive properties and that the conditioned response is not reflective of response reinforcement (D. R. Williams & Williams, 1969). Together, these studies were amongst the first to demonstrate that Pavlovian learning can transform a cue (CS) into an incentive stimulus or “motivational magnet” (Berridge, 2000); making it so irresistibly attractive that it can elicit maladaptive and seemingly compulsive behavior (see also Boakes et al., 1978; Breland & Breland, 1961).

It is the process of incentive salience attribution that transforms a Pavlovian cue into an incentive stimulus, making it not just predictive, but also “wanted” (Berridge, 1996, p. 199; Bindra, 1978; Bolles, 1972; F. M. Toates, 1986). There are three fundamental properties of incentive stimuli (Berridge, 2000; Cardinal et al., 2002; Flagel et al., 2009): 1) they have the ability to elicit approach towards them, as described above (see also Flagel et al., 2007; Peterson et al., 1972); 2) they can energize ongoing instrumental actions, as indicated by Pavlovian-to-instrumental transfer (Lovibond, 1983; Rescorla & Lolordo, 1965; Wyvell & Berridge, 2001), and 3) they can reinforce learning of new instrumental actions, or act as conditioned reinforcers (Di Ciano & Everitt, 2004; B. A. Williams & Dunn, 1991). For emphasis, studies of Pavlovian instrumental transfer illustrate that a Pavlovian cue can drive an animal to interact with a reinforced instrument, like a lever, resulting in the delivery of a reward not previously associated with that cue (Lovibond, 1983; Rescorla & Lolordo, 1965). Thus, a cue (CS) can not only serve as the target of incentive motivation but, even in the absence of the associated reward (UCS), a cue (CS) can trigger a conditioned motivational response. Such findings demonstrated the ability of reward cues to elicit complex emotional and motivational states and led to the notion that excessive attribution of incentive salience to reward cues may contribute to maladaptive behaviors characteristic of psychopathology (Berridge & Robinson, 1998; Bindra, 1974, 1978; Bolles, 1972; F. Toates, 1994; F. M. Toates, 1986; Tomie, 1996).

Incentive motivation and psychopathology

Incentive motivation theories have greatly informed our understanding of SUD and the phenomenon of relapse (e.g., T. E. Robinson & Berridge, 1993). Stimuli (e.g., people, places, paraphernalia) that have been previously associated with drug use not only predict the availability of drugs but can also reinstate drug-seeking behavior even against the will of an individual with addiction (Saunders & Robinson, 2010, 2013). This is because, through Pavlovian conditioning, drug associated stimuli (cues) acquire incentive salience (T. E. Robinson & Berridge, 1993; Saunders & Robinson, 2010; Yager & Robinson, 2013). Individuals with addiction are more likely to instill incentive motivational properties onto a drug cue; converting it from a Pavlovian stimulus into an incentive stimulus that takes hold of their attention and becomes irresistible (Berridge, 2000). In turn, such cues can control their behavior in an inordinate manner. To illustrate, upon exposure to drug-cues, individuals with addiction report “wanting” or craving for a drug, even when they do not “like”, or derive hedonic pleasure, from the drug (Berridge, 1996; T. E. Robinson & Berridge, 1993). Beyond SUD, the incentive salience framework has informed our understanding of pathological gambling, food addiction, and psychotic disorders, including schizophrenia (Hellberg et al., 2019; Jensen et al., 2008; Kapur et al., 2005; M. J. F. Robinson et al., 2015). Below we will discuss the potential relevance of Pavlovian incentive learning to other psychiatric disorders and their shared symptomatology. Specifically, we will describe how individual differences in the propensity to attribute incentive salience to reward cues may inform our understanding of individual differences in vulnerability to psychiatric disorders.

Individual differences in the propensity to attribute incentive salience to reward cues: The sign-tracker/goal-tracker animal model

The sign-tracker (ST)/goal-tracker (GT) animal model captures individual differences in the propensity to attribute incentive salience to reward using a Pavlovian conditioned approach paradigm (Flagel et al., 2009; Meyer et al., 2012). Within this paradigm, the presentation of an illuminated lever-cue (CS) precedes the non-contingent delivery of food reward (UCS). Thus, no response is required for the food to be delivered. After the CS-UCS association has been made with repeated presentations of the lever-food sequence, rats may develop one of two distinct conditioned responses: sign-tracking or goal-tracking (Figure 1). As described in the 1970’s (Boakes et al., 1978; Hearst & Jenkins, 1974), sign-tracking is used to depict cue (CS)-elicited approach to the “sign” that predicts reward; and goal-tracking refers to cue (CS)-elicited approach to the “goal” or location of reward delivery. In our paradigm, upon lever-cue presentation, sign-trackers approach and interact with the cue itself: nibbling it, sniffing it, and pressing on it (Figure 1). They only approach the food delivery site upon retraction of the lever-cue, at which time the food reward is delivered. In contrast, goal-trackers, upon lever-cue presentation, acknowledge and sometimes orient towards the cue, but then quickly approach and interact with the food delivery site. For goal-trackers, consummatory behavior may be directed towards the reward delivery site (Mahler & Berridge, 2009) as they wait for the food to be delivered. Another subset of the rats, termed intermediate responders (IR), exhibit both sign- and goal-tracking conditioned responses, with no clear preference for either and often vacillating between the two. In a large population of rats, we find that approximately 1/3 of the population is sign-trackers, 1/3 intermediate responders, and 1/3 goal-trackers (Figure 2).

Figure 1. Neural circuits underlying sign-tracking and goal-tracking.

This graphic illustrates brain areas as well as the afferent and efferent connections thought to underlie (A) goal-tracking and (B) sign-tracking. (A) Goal-trackers (GTs) attribute predictive value to the lever-cue (conditioned stimulus, CS) that precedes the delivery of food reward (unconditional stimulus, US) and approach the location of the reward upon presentation of the lever-cue (CS). (B) Sign-trackers (STs) attribute both predictive and incentive value to the lever-cue (CS) and approach and manipulate it upon presentation. Brain regions believed to contribute to the respective behaviors are presented in color, while regions that do not appear to play significant role are in gray. Projections that are depicted as dotted black lines indicate the presence of neuronal connections but their influence on sign- or goal-tracking have not yet been elucidated. Neuronal projections that have been shown to play a role are depicted by the axonal schematic. (A) Goal-tracking behavior appears to depend largely on cortical systems, while (B) sign-tracking behavior relies on sub-cortical systems. The paraventricular nucleus of the thalamus (PVT; orange neuron) and nucleus accumbens (NAc; neuron) have been implicated in both behaviors, but likely with different “weights” of inputs from cortical and subcortical structures in goal-trackers versus sign-trackers. (A) The green neuron represents the dense glutamatergic (Glu) innervation from the prelimbic cortex (PrL)-PVT. (B) The purple neuron represents orexinergic (OX) innervation from the lateral hypothalamus (LH)-PVT. The orange neuron is illustrating glutamatergic (Glu) innervation from the PVT- NAc. The turquoise neuron represents dopaminergic (DA) innervation from the ventral tegmental area (VTA)-NAc. Abbreviations: Amy, amygdala; Hipp, hippocampus; LH, lateral hypothalamus; NAc, nucleus accumbens; PrL, prelimbic cortex; PVT, paraventricular thalamic nucleus; VTA, ventral tegmental area; DA, dopamine; Glu, glutamate; OX, orexin.

Figure 2. Distribution of the sign-tracker/goal-tracker phenotypes in a large rat population and associated behaviors of relevance to psychiatric disorders.

This histogram represents a frequency distribution of the propensity to attribute incentive salience to a reward cue in 6182 Sprague-Dawley rats. The Pavlovian conditioned approach (PavCA) Index, is a composite score used to assess the propensity of an individual rat to approach the lever-CS (sign-tracking) vs. the food cup (goal-tracking) across five conditioning sessions. An index of −0.5 to −1.0 indicates that the rat is a goal-tracker (GT). An index of 0.5 to 1.0 indicates that the rat is a sign-tracker (ST). An index between −0.5 to 0.5 indicates that the rat is an intermediate responder (IR). The large population of rats came from a database of rats that have been screened for Pavlovian conditioned approach behavior in the labs of Drs. Shelly Flagel, Jonathan Morrow, Terry Robinson, and Martin Sarter at the University of Michigan. The behavioral characteristics of each phenotype (described in the main text) are indicated below the goal-trackers (orange) and sign-trackers (blue). Abbreviations: ICD, Impulse Control Disorders; ADHD, Attention Deficit Hyperactivity Disorder; PTSD, Post-traumatic Stress Disorder; SUD, Substance Use Disorder; OCD, Obsessive-Compulsive Disorder

These individual differences in response to a reward cue reflect two distinct processes in reward learning: associative (or predictive) and incentive processes (see Figure 1). In this rodent model, the lever-food association is made by both sign-trackers and goal-trackers, and both assign predictive value to the lever-cue. As a result, the lever-cue functions as an equally effective predictor and triggers a conditioned response in all rats. However, in addition to predictive value, sign-trackers also attribute incentive salience to the lever-cue (Flagel et al., 2009; Meyer et al., 2012). Thus, only for sign-trackers does the lever-cue become an incentive stimulus, capable of eliciting approach, invigorating ongoing instrumental behavior, and acting as a conditioned reinforcer (T. E. Robinson & Flagel, 2009; Yager & Robinson, 2010). The sign-tracker/goal-tracker model, therefore, allows us to dissociate predictive vs. incentive learning processes and study the underlying mechanisms. Importantly, this individual variation in conditioned responding appears to be stable and heritable (Campus et al., 2016; Dickson et al., 2015; Flagel et al., 2010); suggesting that the propensity to attribute incentive motivational value to a reward cue is an inherent trait with a presumed genetic component (see also Gileta et al., 2022).

The sign-tracker/goal-tracker model yields a neurobehavioral endophenotype of relevance to psychiatric symptomatology

Underlying neurobiology

The sign-tracker/goal-tracker animal model has significantly advanced our understanding of the neurobiological processes that promote the attribution of incentive motivational value to reward cues (Flagel & Robinson, 2017; Kuhn et al., 2018; T. E. Robinson & Flagel, 2009). With this model, we continue to uncover the intricate aspects of cue-reward learning at a molecular and circuit level in the brain, as described below.

The role of dopamine in cue-reward learning

The sign-tracker/goal-tracker rodent model has refined our understanding of dopamine in reward learning (Flagel, Clark, et al., 2011; Iglesias et al., 2023), a topic that has long-been debated. Dopamine has been regarded as an indicator of “prediction error” (Balleine et al., 2009; Schultz et al., 1997; Waelti et al., 2001), but also as the required agent for incentive salience attribution (Berridge, 1996, 2007; Berridge et al., 2009). Under the reward prediction error (RPE) theory, dopamine acts as a “teaching signal” for associative learning by reinforcing new links between stimuli and outcomes (Everitt et al., 2001; Montague et al., 2004; Schultz, 2004). This theory is supported by the observations that dopamine neurons are activated when a reward (UCS) is initially presented, and, with learning, these bursts in dopamine activity shift to occur in response to the presentation of the cue (CS) that precedes the reward (Schultz et al., 1997). Thus, simply put, dopamine activation by a cue (CS) that has been associated with reward (UCS) results in the prediction or expectation of the future reward (de la Fuente-Fernández et al., 2002; Phillips et al., 2003; Tobler et al., 2006). Hence, it is suggested that dopamine encodes the predictive value of a cue during reward learning.

The incentive salience theory posits that dopamine is not a “teaching signal” per se, nor does it encode the hedonic value of rewards (Berridge, 2000, 2007). Rather, dopamine is believed to act as a necessary agent for conditioned “wanting”. Thus, during reward learning, dopamine activity is thought to reflect the incentive motivational value of a reward-associated cue (Berridge, 2007; Berridge & Robinson, 1998). This theory is supported by evidence that dopamine depletion or inactivation does not attenuate or remove the ability to: enjoy or “like” a rewarding UCS (Berridge et al., 1989; Cannon & Bseikri, 2004; Treit & Berridge, 1990), form new associative learning (S. Robinson et al., 2005), or form preferences for a rewarding UCS (Cannon & Palmiter, 2003; Hnasko et al., 2005). These studies bolstered the incentive salience theory of dopamine, suggesting that dopamine signaling observed during reward learning is responsible for the attribution of incentive motivational value to rewards and their associated cues and not hedonic reactions or associative (predictive) learning (Berridge & Robinson, 1998).

The incentive salience theory of dopamine has been further tested using the sign-tracker/goal-tracker rodent model (Flagel, Clark, et al., 2011; Iglesias et al., 2023; Saunders & Robinson, 2012). Rats were exposed to the previously described Pavlovian conditioning paradigm, in which the presentation of a lever-cue (CS) is paired with the delivery of food reward (UCS) (Flagel, Clark et al., 2011). Additionally, because the nucleus accumbens core (NAcC) is a key structure for motivated behavior (Cardinal et al., 2002; Ciano et al., 2001; Parkinson et al., 2002) and prediction-error signals (Day et al., 2007), carbon-fiber microelectrodes were implanted into the NAcC to record patterns of phasic dopamine release during Pavlovian learning using fast scan cyclic voltammetry (FSCV). Neurochemical analysis revealed that, in sign-trackers, as the cue-reward (CS-UCS) association was established, there was a shift in dopamine away from reward (UCS) presentation to lever-cue (CS) presentation. This shift in dopamine signaling mirrors the transfer of dopamine signals from the UCS to the CS that is traditionally associated with encoding the predictive value of a reward cue (Schultz, 2016; Schultz et al., 1997). However, this dopamine shift was only apparent in sign-trackers; goal-trackers exhibited a near equal rise in dopamine concentration in response to the lever-cue (CS) and food reward (UCS) throughout training (Flagel, Clark, et al., 2011). This indicated, therefore, that the acquisition of the goal-tracking response, which relies on the attribution of predictive value upon a CS, does not require the dopamine shift from the reward (UCS) to the lever-cue (CS). In support, when rats were systemically administered the nonspecific DA receptor antagonist, flupentixol, prior to each session of Pavlovian conditioning, both sign- and goal-tracking behavior was attenuated; but when subsequently tested under “drug-free” conditions, only goal-tracking (and not sign-tracking) was restored (Flagel, Clark, et al. 2011). Thus, dopamine receptor blockade did not interfere with the predictive learning needed for the development of a goal-tracking response, but did impede incentive learning and thereby the acquisition of a sign-tracking response (Flagel, Clark, et al., 2011). Several subsequent studies have supported these findings, including those demonstrating that local blockade of dopamine receptors (Saunders & Robinson, 2012) or lesions of the nucleus accumbens (Chow et al., 2016) diminishes sign-tracking, but not goal-tracking behavior. More recently, we showed that optogenetic inhibition of dopamine neurons in the ventral tegmental area specifically at the time of lever-cue (CS) presentation prevents the development of a sign-tracking conditioned response (Iglesias et al., 2023). Taken together, these findings demonstrate that the primary role of dopamine is not to encode the predictive value of reward cues (CS), but instead, dopamine is essential for encoding the incentive motivational value of reward cues (CS) (Chow et al., 2016; Flagel, Clark, et al., 2011; Iglesias et al., 2023; Saunders & Robinson, 2012).

Bottom-up versus top-down processing in sign-trackers and goal-trackers

The distinct conditioned responses exhibited by sign-trackers and goal-trackers are, undoubtedly, driven by differences in brain activity beyond the dopamine system (but see also Flagel et al., 2010; Singer et al., 2016). To identify the brain regions involved in generating the sign-tracking and goal-tracking conditioned responses, we assessed cue-elicited neuronal activity, using c-fos as a marker. Cues associated with either food- or drug-rewards elicit increased neuronal activity in sign-trackers relative to goal-trackers in multiple brain regions associated with motivation and reward (Flagel, Cameron, et al., 2011; Yager et al., 2015). These include the prelimbic cortex, orbitofrontal cortex, striatum (both the caudate and nucleus accumbens), thalamus (paraventricular, intermediodorsal, and central medial nuclei), and lateral habenula (Flagel, Cameron, et al., 2011; Yager et al., 2015). Additionally, inter-regional correlations of cue-induced neuronal activity revealed distinct patterns of connectivity between brain regions in sign-trackers compared to goal-trackers (Flagel, Cameron, et al., 2011). Specifically, in sign-trackers, there was a strong correlation in cue-induced neuronal activity between subcortical regions, such as the paraventricular nucleus of the thalamus (PVT) and the nucleus accumbens (NAc). In contrast, in goal-trackers, there was a correlation in activity between the prefrontal cortex (PFC) and the PVT. These data highlighted the PVT as an important node in processing the value of the reward cue for both sign-trackers and goal-trackers, and supported the notion that the encoding of predictive and incentive value may be under the control of seemingly opposing neural networks.

To further explore the role of the PVT and its surrounding circuitry in incentive motivational processes, we looked specifically at cue (CS)-induced neuronal activity in neurons projecting to and from the PVT (using a retrograde tracer and c-fos; Haight et al., 2017). We found similar levels of cue-induced activity in sign-trackers and goal-trackers in neurons in the prelimbic cortex (PrL) that project directly to the PVT; but the subcortical circuitry surrounding the PVT was engaged to a greater extent in sign-trackers relative to goal-trackers (Figure 1). This was especially apparent in neurons that projected from the lateral hypothalamus (LH) to the PVT, and those that projected from the PVT to the nucleus accumbens (NAc) (Haight et al., 2017; Figure 1). In support of a role for the LH-PVT pathway, we subsequently showed that lesions of the LH attenuate sign-tracking behavior, as does blockade of orexin signaling within the PVT (Haight et al., 2020). Together, these data suggest that the subcortical circuitry surrounding the PVT plays a predominant role in the encoding of incentive value and, thereby, the expression of sign-tracking behavior (Figure 1).

Further investigation of the PVT revealed that loss of PVT function (via lesions) before a conditioned response is acquired enhances the tendency to sign-track and decreases the tendency to goal-track (Haight et al., 2015). However, if PVT function is lost after a conditioned response has already been learned, goal-trackers show greater sign-tracking behavior, while there is no consequence on the response of sign-trackers (Haight et al., 2015). These data suggested that, in goal-trackers, the PVT is needed to suppress incentive salience attribution (Haight et al., 2015). In support, we later showed that neuronal projections from the prelimbic cortex (PrL) to the PVT act to suppress sign-tracking behavior (Campus et al., 2019). Thus, when this pathway is chemogenetically inhibited in goal-tracker rats, sign-tracking behavior emerges. In contrast, when the same PrL-PVT pathway is stimulated in sign-tracker rats, sign-tracking behavior diminishes (Campus et al., 2019). Remarkably, inhibition of the PrL-PVT pathway does not affect the behavior of sign-trackers; and stimulation of this pathway does not affect the behavior of goal-trackers. Taken together, these data suggest that goal-trackers rely on a predominant “top-down” cortico-thalamic control mechanism that acts to encode the predictive value of reward cues and inhibit incentive motivational processes; whereas sign-trackers are dominated by “bottom-up” subcortical emotional and motivational processes that encode the incentive value of reward cues and potentially override cortical control systems (Campus et al., 2019; Flagel, Cameron, et al., 2011; Haight et al., 2015, 2017, 2020). This then, might explain, why the neurobehavioral endophenotype of sign-trackers reflects a general loss of inhibitory control.

Translational relevance: Correlates to human psychopathology

The top-down vs. bottom-up mechanistic framework gains additional support from the observation of other behavioral traits expressed by sign-tracker and goal-tracker rodents. These traits resemble symptoms of psychopathology and are assessed using tests that are similar to those used in human subjects. For instance, using a sustained attention task (SAT), Paolone et al. (2013) investigated attentional capacity in sign-trackers and goal-trackers. Relative to goal-trackers, sign-trackers were found to have attentional deficits that were associated with diminished acetylcholine (ACh) levels in the prefrontal cortex (Paolone et al., 2013). Sign-trackers are also more impulsive than goal-trackers on tests of impulsive action, demonstrating premature responding or the inability to withhold a response (King et al., 2016; Lovic et al., 2011). In relation, in a rodent analog of obsessive-compulsive disorder (OCD), the observing response task, sign-trackers display elevated levels of dysfunctional checking behavior - a response pattern linked to increased dopamine function (Eagle et al., 2020; Vousden et al., 2020). Together, these findings support the notion that sign-trackers rely predominantly on “bottom-up” motivational processes that may override “top-down” inhibitory control mechanisms, resulting in their characteristic behavioral disinhibition.

Sign-trackers also exhibit a number of addiction-related behaviors. Rats that sign-track to food-associated cues, sign-track to discrete cues associated with cocaine or opioid delivery as well (Flagel et al., 2010; Yager et al., 2015). Relative to goal-trackers, sign-trackers also show greater psychomotor sensitization to cocaine (Flagel et al., 2007), an increased propensity to acquire cocaine self-administration (Beckmann et al., 2011), and greater cue-induced relapse to cocaine-seeking following relatively limited drug-taking experience (Saunders & Robinson, 2010). Conversely, after prolonged and intermittent access to cocaine, sign-trackers and goal-trackers do not differ on measures of addiction-like behavior (Kawa et al., 2016). Further, relative to sign-trackers, goal-trackers rats exhibit increased context-induced cocaine-seeking behavior (Saunders et al., 2014) and greater reinstatement of cocaine-seeking in response to a discriminative stimulus or “occasion setter”, a process that is dependent on cholinergic neurons of the basal forebrain (Pitchers et al., 2017). Thus, while the constellation of neurobehavioral traits described above may render sign-trackers more susceptible to addiction, certain factors, and conditions, especially those that tap into cortical control, may make goal-trackers especially prone (T. E. Robinson et al., 2018).

Hyperresponsivity to cues associated with emotionally salient stimuli is characteristic of externalizing disorders, like substance use disorder (SUD) and impulse control disorder (ICD), as well as internalizing disorders, like obsessive-compulsive disorder (OCD) and post-traumatic stress disorder (PTSD; Casada et al., 1998; Ehlers & Clark, 2000). Interestingly, sign-tracker rats exhibit elevated fear responses to discrete cues linked to aversive stimuli (Morrow et al., 2011). These fear responses intensify or “incubate” over time, and this “incubation” is correlated with reduced expression of brain-derived neurotrophic factor (BDNF) in the prefrontal cortex (Morrow et al., 2015). Such increased cue-induced fear responses have been proposed to be a result of impaired fear extinction caused by the amplification of amygdala activity and a reduction of cortical processing within the ventromedial prefrontal cortex (vmPFC) in both humans and rodents (Bremner, 2007; Pickens et al., 2009; Quirk & Mueller, 2008; Rauch et al., 2006). These findings align with the notion of decreased “top down” cortical control and increased “bottom-up” processing in sign-trackers. In contrast, goal-trackers exhibit a greater fear response upon exposure to the original fear-conditioning context in the absence of discrete cues, and they do not show “incubation” of this response (Morrow et al., 2011, 2015). Thus, goal-trackers appear to utilize context to appropriately modify their reactions to emotionally salient stimuli (Morrow et al., 2011; Figure 2); whereas sign-trackers tend to be hyper-responsive to cues associated with either appetitive or aversive stimuli.

The studies described above demonstrate that individual variation in response to a Pavlovian cue associated with food reward can be leveraged to better understand the psychological and neural mechanisms that contribute psychiatric symptomatology. Sign-trackers seem to represent a neurobehavioral endophenotype that confers greater susceptibility to impulse control disorders (ICD), obsessive compulsive disorder (OCD), attention-deficit/hyperactivity disorder (ADHD), substance use disorder (SUD), and post-traumatic stress disorder (PTSD; Figure 2). Of course, however, human studies are needed to determine the direct relevance of the sign-tracker/goal-tracker animal model to human behavior and psychopathology.

Capturing the tendency to sign-track or goal-track in humans

In the realm of motivated behavior and decision-making, there are clearly individual differences in how people respond to rewards and their associated cues. This encompasses a spectrum ranging from meticulous planning to highly impulsive actions: some individuals deliberate extensively prior to pursuing a reward, while others act impulsively with little foresight (Kuo et al., 2009). While we can readily recognize these differences in human behavior in our modern environment, only a few studies to date have directly assessed the propensity to sign-track in people.

Garofalo and Pelligrino (2015) were the first, to our knowledge, to assess individual differences in response to a Pavlovian reward cue in humans using the sign-tracker/goal-tracker framework (Garofalo & di Pellegrino, 2015). Individuals were characterized as sign-trackers or goal-trackers based on their learned eye-gazes (CR) directed towards either a visual cue (CS) or reward (US) location upon cue (CS) presentation. These subjects were subsequently tested on a Pavlovian-to-instrumental transfer (PIT) task, and it was shown that the reward-related cue biased behavior to a greater extent in sign-trackers than goal-trackers. That is, there was a stronger PIT effect in sign-tracking individuals, and the same individuals also showed more impulsiveness than those categorized as goal-trackers (Garofalo and Pelligrino, 2015). These findings are consistent with those reported in rodents and corroborate the idea that propensity to sign-track captures a broader profile of individual differences, which may be clinically relevant (Garofalo and Pelligrino, 2015).

In a subsequent study, Schad et al. (2020) investigated eye gaze direction and pupillary response of healthy adult participants engaging in a Pavlovian conditioning task during functional magnetic resonance imaging (fMRI). Similar to Garofalo and Pelligrino (2015), individuals classified as sign-trackers showed a preference for eye-tracking the reward-associated cue (CS); whereas those categorized as goal-trackers primarily focused on the location of the reward (UCS), progressively shifting their gaze away from the cue (CS) and toward the reward (UCS) as the trials continued. The sign-tracking response correlated with blood-oxygen-level-dependent (BOLD) fMRI signals in the NAc, VTA, dorsal striatum, amygdala, and vmPFC. Consistent with the findings reported in Flagel, Clark, et al., (2011), there were no corresponding BOLD signals in the ventral or dorsal striatal structures, including the NAc, in goal-trackers. In further alignment with the sign-tracker/goal-tracker rodent model (T. E. Robinson & Flagel, 2009) and in agreement with Garafolo and Pelligrino (2015), the results of a PIT task indicated that both rewarding and aversive conditioned stimuli had acquired incentive salience in sign-trackers but not goal-trackers (Schad et al.,. 2020).

Using a Pavlovian conditioning paradigm and apparatus more similar to that we use in rodents, sign- and goal-tracking tendencies have recently been assessed in human children (Joyner et al., 2018). Colaizzi et al. (2023) examined Pavlovian conditioned approach behavior in response to a lever-cue (CS) paired with a monetary reward (US) in healthy human children between the ages of 9–12 years old. While there was variability in the conditioned response exhibited, few children displayed goal-tracking behavior, and the population was generally skewed towards sign-trackers (Colaizzi et al., 2023). These findings are, perhaps, not surprising since cortical development is ongoing at this age; thus, lending support to the notion that “top-down” executive control promotes the goal-tracking response. In agreement, using fMRI, it was found that children who were less likely to sign-track (i.e., “non sign-trackers”) had greater activation in the inferior parietal lobe (IPL) in response to a reward cue; whereas those categorized as sign-trackers showed greater amygdala activation in response to a reward cue (Colaizzi et al., 2023). Further, parent reports revealed that, relative to non-sign-trackers, those categorized as sign-trackers had increased externalizing characteristics such as symptoms of ADHD, susceptibility to symptoms of fear, oppositional defiance and social problems, and negative affect. Lastly, investigation into environmental interactions revealed a significant interplay between early life adversity and the manifestation of a sign-tracking response (Colaizzi et al., 2023).

Collectively, the outcomes of these studies indicate that humans employ distinct associative learning processes, similar to those used by animal subjects, resulting in a spectrum of cue-motivated behaviors. This behavioral spectrum spans from highly deliberative individuals (goal-trackers) to those who are highly impulsive (sign-trackers). In studies with both children and adults, the tendency to sign-track was associated with a lack of inhibitory control and other behavioral traits, like impulsivity (Colaizzi et al., 2023; Cope et al., 2023; Garofalo & di Pellegrino, 2015). These results strengthen the connection between the degree of sign-tracking behavior and externalizing traits, suggesting that individuals predisposed to sign-tracking may be more susceptible to externalizing disorders characterized by behavioral disinhibition. In relation, the fMRI studies corroborate the existence of a dual-systems framework, emphasizing the dissociation between cortical and subcortical control in goal-trackers and sign-trackers, respectively. In sum, these human studies highlight the value of the sign-tracker/goal-tracker animal model and its potential utility in advancing our understanding of psychiatric symptomatology and the underlying neural processes.

Computational correlates of sign-tracking and goal-tracking behavior

The ability to parse predictive and incentive learning processes using the sign-tracker/goal-tracker model has informed computational learning theories as well (Cinotti et al., 2019; Huys et al., 2014; Lesaint et al., 2014; Moin Afshar et al., 2023; Schad et al., 2020). In particular, the use of classical reinforcement learning models – model-free versus model-based learning – to account for Pavlovian conditioned responses has been revisited since the emergence of the sign-tracker/goal-tracker rodent model (T. E. Robinson & Flagel, 2009). The findings demonstrating that dopamine is involved in incentive, but not predictive, learning (Flagel, Clark, et al., 2011) were especially powerful in prompting the reconsideration of the classical reinforcement learning models and the underlying neural substrates (Huys et al., 2014; Lesaint et al., 2014; Schad et al., 2020).

In the model-free framework, learning is believed to occur via reward prediction errors (RPEs), which, as noted above, serve as a “teaching signal” (Fiorillo et al., 2003). The difference between the value of a reward (US) and the prediction of that value made on the basis of the preceding cue (CS) is quantified with expectations and actions adjusted accordingly (Huys et al., 2014). Model-free learning does not account for potential changes in environmental or physiological circumstances, but rather focuses on inconsistencies between a cue and its associated reward, and any consequential adjustments in the predicted value (Dayan & Berridge, 2014; Gläscher et al., 2010; Huys et al., 2014). Simply put, model-free learning can be likened to living in the moment, experiencing events, and observing the consequences.

In contrast to model-free learning, for which emphasis is placed on retrospectively evaluating past experiences, model-based learning relies on prospective planning. Thus, model-based learning describes the mental process through which individuals construct an internal representation of how different aspects of their environment, such as their own physiological state or their perceptions of stimuli, situations, and conditions, may change when they take various actions. This mental model assists in the selection of actions that are expected to result in the most rewarding outcome (Dayan & Niv, 2008). This process involves a prospective evaluation, similar to planning, and is closely associated with goal-directed behaviors (Gläscher et al., 2010). Briefly, a model-based strategy requires the construction of a model that is based on past experiences and/or associated outcomes. Subsequently, this model is used to predict the most advantageous sequence of actions while considering the individual’s present or anticipated state. Here the value of a reward or outcome is prioritized, therefore the value of stimuli, such as cues, can be adjusted based on the current state of the individual and their environment (Dayan & Berridge, 2014; Huys et al., 2014).

As described above, the so-called reward prediction error (RPE) signal encoded in the activity of dopamine neurons is apparent in sign-trackers, but not goal-trackers, and this has been shown in both rodents (Flagel, Clark et al., 2011) and humans (Schad et al., 2020). Further, in rodents, dopamine blockade prevents the learning of a sign-tracking conditioned response, but not goal-tracking (Flagel, Clark, et al., 2011; Saunders & Robinson, 2012). These findings, together with the fact that sign-trackers will work to obtain a cue (CS) after learning the associative cue-reward relationship (T. E. Robinson & Flagel, 2009), support the notion that sign-trackers rely on model-free Pavlovian learning. That is, the predictive value of the cue is cached and can conflate stimulus identity and the reward, rendering the cue itself rewarding (Schad et al., 2020). In contrast, goal-trackers appear to rely on model-based learning, which is thought to be more resistant to Pavlovian response biases and depend on higher-level cognitive processing (Dayan et al., 2006; Dayan & Berridge, 2014; Schad et al., 2020; Sebold et al., 2016).

To determine if the model-free and model-based computations utilized by sign-trackers and goal-trackers, respectively, on Pavlovian-based procedures are conserved across tasks, Moin Afshar and colleagues (2023) employed an instrumental multistage decision-making (MSDM) task. The MSDM task has reliably been used to determine the learning strategies underlying instrumental behavior (Daw et al., 2011; Groman et al., 2019). Thus, Moin Afshar and colleagues (2023) first characterized rats as sign-trackers or goal-trackers based on their Pavlovian conditioned approach response and subsequently examined behavior using the MSDM task. Their findings suggest that Pavlovian and instrumental learning processes are driven by common reinforcement-learning mechanisms. Specifically, sign-tracking behavior was associated with greater reward-mediated, model-free reinforcement learning, which was linked to model-free reinforcement learning on the instrumental MSDM task (Moin Afshar et al., 2023). Thus, in sign-trackers, the mechanism responsible for assigning value to reward-associated cues is also responsible for continuously updating representations following rewarded actions. There was no relationship observed between Pavlovian conditioned approach behaviors and model-based updating in the MSDM task, but this is likely due to the limited number of goal-tracker rats in this study (Moin Afshar et al., 2023).

Taken together, behavioral, neural, and computational data suggest that that sign-trackers rely on dopamine-dependent model-free learning which underlies incentive learning (Dayan et al., 2006; Dayan & Berridge, 2014; Schad et al., 2020; Sebold et al., 2016). As a result, for sign-trackers, reward cues (CS) are transformed into motivationally relevant stimuli and become “wanted” and capable of controlling behavior in a potentially maladaptive manner. In contrast, goal-trackers appear to rely on dopamine-independent model-based learning and cognitive representations of rewards and their associated stimuli to guide behavior.

Conclusion

The sign-tracker/goal-tracker model highlights the concept of individual differences in motivated behavior, which is extremely relevant to the human condition, but often overlooked. The animal model has proven itself as a reliable, translational tool for investigating the fundamental mechanisms underlying cue-motivated behavior. The current use of this model has expanded our understanding of reward learning on nearly all levels of investigation, allowing us to build upon learning theories and add novel perspectives to the field. This animal model stands on an increasingly strong foundation that promotes progressive hypotheses that can be tested on various levels of interpretation (i.e., behaviorally, neurobiologically, computationally, clinically). Further investigations utilizing this animal model can uncover the mechanisms through which distinct learning patterns influence vulnerability to addiction and other psychiatric disorders characterized by hyperresponsivity to emotionally relevant stimuli. Furthermore, using this model, research may provide insights into the modifiability of these neurobehavioral patterns with the aim to reduce the risk or symptoms of disorders of behavioral disinhibition.