On the Motivational Properties of Reward Cues: Individual Differences

Abstract

Cues associated with rewards, such as food or drugs of abuse, can themselves acquire motivational properties. Acting as incentive stimuli, such cues can exert powerful control over motivated behavior, and in the case of cues associated with drugs, they can goad continued drug-seeking behavior and relapse. However, recent studies reviewed here suggest that there are large individual differences in the extent to which food and drug cues are attributed with incentive salience. Rats prone to approach reward cues (sign-trackers) attribute greater motivational value to discrete localizable cues and interoceptive cues than do rats less prone to approach reward cues (goal-trackers). In contrast, contextual cues appear to exert greater control over motivated behavior in goal-trackers than sign-trackers. It is possible to predict, therefore, before any experience with drugs, in which animals specific classes of drug cues will most likely reinstate drug-seeking behavior. The finding that different individuals may be sensitive to different triggers capable of motivating behavior and producing relapse suggests there may be different pathways to addiction, and has implications for thinking about individualized treatment.

1. Introduction

Some things that promote survival are endowed by evolution with rewarding properties. “Natural” rewards have the ability to produce pleasure (they are “liked”), they act as incentives, motivating behavior (they are “wanted”), and they increase the frequency of actions that produce them (they act as positive reinforcers; Berridge, 2001; Berridge and Robinson, 2003). However, much daily behavior is not controlled directly by primary rewards themselves, but by previously neutral stimuli that have a predictive relationship with biologically significant events or objects (unconditional stimuli, USs). Stimuli (sights, sounds, smells, places) that predict the receipt or availability of rewards can acquire a number of important properties by which they can instigate and control behavior. Best known of these is the ability to act as a conditional stimulus (CS). That is, because of an association with a reward, previously neutral stimuli can acquire the ability to evoke conditional responses (CRs), which are sometimes similar to responses evoked unconditionally by the primary reward itself (unconditional responses, URs). For example, even in the absence of food, a cue or context previously associated with food can evoke conditioned insulin release and salivation. Of course, drugs of abuse are not natural rewards in the sense that they necessarily promote survival, but they have many of the properties of natural rewards (Nesse and Berridge, 1997), and stimuli previously associated with drugs can also evoke CRs. For example, a stimulus associated with cocaine can produce conditioned cardiovascular responses (Cascella et al., 1989).

However, reward cues not only acquire the ability to evoke simple reflexive or autonomic CRs, but they may also acquire the ability to directly activate complex emotional and motivational states (Berridge, 2001; Bindra, 1978; Lajoie and Bindra, 1976; Rescorla, 1988), and as such, to act as incentive stimuli. This paper will focus primarily on a recent series of studies on individual variation in the extent to which reward associated cues (especially food and drug cues) acquire incentive stimulus properties, and thus the ability to exert control over motivated behavior.

2. Incentive Stimuli

Incentive stimuli (stimuli attributed with incentive salience, Zhang et al., 2009) are defined as stimuli that acquire three fundamental properties because of their relationship with a reward (Berridge, 2001; Bindra, 1978; Cardinal et al., 2002; Milton and Everitt, 2010). One, incentive stimuli bias attention towards them, and if they are localizable, they attract, eliciting approach into close proximity with them (often quantified by Pavlovian conditioned approach behavior). Given that cues predictive of rewards are often located at the same place as the reward itself, this feature of an incentive stimulus will tend to draw an individual to the location where a reward is to be found. Two, incentive stimuli themselves are sought after, in the sense that animals will work to get access to them (i.e., they can act as conditioned or secondary reinforcers). This feature of an incentive stimulus can maintain (reinforce and motivate) reward-seeking behavior over long periods of time in the absence of the primary reward. Three, an incentive stimulus can evoke the relevant motivational state (`conditioned motivation'), thus instigating seeking for a reward, or energizing ongoing seeking behavior. This feature of an incentive stimulus is measured by so-called Pavlovian-to-Instrumental transfer (PIT) effects in both humans (Talmi et al., 2008) and non-human animals (Estes, 1943, 1948; Holmes et al., 2010; Lovibond, 1983), and can also be measured in humans by implicit measures of desire or subjective craving states (Hester et al., 2006; Rosenberg, 2009). In the laboratory these three features of an incentive stimulus are psychologically and neurobiologically dissociable, but as pointed out by Everitt and his colleagues (Cardinal et al., 2002; Milton, 2012; Milton and Everitt, 2010), under “real world” conditions they often act in concert to motivate behavior directed towards the acquisition of desirable outcomes. It is important to note that incentive stimuli can influence behavior implicitly, acting outside of conscious awareness (e.g., Childress et al., 2008), and at other times their effects may rise to the level of conscious awareness, resulting in many complex cognitive processes – including rationalizations for decisions and actions already made (Robinson and Berridge, 1993).

The ability of incentive stimuli to motivate actions and bias behavior towards particular outcomes is highly adaptive, because this will increase the likelihood that an animal will acquire rewards that are necessary for survival (e.g., food, water, safety) and for propagation of the species (e.g., a mate). However, incentive stimuli can also act as temptations that promote maladaptive behavior. For example, in modern environments flooded with cues that signal the availability of an inordinate abundance of high fat and sugar-rich foods, such cues can motivate over-eating, contributing to obesity (Berridge, 2012; Cornell et al., 1989; Jansen, 1998). Importantly, there is considerable individual variation in the ability to resist temptations provoked by reward-associated cues. In the following we will first briefly summarize preclinical studies in rats indicating that there are large individual differences in the propensity of animals to attribute incentive motivational properties (incentive salience) to food cues. Then, we will review studies showing that the extent to which rats attribute incentive salience to food cues predicts their propensity to attribute incentive salience to particular classes of drug cues – which has implications for thinking about individual variation in susceptibility to addiction.

3. Individual Variation – Food Cues

In many studies of appetitive Pavlovian approach conditioning, using rats, an auditory stimulus (CS) predicts food or water delivery (US) into a cup a few seconds later. In this situation, rats quickly learn, upon presentation of the tone, to approach the hopper where the cup is located, and they make anticipatory head entries into it (Cleland and Davey, 1983; Farwell and Ayres, 1979; Holland, 1977, 1980; Wassum et al., 2011). Rats do not learn to approach the location of an auditory cue itself, even if it is localizable (Cleland and Davey, 1983). In many other studies of this sort the CS may consist of a house light that illuminates the entire test chamber (Holland, 1977; Holland, 1981), a compound tone-light CS (Ross and Holland, 1981), or a discrete light-CS located either within the hopper holding a food cup or close to the food cup (Durlach and Shane, 1993; Wan and Peoples, 2006). As with an auditory CS, in these latter situations the CR consists primarily of anticipatory head entries into the hopper. In yet other studies a light-CS is located above the food hopper and in this case one may see more than one response – initial orientation towards the light (rearing) followed by head movements into the food hopper (Gallagher et al., 1990; McDannald et al., 2004). In all of these situations the CR that emerges with CS-US pairings primarily consists of anticipatory approach to the goal – the location of reward delivery – and head poking into it. This CR is typically referred to as a `goal-tracking' (GT) (Boakes, 1977). Importantly, with these procedures there is relatively little individual variation in what CR is acquired; that is, animals may vary in the rate at which they acquire the CR, but their behavior is dominated by a GT CR.

However, it has been known for many years that when a discrete localizable cue (like a focal light source or lever) predicts the delivery of a food reward at a different location than where the CS is located, some animals come to find the cue itself attractive, and with repeated CS-US pairings they start to approach and engage it (Brown and Jenkins, 1968; Jenkins et al., 1978; Zener, 1937). For example, pigeons peck at an illuminated disk (Brown and Jenkins, 1968) and rats bite and nibble on a lever that predicts food delivery some distance away (Davey and Cleland, 1982; Hearst and Jenkins, 1974; Mahler and Berridge, 2009). This behavior is called a `sign-tracking' (ST) CR, because animals are attracted to the cue or `sign' associated with reward delivery (the procedure itself is sometimes called `autoshaping'). Indeed, the cue itself can become so powerfully attractive that animals may continue to approach it even if this results in loss of the reward (Hearst and Jenkins, 1974; Killeen, 2003; Williams and Williams, 1969). However, it has been frequently noted that there is considerable individual variation in the acquisition of a ST CR – not all animals acquire this behavior even when they are trained under exactly the same conditions (Boakes, 1977; Davey and Cleland, 1982; Hearst and Jenkins, 1974; Tomie et al., 2000). This is not because they fail to learn the CS-US association, but it turns out that in this situation, rather than acquiring a ST CR, many rats acquire a GT CR (Flagel et al., 2009; Flagel et al., 2007; Meyer et al., 2012a; Robinson and Flagel, 2009; Saunders and Robinson, 2013), as first described by Boakes (1977). Thus, both sign-trackers (STs) and goal-trackers (GTs) learn the relationship between the CS and US, and both even learn a Pavlovian conditioned approach response; they just direct their behavior to different locations in the environment upon CS onset. Interestingly, whilst GTs do not approach the CS they do acquire a conditioned orienting response directed towards it (Yager and Robinson, 2013). That is, in rats, upon presentation of a lever-CS both STs and GTs initially orient towards the lever, but then only STs approach into close proximity with it. After an initial glance at the lever GTs do not approach it but instead direct their behavior towards the food cup, a phenomenon first described in dogs by Zener (1937). Other rats are ambivalent and vacillate between making a ST CR and a GT CR. We have now characterized these phenotypes in a large number of rats and their distribution in the population is provided in Meyer et al. (2012a).

Based on these studies it is clear that both STs and GTs learn a CS-US association, as indicated by the fact that both acquire a conditioned orienting response, and that in both the CS evokes a conditioned approach response (sign-tracking or goal-tracking, respectively). So, why does a stimulus that is a perfectly good predictor, and acts as an “excitor” (CS+) in both STs and GTs, become itself attractive only in STs? We have suggested an answer by considering the data in the theoretical framework of incentive motivation (Berridge, 2001; Bindra, 1978; Bolles, 1972; Toates, 1986). We have hypothesized that for rats that learn a ST CR the CS itself (the lever) becomes attributed with incentive salience. This is indicated by the fact that for these animals it acquires one of the properties of an incentive stimulus; it becomes attractive and they approach it. Importantly, we have shown that it also acquires other properties of an incentive stimulus preferentially in STs. The lever-CS serves as a more effective conditioned reinforcer in STs than GTs, in that it is more effective in reinforcing a new instrumental response (Lomanowska et al., 2011; Robinson and Flagel, 2009). Furthermore, a cue associated with food delivery in an instrumental task is more effective in reinstating food-seeking behavior, following extinction of the instrumental response, in STs than GTs (Yager and Robinson, 2010). We conclude that the cue is an equally effective CS in both STs and GTs, but it acts as a potent incentive stimulus only in STs. These studies are important because they establish that the acquisition of conditioned stimulus (CS+) properties is not in itself sufficient to confer incentive stimulus properties upon a reward associated cue. A perfectly effective CS+ may or may not also act as an incentive stimulus.

Presumably the psychological processes governing conditioned responding differ in STs and GTs. The best evidence to date that this is the case comes from studies showing that the learning and the performance of ST and GT CRs are mediated by dissociable neural systems (Clark et al., 2012; Danna and Elmer, 2010; Flagel et al., 2011a; Flagel et al., 2011b; Saunders and Robinson, 2012). Flagel et al. (2011b) reported that the systemic administration of the D1/D2 dopamine (DA) antagonist, flupenthixol (FLU) blocked the acquisition of a ST CR, but not a GT CR. In addition, learning a ST CR was associated with the development of a phasic DA response to the CS in the core of the nucleus accumbens, but learning a GT CR was not. More recently, Saunders and Robinson (2012) examined the effect of FLU microinjection into the core of the accumbens on the performance of a ST and GT CR after the CRs were already well learned. FLU dose-dependently attenuated the performance of a ST CR, but not a GT CR. Furthermore, DA blockade did not attenuate performance of the conditioned orienting response, even in STs. This observation, along with the fact that DA blockade attenuated performance of a ST CR on the very first trial, suggests that DA blockade degraded, “the motivational properties of the CS, which are required for the CS to become attractive, but without necessarily compromising the CS–US association” (Saunders and Robinson, 2012, p. 2529). These data, along with evidence that a lever-CS engages different brain regions in STs and GTs (Flagel et al., 2011a), supports the hypothesis that sign-tracking and goal-tracking reflect the operation of different neural systems, and therefore, presumably different psychological processes (see Meyer et al., 2012a; Saunders and Robinson, 2012 for further discussion of this point). We have proposed, therefore, that STs and GTs vary on a trait characterized by a the propensity to attribute incentive salience to localizable reward cues (Meyer et al., 2012a). With this hypothesis in mind we recently conducted a series of studies asking whether variation in the propensity to attribute incentive salience to a food cue predicts the extent to which drug cues acquire motivational control over behavior. These studies are reviewed next.

4. Individual Variation in the Extent to Which Drug Cues Acquire Incentive Stimulus Properties

The idea that incentive stimuli are especially important in drug-motivated behavior has a long history. Nearly 30 years ago, building on earlier work by Bindra and others (Bindra, 1978; Bolles, 1972; Konorski, 1967; Toates, 1981; Young, 1966), Stewart et al. (1984) argued that,

“need and drive views of motivation are gradually being replaced by a view … that ascribes a primary role to incentive stimuli as the generators of motivational states and elicitors of actions” (p.251). They went on to state that it is, “the drug itself, or the presentation of a stimulus previously paired with the drug, [that] acts to create a motivational state that facilitates drug-seeking behavior” (p. 256).

Indeed, each feature of an incentive stimulus discussed above may contribute to drug use in different but complementary ways, providing what Milton and Everitt (2010) described as “three routes to relapse”. The ability of an incentive stimulus to attract attention to it and to elicit approach behavior will bring an addict into close proximity to places where drugs are to be found, or devices for administering drugs. The ability of an incentive stimulus to act as a conditioned reinforcer will maintain drug-seeking behavior even when the drug itself is not immediately available. This feature can be especially insidious because it is resistant to extinction (Di Ciano and Everitt, 2004) and persists even when the primary reinforcer is devalued (Davis and Smith, 1976; Parkinson et al., 2005). Finally, the ability of an incentive stimulus to arouse a state of conditioned motivation (desire) for drug will serve to both maintain ongoing drug-seeking and drug-taking behavior, and during a period of abstinence it may trigger relapse, especially if the attractive features of the drug cue have brought an individual into proximity with drugs.

Although there have been significant advances in understanding the critical role drug cues and contexts play in controlling behavior, and the neurobiological systems by which they exert control (Bouton and Swartzentruber, 1991; Caggiula et al., 2001; Cardinal et al., 2002; Leyton and Vezina, 2013; Milton and Everitt, 2010; Phillips et al., 2008; Shaham et al., 2003; Stewart et al., 1984; Tomie et al., 2008; Volkow et al., 2006; Wheeler and Carelli, 2009), there are still significant gaps in our knowledge. One such gap is - why do some individuals, but not others, have such difficulty resisting drug cues? Put another way, why do cues act as potent incentive stimuli, motivating drug-seeking and consumption, to a much greater degree in some individuals than others?

To begin to explore this question we asked whether rats prone to attribute incentive salience to a food cue are also prone to attribute incentive salience to cues associated with drugs; that is, is this phenotype specific to food cues or does it represent a more general trait that may bias behavior controlled by different classes of incentive stimuli. To do this, and using a drug (mostly cocaine) as the US, we have examined each of the defining characteristics of an incentive stimulus: its ability to elicit approach towards it, to serve as a conditioned reinforcer, and to evoke a conditioned motivational state, using multiple behavioral measures. Given that these properties of an incentive stimulus are dissociable, and mediated by somewhat different (although overlapping) neural systems (Cardinal et al., 2002; Milton and Everitt, 2010), it is important to examine all three. In the following each property of an incentive stimulus is discussed in turn.

4.1. Conditioned Approach

There have been a number of reports that use of an autoshaping procedure can facilitate the acquisition of lever-pressing for a drug reward (Campbell and Carroll, 2000; Carroll and Lac, 1993, 1997). However, in these studies, the procedure also involved reinforcing a lever press by drug delivery, and therefore, it was not a strictly Pavlovian procedure and does not provide an unambiguous measure of Pavlovian approach behavior. It was unclear until only recently whether a cue associated with drug delivery, using strictly Pavlovian conditioning procedures, would come to elicit approach towards it, that is, produce a ST CR. Tomie and colleagues (Tomie, 2001; Tomie et al., 2003) first reported that rats learned to approach a cue that predicted the delivery of a sweetened alcohol solution, but despite a number of controls to insure the ST CR was motivated by the alcohol, there was some concern whether the sweet solution might have played a role. The first report that an intravenous (IV) injection of a drug can produce a ST CR, at least in some animals, was by Uslaner et al. (2006), who paired a lever-CS with an IV injection of cocaine. Since then there have been a number of reports that a drug US can support learning a ST CR (Aragona et al., 2009; Flagel et al., 2010; Yager and Robinson, 2013), including unsweetened alcohol (Krank et al., 2008). But, as with a food US (Tomie et al., 2000), there is considerable individual variation in the extent to which rats will approach a drug cue.

We recently asked whether individual variation in the propensity to attribute incentive salience to a food cue, as described above, predicts variation in the propensity to approach a drug cue. In the first study to do this Flagel et al. (2010) used two lines of rats selectively-bred for high (bHR) or low (bLR) locomotor activity when placed in a novel environment. It so happens that when using food as the US, bHR animals almost exclusively learn a ST CR and bLR animals a GT CR, and therefore, these phenotypes can be accurately predicted prior to Pavlovian training (Flagel et al., 2011b; Flagel et al., 2010). When a lever-CS was repeatedly paired with an IV injection of cocaine bHR animals (STs) learned to approach the cocaine-associated lever, and did so more and more rapidly. It is important to note that when cocaine is used as the US rats do not reliably deflect the lever, as reported by Uslaner et al. (2006), similar to when rewarding electrical brain stimulation is used as the US (Peterson et al., 1972). Thus, the CR consisted of approach, sniffing and exploration in the immediate vicinity of the lever-CS. In contrast, bLR (GTs) animals did not acquire a ST CR. Of course, when using drug as the US there is no `goal' to approach, and so in this study no GT CR was evident. This raises the question of whether bLR animals did not approach the cocaine cue because it was not attributed with sufficient incentive salience, or, whether they failed to learn the CS-US association.

This question was addressed by Yager and Robinson (2013) in a study using outbred Sprague-Dawley rats. After identifying rats as STs or GTs using the standard Pavlovian conditioning procedures described above (lever-CS and food US), illumination of a light was associated with an IV injection of cocaine (Paired groups). Independent groups received the same number of light presentations and cocaine injections but these were not associated in time (Unpaired groups). In this situation an approach CR was recorded if a rat brought its nose into close proximity to the light-CS during the CS period, which because of the location of the light required the rat to rear. In addition to scoring conditioned approach, Yager and Robinson (2013) also scored whether an animal made an orienting response to the light-CS, defined as making a head and/or body movement in the direction of the CS during the CS period, even if it did not rear in close proximity to the light. It was found that Paired STs showed greater conditioned approach to the cocaine cue than Paired GTs, and the higher dose of cocaine elicited greater approach than the lower dose. Interestingly, both Paired STs and GTs (but not Unpaired STs and GTs) learned a conditioned orienting response to the cocaine cue, and did not differ in conditioned orienting behavior, even though GTs were less likely to approach the cocaine cue. This is important because it establishes that GTs learned the CS-US association, even though they were less strongly attracted to the CS than STs. Finally, in unpublished studies Yager and Robinson found similar individual variation in the ability of a cue paired with the mu opioid agonist, remifentanil, to elicit approach towards it, suggesting this effect is not specific to psychostimulant drugs.

4.2. Conditioned Reinforcement

Another well-established way of assessing whether a reward-associated cue has acquired incentive motivational properties is to determine if animals will work to obtain it; that is, whether it will serve as a conditioned reinforcer (Berridge, 2001; Cardinal et al., 2002). In these kinds of studies the reward cue is presented contingent upon an instrumental action, and the question is whether the cue itself will reinforce instrumental responding in the absence of the primary reinforcer. As noted above, we have reported in a number of different studies that a food cue is a more effective conditioned reinforcer in rats that learn a ST CR than in those that learn a GT CR (Lomanowska et al., 2011; Meyer et al., 2012a; Robinson and Flagel, 2009; Yager and Robinson, 2010). The question here is whether the tendency to find a food cue attractive also predicts the ability of a drug cue to act as a conditioned reinforcer.

In two studies we used an extinction-reinstatement procedure (Shaham et al., 2003), following a period of cocaine self-administration, to assess the ability of a cocaine cue to reinforce/motivate responding in the absence of the primary reinforcer. In one study the cocaine cue acquired motivational properties in an instrumental (self-administration) setting and in the other using a classic Pavlovian conditioning procedure. Saunders and Robinson (2010) trained rats to self-administer IV cocaine, and in this experiment the instrumental response required to receive cocaine was a nose poke into a port that also was illuminated upon that action. Thus, the light in the nose port served as the cocaine-associated cue, and like cocaine itself, its presentation required an action. After the animals acquired a stable pattern of self-administration behavior they underwent extinction training, during which time a nose poke produced neither cocaine nor the light cue, and responding fell to low levels. Later, the critical reinstatement test took place, where a nose poke resulted in illumination of the nose port, but no cocaine was delivered. We found that STs made significantly more nose pokes for the cocaine cue than GTs, indicating that the cocaine cue acted as a more effective conditioned reinforcer in STs than in GTs.

In another experiment Saunders and Robinson (2010) looked at the ability of a cocaine cue to maintain ongoing self-administration behavior, using a cue removal procedure. There are a number of reports that during self-administration the cue associated with drug delivery plays an important role in maintaining high levels of responding, because its omission decreases self-administration behavior (Arroyo et al., 1998; Caggiula et al., 2001; Schenk and Partridge, 2001). Removal of the light that accompanied cocaine self-administration greatly decreased responding in STs but not GTs, despite the fact that a nose poke still produced an injection of cocaine. This indicates that the cue itself played an important role in reinforcing (and/or motivating) ongoing self-administration behavior in STs, but not in GTs, which is consistent with the results of the extinction-reinstatement study described above.

Cues that acquire motivational properties in an instrumental setting, as above, may rely on somewhat different psychological and neurobiological processes than cues that acquire motivational properties through classic Pavlovian conditioning, whereby the cue is associated with reward independent of an animal's action (Cardinal et al., 2002; Dickinson et al., 2000; Thomas et al., 2003; Thomas and Everitt, 2001). The latter situation is often the case in humans, where cues are present prior to an action resulting in drug administration – they do not suddenly appear as a consequence of taking a drug, thus reinforcing the action. We thought it important, therefore, to determine whether a cue associated with cocaine using Pavlovian conditioning procedures would also reinforce drug-seeking behavior differently in STs and GTs, again using an extinction-reinstatement procedure.

Yager and Robinson (2013) trained STs and GTs (identified by the propensity to approach a food cue) to self-administer IV cocaine by making a nose poke, but importantly, in this experiment cocaine self-administration was not accompanied by the presentation of any cue. After self-administration behavior became stable, the animals received two days of Pavlovian training, during which time the nose ports were removed and animals were given experimenter-administered IV injections of cocaine (US), paired with illumination of a light. After Pavlovian training they were again allowed to self-administer cocaine until their behavior was stable. Following this, all animals underwent extinction training, during which time a nose poke had no consequence, and responding fell to low levels in all animals. After the last day of extinction the crucial test for reinstatement of drug seeking behavior was conducted. On this day the animals were placed back into the test chamber, as on the last day of extinction, but now a nose poke resulted in illumination of the Pavlovian cocaine cue light (but cocaine was not delivered) (Kruzich et al., 2001; See, 2005). We found that the cocaine cue reinforced significantly more responding in STs than GTs.

In summary, we have found that a cocaine cue serves as a more effective conditioned reinforcer in STs than GTs, whether the cue acquires its motivational properties in an instrumental setting (i.e., during self-administration) or using a Pavlovian conditioning procedure. It is important to note that in the Yager and Robinson (2013) study, unlike Saunders and Robinson (2010), the cocaine cue had never before been presented contingent upon any action, prior to the reinstatement test. Thus, the cocaine cue could not evoke a learned S-R habit, and therefore, this provides a more stringent test of the motivating properties of the cocaine cue.

One additional study (Meyer et al., 2012b) used a very different procedure, whereby the drug cue acquired motivational properties using a Pavlovian training procedure, but the test for conditioned reinforcement consisted of determining whether the drug cue would reinforce actions to stay in close proximity to the cue. This is a modification of a conditioned place preference procedure, better termed conditioned cue preference (Cunningham et al., 2006; van der Kooy, 1987). After standard Pavlovian training using food as the US, to identify STs and GTs, rats were placed into a chamber with a floor that had one of two textures and given an IP injection of either of cocaine or saline. The next day, rats received the opposite injection paired with a different floor, and this conditioning continued for several successive days. This procedure did not involve context (place) conditioning, because importantly, training was conducted in the dark and therefore there was only one cue that was reliably associated with cocaine, the texture of the floor. On the test day animals were placed back into the chamber but now half the floor surface consisted of the cocaine-associated texture and the other half the saline-associated texture, and the time spent in contact with each tactile stimulus was recorded. Meyer et al. (2012b) found that STs showed a greater preference for the cocaine-associated floor than did GTs, suggesting that contact with the cocaine-associated floor was more effective in reinforcing actions necessary to maintain contact with it in STs than GTs (Cunningham et al., 2006; Vezina and Stewart, 1987).

In summary, there are now a number of studies (Flagel et al., 2010; Meyer et al., 2012b; Saunders and Robinson, 2010; Yager and Robinson, 2013), using quite different procedures, all indicating that the propensity to attribute incentive salience to a food cue predicts the extent to which a cocaine cue acquires incentive motivational properties, in this case, as assessed by its ability to reinforce actions to get it (i.e., to act as a conditioned reinforcer).

4.3. Conditioned Motivation

As well as eliciting approach towards them and reinforcing actions already taken, incentive stimuli are also capable of generating conditioned motivational states that can instigate actions to obtain a reward and/or to energize ongoing seeking behavior. Many theorists have described this motivational property of Pavlovian CSs, and its ability to influence instrumental actions (Berridge, 2001; Bindra, 1968; Milton and Everitt, 2010; Rescorla and Solomon, 1967). For example, Bindra (1968) referred to the ability of Pavlovian CSs to generate what he called a “central motive state”. As Berridge (1996) has pointed out, generation of a conditioned motivational state may influence behavior implicitly (Childress et al., 2008), as captured in part by the concept of “wanting” or “craving” (in quotation marks), or, when it arises to the level of conscious awareness as wanting or craving (without quotation marks), which refers to, “a conscious cognitive desire for a declarative goal in the ordinary sense of the word” (Berridge et al., 2010, p.45).

In preclinical studies the ability of incentive stimuli to generate conditioned motivation is usually assessed by the ability of a Pavlovian CS to influence instrumental actions, as indicated by so-called Pavlovian-to-Instrumental Transfer (PIT) effects (Estes, 1943, 1948; Holmes et al., 2010; Lovibond, 1983; Rescorla and Solomon, 1967). In a typical PIT experiment an animal is trained to make an instrumental action to receive a reward, such as bar pressing for a food pellet. In separate Pavlovian conditioning sessions a CS (usually a tone) is paired with food delivery, independent of any action. On the test day the Pavlovian CS is briefly presented during performance of the instrumental action (usually under extinction conditions) and it is observed that the Pavlovian CS increases the rate of instrumental responding, even though the CS had never before been associated with that action. This is PIT, and it is thought to reflect the generation of a Pavlovian conditioned motivational state that energizes instrumental actions to obtain the reward. Interestingly, Wassum et al. (2013) recently reported, using a PIT procedure, that a surge in dopamine neurotransmission in the core of the accumbens is associated with the invigoration of behavior produced by a food cue. This is consistent with earlier reports that increasing dopamine transmission in the accumbens by local injections of amphetamine, or sensitization to amphetamine, increases cue-evoked pursuit of a food reward (Wyvell and Berridge, 2000; Wyvell and Berridge, 2001). As an aside, there are both general (Balleine, 1994) and specific forms of PIT (Colwill and Rescorla, 1988; Kruse et al., 1983) but that will not be discussed here.

There have been many demonstrations of PIT using natural rewards (see Holmes et al., 2010 for review), but very few using a discrete CS and a drug as the US (Corbit and Janak, 2007; LeBlanc et al., 2012). In a recent example, LeBlanc et al. (2012) first paired an auditory stimulus with an IV injection of cocaine and the rats were then trained to self-administer cocaine using a seeking-taking chain in which responses on one lever (seeking lever) would give access to a second taking lever, on which a response delivered cocaine. Although the exact conditions varied, to optimize the probability of seeing PIT, the test for PIT consisted of presentation of the CS+ (or CS−) and quantification of the effect on instrumental responding (under extinction conditions). They reported that, “Rats showed significant transfer, increasing task performance during cocaine-paired cues” (p. 681), and concluded that, “cocaine-paired cues can provoke the pursuit of cocaine through a Pavlovian motivational process” (p. 681).

We were interested in examining whether STs and GTs differ in the extent to which a drug cue acquires conditioned motivational properties, as indicated by PIT. However, this is complicated by the fact that if an auditory stimulus is used as the CS both STs and GTs attribute incentive salience to it, as indicated by its ability to serve as a conditioned reinforcer (Meyer et al., 2010). This precludes the use of an auditory CS to explore individual variation. The distinction between STs and GTs is most evident when the CS is discrete and localizable. But in a typical PIT experiment this kind of CS is problematic because of response competition – presentation of a lever-CS, or a focal light-CS, for example, would draw STs (but not GTs) towards it, thus interfering with the ability to measure any increase in the rate of an ongoing instrumental action.

To circumvent this problem Saunders and Robinson (2011a) modified a procedure developed by Cooper et al. (2007), to examine the ability of a drug cue to instigate an instrumental action (rather than to modify the rate of an ongoing action, Marchant et al., 2013). After determining the propensity to approach a food cue, rats were trained to self-administer IV cocaine, and cocaine injections were paired with illumination of the nose port (the CS). After acquiring stable self-administration behavior, the front two thirds of the floor of the chamber was electrified (first with a very low current and then across days of testing with increasing current) such that to make a nose poke and receive drug the animal had to cross the electrified floor. The back one third of the floor was not electrified, so an animal could choose to not take drug, in which case it would not receive any shock. As the current increased across days all animals decreased their level of responding, until they essentially stopped responding. Thus, in this case, abstinence was a consequence of increasing negative consequences. On the test day the animals were again placed into the chamber, with the floor still electrified, and the drug cue presented for 20 sec once every 3 min. Critically, during this test, cue presentation was not contingent upon the animal making any action and a nose poke did not result in presentation of the cue or delivery of the drug (that is, the cue could not act as a conditioned reinforcer). Saunders and Robinson (2011a) found that the cocaine cue was more effective in reinstating responding in STs than GTs, in the face of an aversive consequence, and in fact, using all animals (including intermediates) there was a significant positive correlation (r²=0.25) between the propensity to approach a food cue and the ability of the cocaine cue to reinstate drug-seeking behavior.

Our interpretation of these findings is that non-contingent presentation of the cocaine cue aroused a conditioned motivational state that instigated drug-seeking behavior, even in the face of continued adverse consequences, and did so to a greater extent in rats prone to attribute incentive salience to a food cue (STs). This is an example, therefore, of the ability of a Pavlovian cue to influence an instrumental action, and may represent the operation of the same psychological process responsible for more traditional measures of PIT. Saunders and Robinson (2011a) also showed that this effect was dependent on DA in the core of the accumbens, and therefore, this was presumably due to the general form of PIT, which requires dopamine, as the specific form is not dopamine-dependent (Dickinson et al., 2000; Ostlund and Maidment, 2012; Wyvell and Berridge, 2000). One consideration, however, is that in this experiment the cocaine cue acquired its motivational properties in an instrumental setting rather than through classical Pavlovian conditioning. Nevertheless, this may represent a good animal model of the craving evoked by drug cues in addicts (Barnea-Ygael et al., 2012; Cooper et al., 2007).

A second phenomenon, that is also likely due to the ability of a stimulus to arouse a motivational state (“craving” and/or craving) and thereby reinstate drug-seeking behavior and relapse in addicts, is that produced by exposure to drug itself. It is well established that a “taste” of a drug can induce craving and relapse in otherwise abstinent addicts (de Wit and Chutuape, 1993; Jaffe et al., 1989), and a drug prime can reinstate responding in animals following extinction of self-administration behavior (de Wit and Stewart, 1981; Shaham et al., 2003). One interpretation of this effect is that the interoceptive cues produced by the drug have come to be associated with the unconditional motivational properties of the drug itself, and therefore, the interoceptive cues produced by even a small dose of drug produce a conditioned motivational state that instigates drug-seeking and drug-taking behavior. Thus, to further examine individual variation in the ability of drug cues (in this case interoceptive cues) to motivate drug-seeking behavior Saunders and Robinson (2011b) studied drug-induced reinstatement in STs and GTs.

STs and GTs were first trained to self-administer cocaine, and importantly, during self-administration (produced by nose pokes) no experimenter-provided explicit stimulus was associated with the injection of cocaine. These animals then underwent two tests for the ability of cocaine itself to motivate drug-seeking and drug-taking behavior. First, after the acquisition of stable self-administration, rats were tested for two days using a progressive ratio schedule. Second, following extinction training, before being put back into the chamber, the rats were given an IP injection of cocaine, but now nose pokes had no consequence (no drug was delivered). We found that 1) STs worked harder than GTs for cocaine (i.e., they had a higher breakpoint on the progressive ratio schedule), and 2) STs showed significantly more robust drug-induced reinstatement of drug-seeking behavior than did GTs.

Taken together, these studies suggest that there is considerable variation in the ability of either discrete environmental stimuli or interoceptive cues to evoke a conditioned motivational state that instigates drug-seeking and drug-taking behavior. Furthermore, variation in the propensity to attribute incentive salience to a food cue predicts the extent to which these drug cues acquire motivational control over behavior. Of the three features of an incentive stimulus discussed in the introduction to this paper (Berridge, 2001; Cardinal et al., 2002; Lovibond, 1983) the ability of cues to arouse a state of conditioned motivation is probably most closely related to craving states evoked by similar stimuli in humans. It is especially interesting, therefore, that Mahler and DeWit (2010) found that nicotine cues evoke greater craving in those abstinent smokers for whom food cues evoke the greatest craving when they are hungry (also see, Styn et al., 2012).

4.4. Interim summary

The available evidence indicates that discrete localizable drug cues (mostly cocaine cues) acquire all three properties of an incentive stimulus to a greater extent in rats prone to attribute incentive salience to a discrete food cue. A discrete cocaine (and remifentanil) cue is more attractive in STs than GTs, it is a more effective conditioned reinforcer in STs than GTs, and discrete and interoceptive cocaine cues are more effective in evoking a conditioned motivational state in STs than GTs. Of course, it is important to acknowledge that in some of these tests more than one property of an incentive stimulus may simultaneously influence behavior, which can complicate unambiguous interpretations. In a test of conditioned reinforcement, for example, presentation of a reward cue contingent upon an action allows the cue to reinforce the action that preceded its presentation, but at the same time it may serve to keep the animal close to the location of the manipulandum and also evoke a conditioned motivational state that energizes instrumental responding. Indeed, as Tomie (1996) has argued, when reward cues are located close to or as part of a manipulandum, the manipulation of which procures a drug, they may acquire especially strong control over motivated behavior and engender especially strong drug-seeking and -taking behavior. Thus, as pointed out by Milton and Everitt (2010),

“drug-associated conditioned stimuli can influence relapse behaviour through at least three different processes: conditioned reinforcement, conditioned approach and conditioned motivation ([their] Fig. 4). Although these processes are psychologically and neurobiologically separable, and can be studied in isolation in a laboratory setting, in the real world of an addicted individual attempting to remain abstinent, all of these processes can be engaged by drug-associated stimuli and are therefore able simultaneously or sequentially to contribute to relapse; effectively, in pavlovian terms there are `three routes to relapse'” (p. 2314).

Based on our studies described so far we would add that there is considerable individual variation in the propensity to relapse, in part because some individuals are especially prone to attribute incentive salience to drug cues. However, an additional set of experiments by Ben Saunders and his colleagues, discussed next, suggest the reality may be more complicated than that.

5. Variation in the Influence of a Drug-Associated Context on Motivated Behavior

Places where drugs are taken and/or procured – a pub, for example – represent another type of stimulus known to play an important role in relapse in addicts. Contextual stimuli associated with drug use are thought to readily arouse conditioned motivational states (“craving” and/or craving) that can renew drug-seeking and drug-taking behavior in both humans (Foltin and Haney, 2000; Mayo et al., 2013; O'Brien et al., 1992) and rats (Crombag and Shaham, 2002; Fuchs et al., 2005; McFarland and Ettenberg, 1997), although we readily acknowledge that they can, in fact, exert control over behavior via multiple psychological processes (Bouton and Swartzentruber, 1991; Crombag et al., 2008). There is a considerable literature showing that the neural systems involved in learning about contexts (a configuration of cues) are different (although overlapping) from those involved in learning about a simple discrete cue (Cardinal et al., 2002; Fuchs et al., 2008; Parkinson et al., 1999). For example, the difference between context and cue conditioning has been well characterized in the fear conditioning literature (Fendt and Fanselow, 1999; Phillips and LeDoux, 1992).

Given the importance of contextual cues in relapse we were interested in studying their effects in STs and GTs to determine if variation in context control over motivated behavior would be similar to the other classes of stimuli discussed thus far. Saunders et al. (2012) examined this question using two different procedures. One, by quantifying the ability of a drug-paired context to produce conditioned hyperactivity, which is thought to reflect activation of a conditioned motivational state (Beninger et al., 1981; Jones and Robbins, 1992), and two, by quantifying the ability of a drug-associated context to reinstate drug-seeking behavior in STs and GTs (Crombag et al., 2008).

In the first type of experiment rats were classed as STs or GTs using our standard Pavlovian measure of attraction to a food cue. Then, immediately prior to placement into a test chamber for recording locomotor activity STs and GTs received an IP injection of either cocaine (Paired groups) or saline (Unpaired groups), each day for 5–6 days. When returned to their home cages the rats received either saline (Paired groups) or cocaine (Unpaired groups). On the test day all rats received saline before being placed into the locomotor activity chambers and behavior was recorded. In two independent experiments we found that GTs showed greater conditioned locomotor activity than STs. This result is obviously quite different from those described above using other classes of drug-associated cues, but it is similar to one obtained using an aversive stimulus and a fear conditioning procedure. Morrow et al. (2011) found that although STs showed greater cue (tone)-evoked freezing, GTs showed greater context fear conditioning.

Saunders et al. (2012) also looked at context-induced reinstatement of cocaine-seeking behavior in STs and GTs. In this experiment STs and GTs were trained to self-administer cocaine and after the acquisition of stable self-administration behavior all rats underwent extinction training in either the regular self-administration context (Context A) or in a different context (Context B). After the final day of extinction rats were placed back into Context A, which was just an additional day of extinction training for rats extinguished in Context A but re-exposure to the cocaine context for rats extinguished in Context B. Active and inactive nose pokes were recorded but they had no consequence (i.e., no drug and no cue was delivered). This procedure is different from that used in most studies of context-induced reinstatement because in those not only are animals placed back into the drug-associated context but typically active responses now also result in presentation of a discrete drug cue (Crombag et al., 2008). Thus, in previous studies the context can be thought of as renewing the effect of the cue as a conditioned reinforcer, which is not the case here. We found that GTs showed significantly more robust context-induced reinstatement of cocaine-seeking behavior than STs, and this effect required intact dopamine transmission in the core of the accumbens. Taken together, these experiments suggest that exposure to a drug-associated context arouses a dopamine-dependent conditioned motivational state that spurs drug-seeking behavior and that it does so to a greater extent in GTs than STs – an effect opposite of that seen using other types of cues.

6. Summary and Conclusions

We have found, using a number of different measures and procedures, that there is considerable individual variation in the propensity to attribute incentive salience to food cues, and that the propensity to do so predicts the extent to which drug cues acquire motivational control over behavior. Discrete drug cues in the environment, and interoceptive drug cues acquire greater incentive stimulus properties in STs than in GTs. On the other hand, contextual cues appear to motivate behavior (drug-seeking) to a greater extent in GTs than STs (see Table 1 for a summary of the results). Before we found this dissociation between different classes of drug cues we had hypothesized in a number of papers (Flagel et al., 2009; Meyer et al., 2012a; Robinson and Flagel, 2009; Saunders and Robinson, 2010, 2011b; Yager and Robinson, 2013) that STs may be more susceptible to addiction than GTs, because in the presence of drug cues they were more likely to be motivated to seek drugs and to relapse. This hypothesis is consistent with reports that STs are also relatively action impulsive (Lovic et al., 2011), show greater novelty-seeking behavior (Beckmann et al., 2011), and have relatively poor attentional (`executive') control over behavior, associated with decreased prefrontal cholinergic function (Paolone et al., 2013), all of which are considered additional “risk factors” in addiction (Belin et al., 2011; Belin and Deroche-Gamonet, 2012; Flagel et al., 2010; Jentsch and Taylor, 1999; Molander et al., 2011).

Table 1.

Individual Variation in the Motivational Properties of Different Classes of Cocaine-Associated Cues

Stimulus/measure	Result	Reference
Discrete localizable cue
Conditioned orientation	ST = GT	Yager & Robinson, 2013
Conditioned approach	ST > GT	Flagel et al., 2010; Yager & Robinson, 2013
Conditioned reinforcement
Pavlovian procedure	ST > GT	Yager & Robinson, 2013
Instrumental procedure	ST > GT	Saunders & Robinson, 2010
Conditioned cue preference	ST > GT	Meyer et al., 2012
Cue removal	ST > GT	Saunders & Robinson, 2010
Conditioned motivation
Reinstatement/adverse consequence	ST > GT	Saunders & Robinson, 2011a
Interoceptive cue
Progressive ratio responding	ST > GT	Saunders & Robinson, 2011
Drug prime reinstatement	ST > GT	Saunders & Robinson, 2011
Context cue
Context conditioned hyperactivity	GT > ST	Saunders et al., 2012
Context-induced reinstatement	GT > ST	Saunders et al., 2012

However, the study by Saunders and Robinson (2012) on the incentive motivational properties of contextual drug cues in STs and GTs suggest that the story may not be so simple. GTs showed greater context-conditioned hyperactivity and context-induced reinstatement of drug-seeking than STs. Of course, contextual cues are thought to be very effective in evoking conditioned motivational states that can lead to relapse in addicts (Bouton and Swartzentruber, 1991; Childs and de Wit, 2009; Crombag et al., 2008; Mayo et al., 2013). Thus, taken together, the results reviewed here (Table 1) suggest that different individuals may be sensitive to different `triggers' capable of motivating behavior and producing relapse. That is, STs and GTs may process motivationally salient information in quite different ways, and thus vary in their sensitivity to different classes of drug-associated stimuli. It may be, therefore, that STs are not more susceptible to addiction than GTs, but that for different individuals there are different pathways to addiction. If this is the case it has important implications for the development of individualized treatment approaches.

• Rats vary in the propensity to attribute motivational value to food and drug cues

• Thus, only in some animals will reward cues spur strong reward seeking behavior

• These findings have implications for thinking about vulnerability to impulse control disorders, such as addiction