Dissociating the Predictive and Incentive Motivational Properties of Reward-Related Cues Through the Study of Individual Differences

Abstract

Background

If presentation of a stimulus (conditional stimulus, CS) reliably predicts delivery of a reward the CS will come to evoke a conditional response (CR) through Pavlovian learning, and the CS may also acquire incentive motivational properties. Thus, CSs can have both predictive and incentive properties. We ask here whether it is possible to dissociate the predictive vs. incentive properties of a CS in rats by considering individual differences in the nature of the CR.

Methods

We used Pavlovian procedures to study the ability of a localizable CS (an illuminated lever) to acquire two properties of an incentive stimulus - the ability to attract and the ability to act as a conditional reinforcer.

Results

For some rats the CS evoked a “sign-tracking” CR, consisting of approach towards and engagement with the CS itself. For other rats the CS instead produced a “goal-tracking” CR - approach was directed away from the CS towards the site of food delivery. For sign-trackers (but not goal-trackers) the CS also acted as an effective conditional reinforcer.

Conclusions

The predictive and incentive properties of a CS can be dissociated by considering individual differences in the CR. In a given animal a cue that is predictive of reward, supporting Pavlovian learning, may or may not be attributed with incentive salience. This procedure may provide a powerful means to test hypotheses regarding the role of neural systems in learning vs. incentive motivational functions, and to study individual variation in the extent to which reward-associated stimuli act as incentive stimuli.

Introduction

When environmental stimuli are reliably associated with rewards, including drugs, they serve not only as predictors of reward (conditional stimuli, CS) that can evoke simple conditional responses (CR), but they also acquire the ability to arouse complex emotional and motivational states [1, 2]. The idea that the incentive motivational properties of reward-related cues may be especially important in drug-motivated behavior was presented in a seminal paper by Jane Stewart and her colleagues in 1984 [3]. Building on earlier work by Bindra and others [4, 5], Stewart et al. [3] argued that, “need and drive views of motivation are gradually being replaced by a view … that ascribes a primary role to incentive stimuli as the generators of motivational states and elicitors of actions” [3, p. 251]. They argued that it is, “the drug itself, or the presentation of a stimulus previously paired with the drug, [that] acts to create a motivational state that facilitates drug-seeking behavior” (p. 256). This view led to a renaissance of research on the role of incentive stimuli in the control of drug-seeking behavior and relapse, and served as a central tenet of later theories of addiction [6, for example].

For this reason we have become interested in individual differences in the extent to which cues associated with rewards are attributed with incentive salience [7, for review]. Incentive stimuli have three fundamental properties: (1) they are attractive and elicit approach towards them, as in Pavlovian conditional approach behavior; (2) they can energize ongoing instrumental actions, as in the Pavlovian-instrumental transfer effect (PIT); and (3) they can reinforce the learning of new actions, acting as conditional reinforcers [8, 9]. Although incentive stimuli may have all of these properties, studies using brain manipulations have shown these three features of incentive stimuli are in fact dissociable [9]. In rats it also has been established that there are large individual differences in the tendency to develop one of these properties of an incentive stimulus during Pavlovian training - the propensity to approach a localizable cue that has been associated with the delivery of food [10-12]. Some animals come to approach the cue more and more rapidly and vigorously engage it - this is called a “sign-tracking” CR [13]. However, under exactly the same conditions other animals may not approach the cue, but upon cue presentation approach the site of food delivery (the goal) more and more rapidly. This is called a “goal-tracking” CR [10]. Yet other animals are relatively ambivalent and vacillate between the two responses [7].

In both sign-trackers and goal-trackers the cue is clearly predictive, because it supports the learning of a Pavlovian CR in both - it is just where the CR is directed, the sign or the goal - that distinguishes the two groups. However, we have argued that only in sign-trackers does the CS acquire incentive motivational properties because only in sign-trackers does the CS itself become attractive, eliciting approach towards it [7]. If this hypothesis is correct perhaps only for sign-trackers would a CS acquire other properties of an incentive stimulus, such as the ability to act as a conditional reinforcer. The purpose of the experiments reported here was to test this hypothesis.

Results

Experiment 1: Pavlovian training including an unpaired control group

Pavlovian training

In the first experiment rats were initially trained using a Pavlovian conditioning procedure described previously [11, 14]. (See Supplementary Material for detailed methods). Briefly, an illuminated retractable lever (the CS) located 2.5 cm to the left or right of the food tray was inserted into the chamber for 8 sec. After retraction of the lever a single food pellet (the unconditional stimulus, US) was immediately delivered into the food tray. CS-US pairing occurred on a random interval 90 sec schedule. In another group of animals (Unpaired) the lever was presented, and food pellets were delivered, but these two events were not paired in time. Fig. 1 shows the results of 7 daily Pavlovian training sessions. First, the animals that underwent CS-US pairings were divided into sign-trackers vs. goal-trackers as described previously [11, 14]. The one third of the animals that made the highest number of lever presses were designated sign-trackers and the one third that made the fewest lever presses designated goal-trackers. Across days of training sign-trackers and goal-trackers developed distinct CRs (Fig. 1). Upon CS presentation sign-trackers learned to: (1) more reliably approach the lever-CS (Fig. 1A); (2) to vigorously engage the lever-CS (Fig. 1B); and (3) to do so with increasing rapidity (Fig. 1C). In contrast, upon CS presentation goal-trackers learned to: (1) more reliably approach the food tray (Fig. 1D); (2) to vigorously engage the food tray (Fig. 1E); and (3) to do so with increasing rapidity (Fig. 1F). Unpaired animals did not develop either a sign-tracking or a goal-tracking CR.

Figure 1.

Individual differences in the development of Pavlovian conditional responses (CRs) due to repeated pairing of a conditional stimulus (CS - an illuminated lever) with delivery of an unconditional stimulus (US - a food pellet). In one group of animals (UN, Unpaired, N = 20) the CS and US were presented pseudo-randomly. After training the “paired” group of animals was subdivided into two groups. The one third of animals that made the highest number of lever contacts were designated sign-trackers (ST, N = 18) and the one third that made the lowest number were designated goal-trackers (GT, N = 18). The panels on the left show three measures of behavior directed towards the lever-CS (sign-tracking behavior). Panel A shows the mean ± SEM probability of approaching the lever during the 8 sec CS period. Panel B shows the mean ± SEM number of lever contacts during the CS period, as indicated by sufficient pressure on the lever to record a “lever press”. Note however that lever contacts were typically caused not by “pressing” the lever with the paws, but by animals vigorously grasping and gnawing on the lever. Panel C shows the mean ± SEM latency to the first lever contact after CS presentation. The panels on the right show three measures of behavior directed towards the place where food will be delivered (goal-tracking behavior). Panel D shows the mean ± SEM probability of approach to the food tray during the 8 sec CS period. Panel E shows the mean ± SEM number of food tray beam breaks during the CS period. Panel F shows the mean ± SEM latency to the first beam break after CS presentation.

We also assessed the rate of learning a sign-tracking vs. a goal-tracking CR by directly comparing each of the three measures for the two groups using analyses of variance, in which day was treated as a continuous variable. This assesses whether the pattern of change in behavior over time occurs in parallel in the two groups. There were no differences between sign-trackers and goal-trackers in learning their respective CRs, as indicated by non-significant group by day interactions: Approach behavior (i.e., comparing approach to the lever for sign-trackers vs. approach to the food tray for goal-trackers (p = 0.83); Number of contacts with the lever-CS vs. the food tray (p = 0.98); Latency to approach the lever-CS vs. the food tray (p = 0.77). Thus, not only do sign-trackers and goal-trackers learn their respective CRs, they do so at a comparable rate.

Test for conditional reinforcement

One day after Pavlovian training all animals underwent a test for conditional reinforcement. For this test the food tray was removed and the retractable lever was relocated to the middle of the wall, flanked by two nose poke ports. Nose pokes into one port (designated active) resulted in insertion of the lever for two sec. Nose pokes into the other port (inactive) had no consequences. Fig. 2A shows the number of active and inactive nose pokes in the 3 groups. A 2-way ANOVA resulted in a significant main effect of group (F(2,106) = 3.28, p = 0.04), a significant effect of port (F(1,106) = 14.97, p < 0.001) and a significant group by port interaction (F(2,106) = 6.63, p = 0.002). Follow-up tests (Fisher's) indicated that sign-trackers made significantly more nose pokes into the active port than either goal-trackers (p < 0.001) or Unpaired animals (p < 0.001), and the latter two groups did not differ from one another (p = 0.54). There were no group differences in the number of nose pokes into the inactive port. In this experiment both sign-trackers and goal-trackers (but not the Unpaired group) discriminated between the active and inactive ports, making more active than inactive nose pokes. To more directly compare nose poke behavior in sign-trackers and goal-trackers we conducted a 2-way ANOVA with just these two groups, as a significant group by nose poke port interaction would indicate whether there are group differences in active pokes relative to inactive pokes. Indeed, this analysis resulted in a large interaction effect (F = 10.68, p = 0.002). In summary, these data indicate that that the lever-CS acted as an effective conditional reinforcer for sign-trackers, in that it reinforced the learning of a new instrumental response (nose-poking), but it was relatively ineffective as a conditional reinforcer for goal-trackers and Unpaired rats.

Figure 2.

Test for conditional reinforcement in the sign-trackers (ST, N = 18), goal-trackers (GT, N = 18) and Unpaired animals (UN, N = 20) described in Fig. 1. On this test a nose poke into one port (active) resulted in presentation of the lever-CS for 2 sec. Nose pokes into the other port (inactive) had no consequence. Panel A shows the mean + SEM number of active and inactive nose pokes for each of the three groups. Panel B shows the number of lever contacts during the entire 40 min test.

These data may in fact underestimate the conditional reinforcing effects of the CS in sign-trackers because the lever was so attractive to them. Fig. 2B shows the number of lever contacts during the entire 40 min test, although the lever could only be contacted during the 2 sec presentation following an active nose-poke. Sign-trackers made many more lever contacts than either goal-trackers or the Unpaired group, which did not differ from one another (one-way ANOVA, F(2,53) = 104.6, p < 0.001). Thus, sign-trackers avidly engaged the lever when a response resulted in its presentation, drawing them away from the nose poke port.

Experiment 2: Pavlovian training including an intermediate group

Pavlovian training

To confirm and extend these findings we conducted a second experiment, but without the Unpaired group. Fig. 3A shows the results of 22 sessions of Pavlovian training in animals designated sign-trackers (top one third based on number of lever presses), goal-trackers (bottom one third), and in this experiment we included the intermediate group (middle third). Individual differences in the propensity to approach the lever-CS are clearly evident. The data are not shown, but with training goal-trackers showed changes in behavior directed towards the food tray similar to that illustrated in Fig. 1 [7, as well].

Figure 3.

Panel A shows the results of Pavlovian training over 22 days (25 trials/day) plotted in 50-trial blocks, in animals designated sign-trackers (ST, N = 10), goal-trackers (GT, N = 10) or the intermediate group (IG, N = 9). Panel A shows only the data for lever contacts, but the rest of the dataset is very similar to that illustrated in Fig. 1, so these data are not shown for the sake of brevity. Panel B shows the relationship between locomotor activity in a novel environment and sign-tracking behavior. Panel C shows the results of the test for conditional reinforcement.

Test for conditional reinforcement

One day after Pavlovian training these animals were allowed to nose poke for presentation of the lever-CS, exactly as in the first experiment (Fig. 3C). Analysis of active and inactive nose pokes by 2-way ANOVA resulted in a significant main effect of group (F(2,52) = 4.56, p = 0.015), a significant effect of port (F(1,52) = 14.62, p < 0.001) and a significant group by port interaction (F(2,52) = 3.7, p = 0.031). Follow-up tests (Fisher's) indicated that sign-trackers made significantly more active nose pokes than either goal-trackers (p = 0.027) or the intermediate group (p = 0.014), and the latter two groups did not differ from one another (p = 0.83). There were no group differences in the number of inactive nose pokes. Furthermore, only sign-trackers made significantly more nose pokes into the active than inactive port (p=0.006, Fig. 3C).

In this experiment we also measured the locomotor response to a novel environment in all rats prior to Pavlovian training. Fig. 3B shows the relationship between locomotor activity in a novel environment and sign-tracking behavior, as indicated by the number of lever presses. There was no correlation between these two variables (r² = 0.039, p = 0.305). This indicates that the so-called high responder /low responder phenotype and the sign-tracker/goal-tracker phenotype are dissociable traits in these outbred animals.

Experiment 3: Instrumental training

It is possible that the difference in the ability of the lever-CS to act as a conditional reinforcer in sign-trackers vs. goal-trackers may not be related to group differences in the attribution of incentive motivational properties to the CS, but to group differences in the ability to learn an instrumental response. To test this possibility an independent group of animals first underwent Pavlovian training as described above, and were designated sign-trackers or goal-trackers (Fig. 4A), and in a third group (Unpaired) the CS and US were unpaired. After 8 Pavlovian training sessions the lever was removed and nose poke ports placed on either side of the food tray. One port was designated active and responses into that port resulted in delivery of a food pellet whereas responses into the other inactive port had no consequences. The animals were first trained using a fixed ratio (FR-1) schedule of reinforcement, and then a variable interval 5 sec (VI-5) schedule of reinforcement. Fig. 4B shows that on the FR-1 schedule all groups quickly learned an instrumental response rewarded by food delivery and nose poked preferentially into the active port on all three days of testing. There were no significant group differences in the number of active nose pokes. Fig. 4C shows that when transferred to the VI-5 schedule all groups showed high levels of responding into the active port, relative to the inactive port, increasing their rate of responding into the active port over the 3 days of testing (F(2,50) = 35.50, p < 0.001). However, there were no group differences in the number of active nose pokes (p = 0.35). These data suggest that group differences in the ability to learn an instrumental response probably do not account for group differences in the ability of the lever-CS to act as a conditional reinforcer.

Figure 4.

Instrumental learning in sign-trackers (ST), goal-trackers (GT) and unpaired rats (UN). After 8 sessions of Pavlovian training animals were divided into STs (N = 20) and GTs (N = 20) as above. The intermediate third was not tested further in this experiment. However, in a third group of animals (UN, Unpaired, N = 14) the CS and US were presented pseudo-randomly. Panel A shows the data for lever contacts during Pavlovian training. Panel B shows the effects of subsequent instrumental training on an FR-1 schedule of reinforcement for 3 consecutive days. After the third day of testing on an FR-1 schedule the animals were transferred to a VI-5 (variable interval 5 sec) schedule of reinforcement for 3 days (see Panel C).

Discussion

In two independent experiments we found that for rats that have a propensity to approach a localizable CS that has been associated with food reward the CS also serves as an effective conditional reinforcer, reinforcing learning of a new instrumental response. However, for animals that do not approach a CS associated with food delivery, but instead, go to the place where food will be delivered upon CS presentation, the CS is ineffective as a conditional reinforcer. Furthermore, this difference is not attributable to group differences in the ability to learn an instrumental response.

Although the mechanisms responsible for these individual differences are not known it is useful to think of these findings in the context of incentive motivational processes [3-5, 8, 9]. When otherwise neutral stimuli are associated with primary rewards they can be attributed with incentive motivational properties, or “incentive salience”, making them attractive, able to energize ongoing actions and to act as conditional reinforcers [8, 9, for reviews]. We have shown here that there are large individual differences in the extent to which a discrete localizable cue (a lever) that is associated with food reward acquires two of the properties of an incentive stimulus - its ability to attract and its ability to reinforce new learning. These findings support the interpretation, therefore, that for sign-trackers the lever-CS was attributed with incentive salience whereas for goal-trackers it was not [5].

It is interesting to consider the fact that the lever-CS clearly served as a predictive CS for both sign-trackers and for goal-trackers. That is, with repeated pairings the CS came to evoke a CR in both of these groups, and the two groups learned their respective CRs at a comparable rate. Furthermore, pairing the lever-CS with food delivery was necessary for learning either a sign-tracking or a goal-tracking CR, as the Unpaired group acquired neither. Indeed, both sign-trackers and goal-trackers learned a Pavlovian conditional approach response - they differed only in where the response was directed. In sign-trackers the CR was directed towards the CS itself, whereas in goal-trackers the CR was directed towards the food tray or goal (away from the CS). Combined with the results of the test for conditional reinforcement, these data suggest that it is possible to dissociate the predictive properties of a CS from its incentive properties. Furthermore, the data suggest that the predictive value of a reward-related cue is not sufficient to confer incentive value.

The ability to parse the predictive vs. incentive properties of reward-related cues may provide a valuable tool to test different notions about the role of specific neural systems in learning vs. incentive motivation. For example, it has been suggested that phasic dopamine activity serves as a reward prediction error signal in Pavlovian learning, in part because with experience dopaminergic activity shifts from presentation of the US to the CS [15]. However, others have suggested that dopamine is more important in mediating incentive processes than associative learning per se [16, 17]. A reward learning interpretation must predict similar changes in the dopaminergic response to the CS in sign-trackers and goal-trackers, because the CS is equally predictive in both groups. However, by an incentive motivational interpretation the CS should come to elicit a dopaminergic response in sign-trackers, but not necessarily in goal-trackers, because only in sign-trackers is the CS attributed with incentive salience. Preliminary data are consistent with the latter interpretation [18].

Individual differences in the propensity to attribute incentive salience to environmental cues associated with rewards may also be important in studying individual differences in the propensity for addiction. Addicts have great difficulty in resisting people, places and other signs associated with drugs - their thoughts and actions are strongly drawn towards such stimuli -allowing them (the stimuli) to act, “as a persistent goad to response generation”, precipitating relapse and serving to motivate continued drug-seeking behavior [3, p. 263]. Consistent with this, we have found that animals that sign-track to a CS associated with a food reward also readily sign-track to a CS associated with the intravenous delivery of cocaine - whereas goal-trackers do not [19]. Furthermore, the neural system(s) that mediates these incentive motivational processes is known to sensitize with repeated drug treatment [6], and we have found that sign-trackers are more susceptible to psychomotor sensitization than goal-trackers [14]. Thus, in future studies it will be interesting to determine whether individual differences in the propensity to attribute incentive salience to reward-related cues, and for this process to sensitize, is also related to the extent to which drug-related cues come to maladaptively attract some individuals towards them, and whether such individual differences confer vulnerability or resilience to addiction.