Sign-tracking is an expectancy-mediated behavior that relies on prediction error mechanisms

Abstract

When discrete localizable stimuli are used during appetitive Pavlovian conditioning, “sign-tracking” and “goal-tracking” responses emerge. Sign-tracking is observed when conditioned responding is directed toward the CS, whereas goal-tracking manifests as responding directed to the site of expected reward delivery. These behaviors seem to rely on distinct, though overlapping neural circuitries, and, possibly, distinct psychological processes as well, and are thought to be related to addiction vulnerability. One currently popular view is that sign-tracking reflects an incentive motivational process, whereas goal-tracking reflects the influence of more top-down cognitive processes. To test these ideas, we used illness-induced outcome-devaluation and Kamin blocking procedures to determine whether these behaviors rely on similar or distinct underlying associative mechanisms. In Experiments 1 and 2 we showed that outcome-devaluation reduced sign-tracking responses, demonstrating that sign-tracking is controlled by reward expectancies. We also observed that post-CS goal-tracking in these animals is also devaluation sensitive. To test whether these two types of behaviors rely on similar or different prediction error mechanisms, we next tested whether Kamin blocking effects could be observed across these two classes of behaviors. In Experiment 3 we asked if sign-tracking to a lever CS could block the development of goal-tracking to a tone CS; whereas in Experiment 4, we examined whether goal-tracking to a tone CS could block sign-tracking to a lever CS. In both experiments blocking effects were observed suggesting that both sign- and goal-tracking emerge via a common prediction error mechanism. Collectively, the studies reported here suggest that the psychological mechanisms mediating sign- and goal-tracking are more similar than is commonly acknowledged.

In Pavlovian learning, repeated pairings of a relatively neutral conditioned stimulus (CS) with a biologically significant unconditioned stimulus (US) result in the manifestation of conditioned responses (CR) to the CS over the course of training. In appetitive conditioning procedures, when temporally discrete and spatially localizable CSs (such as the insertion of a lever into the chamber) are paired with food reward, two types of CRs emerge: “sign-tracking” and “goal-tracking” (e.g., Boakes 1977). Sign-tracking CRs are observed when responding is directed toward the CS itself. In perhaps the first observation of this, Pavlov noted that one of his nonharnessed dogs approached a light bulb CS when it was turned on and began licking it in anticipation of a meat powder US (Pavlov 1927). More generally, other investigators have found that pigeons will approach and attempt to “eat” or “drink” keylight CSs that have been paired, respectively, with grain or water USs (Jenkins and Moore 1973), and rats will lick, bite, and handle a response lever CS or moving ball bearing CS that have been paired with a food pellet US (e.g., Davey and Cleland 1982; Timberlake et al. 1982; Flagel et al. 2009). In contrast, responses elicited by the CS that are directed toward the site of reward delivery are classified as goal-tracking or food magazine CRs (Farwell and Ayres 1979; Holland 1979; Delamater 1995; Delamater et al. 2017).

In outbred populations of rats, three mutually exclusive subpopulations of rats emerge in appetitive learning experiments involving response lever CSs and food USs—those that exclusively sign-track, those that exclusively goal-track, and those that display a mixed profile of sign- and goal-tracking phenotypes (Flagel et al. 2007; Meyer et al. 2012; though see Patitucci et al. 2016). Recent studies have linked the propensity to sign-track to the likelihood of developing addiction-like patterns of drug intake (Tomie et al. 2008; Flagel et al. 2009, 2010, 2008; Robinson and Flagel 2009), thereby establishing the sign- and goal-tracking paradigm with rodents as a potentially viable animal model of substance abuse (though see also Saunders et al. 2014). The theoretical concept linking sign-tracking to addiction-vulnerability rests upon the idea that sign-tracking animals rapidly attribute excessive motivational significance (i.e., “incentive salience”) to reward predictive stimuli, whereas goal-tracking animals do not (Flagel et al. 2009). In doing so, sign-trackers render themselves more vulnerable to addiction because their decisions are more strongly controlled by the incentive properties of drug-related CSs rather than any rational forward-looking decision making process (Flagel et al. 2009; see also Robbins and Everitt 2002; Robinson and Berridge 2003).

Numerous studies have demonstrated that sign-tracking and goal-tracking CRs have dissociable neural substrates. Specifically, it has been suggested that sign-tracking may rely more on subcortical circuitry than goal-tracking and therefore sign-tracking behaviors may be independent or at least less reliant on more cortically dependent “outcome-expectancy” mediated processes (Meyer et al. 2012; Flagel and Robinson 2017). The manifestation and maintenance of sign- but not goal-tracking depends upon dopamine (DA) receptor activation. Systemic administration of flupenthixole, a DA receptor (D1/D2) antagonist, blocks the acquisition of sign-, but not goal-tracking, and intracranial infusion of this drug into the nucleus accumbens core blocks the maintenance of sign-, but not goal-tracking (Flagel et al. 2011b; Saunders and Robinson 2012). In addition, exposure to the CS increases cFOS levels in striatum, thalamus, and the lateral habenula to a greater magnitude in sign-tracking versus in goal-tracking rats (Flagel et al. 2011a). Interestingly in this study sign-trackers were also found to have elevated cFOS in the orbitofrontal cortex, a region implicated in the expression of expectancy-mediated behaviors such as US-devaluation effects and outcome-specific Pavlovian-to-instrumental transfer (Delamater and Oakeshott 2007). This latter finding seems to stand in opposition to the interpretation that these cFOS patterns suggest sign-tracking is less dependent on expectancy-mediated processes.

Considering the difference in circuitry mediating these behavioral phenotypes, several papers have proposed that goal-trackers may rely on more “cognitive” learning mechanisms than sign-trackers (Flagel et al. 2009; Flagel and Robinson 2017; Meyer et al. 2012; Lesaint et al. 2014). Thus, an outstanding question that requires clarification is whether the contents of associative learning in sign- and goal-tracking are similar or different. An equally important question is whether these two behavioral phenotypes obey the same general principles of associative learning, or whether they rely on distinct underlying learning mechanisms (as may be suggested by their differing neural substrates). Early sign-tracking (i.e., autoshaping) studies with pigeons used Kamin blocking procedure to show that highly diffuse stimuli could interfere with the acquisition of sign-tracking responses to a discrete visual cue (Blanchard and Honig 1976; Tomie 1976; Leyland and Mackintosh 1978; Khallad and Moore 1996). This result not only suggests that a diffuse stimulus to which the animal cannot sign-track has acquired some associative value, but that the diffuse stimulus can interfere with the development of a sign-tracking phenotype to a localized discrete cue. In other words, acquisition of sign-tracking seems to depend upon the same US prediction error mechanism that others have suggested underlies Pavlovian learning more generally (Rescorla and Wagner 1972). The fact that the food US has been fully predicted by the diffuse stimulus, in this example, could render it less capable of supporting the acquisition of sign-tracking that would ordinarily develop to the discrete localized CS. In a recent study conducted with rats, however, the story seems more complex. Holland et al. (2014) studied the ability of cues that support sign- or goal-tracking CRs to block one another in a Kamin blocking design. They observed that whereas the sign-tracking lever CS could block goal-tracking CRs to a diffuse auditory CS, they failed to observe the reverse to be true. This result is important because it suggests that the learning mechanisms underlying sign- and goal-tracking may, in fact, differ in rats.

The other issue, having to do with potential associative content differences between sign- and goal-tracking, similarly has found mixed results in the literature. Davey and Cleland (1982) addressed this question in rodents using an outcome-devaluation procedure and found that sign-tracking to a lever CS was reduced in a nonreinforced test session conducted after the food US had been devalued. This result suggests that sign-tracking is not solely driven by a general incentive motivational process that would attribute high salience to a discrete CS, but that they can also be guided by an expectation of a devalued US (see also Holland and Straub 1979; Stanhope 1989; Blundell et al. 2003). However, recent attempts to replicate this effect have produced contradictory findings (e.g., Morrison et al. 2015; Patitucci et al. 2016). It may be argued that in both of these studies the outcome-devaluation treatments used were relatively weak. Morrison et al. (2015) devalued the sucrose US by a single pairing with LiCl, whereas Patitucci et al. (2016) used a 15 min sucrose satiation procedure. On the other hand, Robinson and Berridge (2013) demonstrated a robust revaluation effect in sign-tracking rats when a hypertonic salt associated lever CS was tested under extinction conditions while the rats were in a sodium-depleted state. These results suggest that devaluation effects in sign-tracking may depend upon the devaluation treatment itself being strong.

The various inconsistencies between earlier and more recent replication attempts led us to reexamine both outcome-devaluation and Kamin blocking effects using similar procedures across studies. In Experiments 1 and 2, we explored the effects of outcome-devaluation upon sign-tracking CRs using both within and between subject experimental designs. Importantly, we gave a more extensive amount of US devaluation training prior to the devaluation tests than has been done in recent studies. In Experiments 3 and 4 we examined whether CSs that support sign- or goal-tracking can mutually compete with each other in a Kamin blocking task (e.g., Kamin 1968) similar to Holland et al. (2014), but under conditions that might be expected to strengthen the blocking effect (Wagner 1969). If sign-tracking rests on an underlying learning system, distinct from goal-tracking, that is primarily subcortical and not expectancy-mediated, then we would expect to see poor US devaluation effects and, possibly, noncompetitive effects in a blocking task when the pretrained and to be blocked stimuli support different response types.

Results

Experiment 1: within-group US devaluation effect

In Experiment 1, rats were trained with two distinct lever-US associations. Data from both levers was collapsed for analysis of responding during acquisition. Over the course of the 12 training sessions the mean rate of lever responses increased steadily from training block 1 to 6 (Fig. 1A, two-tailed paired t-test, t₍₁₅₎ = 2.98, P = 0.009). Conversely the change in magazine entry rates during CS versus pre-CS responding decreased across training in 14 of 16 rats. Figure 1B depicts the most typical patterns of responding during the final block of training; the left panel shows a sign-tracking rat and the right panel shows a rat who displayed a mixed sign- and goal-tracking profile. Here the sign-tracking rat steadily increased its lever contacts across the 8-sec lever presentation before decreasing somewhat toward the end of the CS interval. Simultaneously, magazine entries were suppressed below pre-CS response levels throughout CS interval. Of the 16 rats in this study, 14 rats displayed this pattern of responding. The remaining two rats were classified as mixed sign- and goal-tracking animals. Both of these animals resembled the pattern shown in Figure 1B (right). Here, lever contacts occurred early in the CS interval, but then dropped nearly to 0 as magazine entries increased in the second half of the CS interval.

Figure 1.

Experiment 1, Pavlovian conditioning and US devaluation training. (A) Mean magazine (goal-tracking; solid symbols) and lever contact (sign-tracking; blank symbols) responses shown in two-session blocks during acquisition of Pavlovian conditioning. Data is collapsed across both CSs. Sign-tracking increases, whereas goal-tracking decrease across conditioning. (B) Response patterns of exemplar sign-tracking (left) and intermediate (right) rats shown in 1-sec intervals during the lever CS presentations during the final block of training. Data is collapsed across both CSs. Sign-tracker (#R4) preferentially engages the lever across the CS–US interval while magazine responding is reduced below pre-CS rates. Intermediate (#B2) engages the lever for the first half of the CS–US interval and then switches to magazine responding during the latter half of the CS–US interval. (C) Magazine responses during presentation of devalued (solid symbols) and nondevalued (empty symbols) USs during devaluation training. Data shown in 5-min bins across five cycles of US devaluation training.

US devaluation was performed across five cycles, where one US was paired with LiCl injections to induce an aversion and the other US was presented equally often but not paired with LiCl. Across the 5 US devaluation cycles, the rats decreased magazine responding during sessions in which the devalued outcome was presented, while they maintained high levels of responding during sessions in which the nondevalued outcome was presented. On the final cycle, magazine responses were significantly higher in the session in which the nondevalued versus the devalued outcome was presented, (Fig. 1C, two-tailed paired t-test, t₍₁₅₎ = 5.72, P < 0.01). Pellet intakes were not systematically recorded during this phase, but all of the nondevalued pellets were consumed while few if any of the devalued pellets were consumed by the end of this phase.

Following devaluation training, rats underwent testing where the levers from training were presented under extinction conditions (three tests). The test data were collapsed across the three test sessions (as similar patterns were seen in each). Lever contacts directed at the lever whose outcome had been devalued were systematically reduced across the 8-sec CS interval compared to lever contacts made to the lever whose outcome had not been devalued (Fig. 2A). A two-tailed paired t-test was performed on the data averaged across the within-CS intervals and comparing the Nondevalued (mean (±SEM) = 0.23 (±0.04) responses per second (rps)) and devalued levers (0.14 (±0.03) rps), t₍₁₅₎ = 2.72, P = 0.016. Complimentary to these data, magazine entries were reduced during presentations of the lever whose associated US was not devalued versus presentations of the lever whose outcome was devalued (Fig. 2B). This was analyzed by averaging across the 8-sec CS and comparing the CS − pre-CS difference scores for each stimulus (means (±SEM) for the nondevalued and devalued levers, respectively, were −0.11 (±0.03) rps and −0.03 (±0.03) rps). This difference was significant by a two-tailed paired t-test, t₍₁₅₎ = 4.74, P = 0.0003. This suppression of goal-tracking in response to the nondevalued lever likely reflects the strong response competition from the tendency to contact the lever in the nondevalued condition thereby bringing goal-tracking responses below baseline levels for the nondevalued lever CS. The tendency to approach the lever was reduced when the associated outcome had been devalued, and so the devalued lever provoked less response competition of magazine entry as a result. A further analysis was conducted on the magazine entry data at the time of expected reward deliveries (i.e., after lever withdrawal in post-CS periods). Figure 2B shows that there was a sharp increase in magazine responding in the first 2 post-CS intervals. However, this increase was significantly reduced following withdrawal of the lever whose outcome had been devalued (mean (±SEM) in first 2 post-CS intervals combined = 0.31 (±0.03) rps) compared to withdrawal of the lever whose outcome had not been devalued (mean (±SEM) = 0.42 (±0.04) rps; two-tailed paired t-test, t₍₁₅₎ = 2.40, P = 0.03).

Figure 2.

Experiment 1, devaluation testing. (A) Mean lever contacts (sign-tracking) shown in 1-sec intervals across CS presentation interval during devaluation testing. Contacts with the lever whose US was devalued (solid symbols) is lower than the lever whose US was left nondevalued (empty symbols). (B) Mean magazine responses (goal-tracking) shown in 1-sec intervals across CS presentation interval and during the post-CS period. During CS presentation magazine responding is greater to presentation of the devalued versus nondevalued CS, but this pattern flips in the post-CS period, such that at the time of expected US delivery magazine responding to the nondevalued CS is greater than the devalued CS. (C) The relationship between the US devaluation score and the Pavlovian conditioned approach score (PCA) for individual animals. The propensity to sign-tracking is positively correlated with the magnitude of devaluation score.

One final analysis was performed on these test data to evaluate the relationship between sign- and goal-tracking and the expression of devaluation effects. Here we tested the correlation between the strength of the US devaluation effect (# nondevalued lever contacts – # devalued lever contacts) with the tendency of the animals to sign- or goal-track (PCA index). To determine the tendency of the rats to sign- or goal-track we adopted the same “PCA” measure reported by Meyer et al. (2012). This measure compares the frequency, probability, and latency of lever versus magazine responses by averaging the following variables: response bias ([lever Rs − magazine Rs]/[lever Rs + magazine Rs]), probability difference ([p(lever R)-p(magazine R)), and difference in response latency ([Mag R latency − Lever R latency]/8). The index is calculated by averaging these variables. With this index, the animal is classified as a sign-tracker by a score between 0.5–1.0, and as a goal-tracker with a score between −1.0 and −0.5. This analysis confirmed that 14 of the 16 rats were sign-trackers and two displayed mixed profiles (Fig. 2C). Importantly, the correlation between these two indices was significantly positive (Fig. 2C, Pearson's correlation test r = 0.51, P = 0.043). This result indicates that the increasing tendency to sign-track predicted an increasing sensitivity to US devaluation.

Experiment 2: between-group US devaluation effect

In Experiment 2, rats were trained on a Pavlovian discrimination task, where one lever (Lev1+) was paired with pellet delivery and another (Lev2−) was presented an equal number of times but never reinforced. Discriminative responding rapidly emerged over the course of training; contacts to Lev1+ increased while contacts to Lev2− remained low across training (Fig. 3A). Averaging over the last 4 sessions of training this discrimination was significant (two-tailed paired t-test: t₍₁₁₎ = 4.56, P = 0.0008). Across training, goal-tracking magazine responses to Lev1+ were, once again, suppressed relative to baseline and relative to Lev2− rates of responding (Fig. 3B). This was true for 10 of the 12 rats classified as “sign trackers.” The remaining two rats displayed an intermediate phenotype (as in Experiment 1) and showed increased magazine responses to Lev1+ over baseline and over Lev2−. Representative rats are shown in Figure 3C. The sign-trackers restricted lever contacts to Lev1+, but did not engage significantly with Lev2−. Following training, half the rats underwent US devaluation training. Magazine responses during this US devaluation phase proceeded as in Experiment 1 (data not shown).

Figure 3.

Experiment 2, Pavlovian conditioning. (A) Discriminative sign-tracking emerges across training, such that by the end of training lever contacts are greater to the reinforced lever (empty symbols) than to the nonreinforced lever (solid symbols). (B) Goal-tracking magazine responses are slightly greater during presentation of the nonreinforced (solid symbols) versus the reinforced (empty symbols) lever. (C) Response patterns of exemplar sign-tracking (top) and intermediate (bottom) rats shown in 1-sec intervals during the lever CS presentations during the final block of training. Sign-tracker (#R10) preferentially engages the reinforced lever (empty squares) across the CS–US interval while magazine responding (empty circles) is reduced below pre-CS rates. Magazine and lever contacts are low and unchanging during presentation of the nonreinforced lever (solid symbols). Intermediate (#R12) engages the reinforced lever (empty squares) for the first half of the CS–US interval and then switches to magazine responding (empty circles) during the latter half of the CS–US interval. Magazine and lever contacts are low and unchanging during presentation of the nonreinforced lever (solid symbols).

After devaluation training, rats underwent testing under extinction conditions to determine the effect of outcome devaluation on conditioned responding. As in Experiment 1, the data from three test sessions were collapsed as there were no differences across these tests. As expected, there was little evidence of sign-tracking to the CS− lever in either devalued or the nondevalued groups whereas sign-tracking to the CS+ lever was substantially higher in Group No Devaluation compared to Group Devaluation (Fig. 4A). These data were analyzed by conducting separate between-group ANOVAs for each lever using a pooled error (MSE = 0.016) and Satterthwaite's correction for denominator degrees of freedom (Satterthwaite 1946). The groups did not differ in their responding to the nonreinforced lever (Lev2−), F_(1,20) = 0.19, P > 0.05, but they did significantly differ to the reinforced lever (Lev1+), F_(1,20) = 20.20, P < 0.05. In addition, goal-tracking (i.e., magazine) responses during the devaluation tests provided additional evidence for a reliable US devaluation effect. Magazine responding here is represented as a change from pre-CS rates of responding. Consistent with the training data, magazine responding during CS presentations was low for both Lev1+ and Lev2− and did not differ between groups. However, goal-tracking in the post-CS period following Lev1+ (at the time of expected reward delivery) was greater in Group No Devaluation versus Group Devaluation (Fig. 4B). Between-group ANOVAs were performed for each trial type and interval (i.e., Lev1+, Lev2−, Post Lev1+ and Post Lev2−) using a pooled error term (MSE = 0.005) and Satterthwaite's correction. The only reliable difference occurred with Post Lev1+ where magazine responding was higher in Group No Devaluation, F_(1,34) = 25.09, P < 0.05 (other Fs < 0.47, Ps > 0.05).

Figure 4.

Experiment 2, devaluation testing. (A) Mean sign-tracking (lever contacts) is higher in response to the previously reinforced lever (solid) versus the nonreinforced lever (empty) presentations in the nondevalued group (left), but not in the devalued group (right). Moreover, sign-tracking is greater to presentation of the previously reinforced lever in the nondevalued versus the devalued group. (B) Mean goal-tracking (magazine) responses do not increase during presentation of the previously reinforced versus nonreinforced lever and this does not differ between nondevalued versus devalued groups. However, goal-tracking in the post-CS period is significantly higher in the nondevalued versus devalued group. Post-CS goal tracking is absent following presentation of the nonreinforced lever in both groups.

Experiment 3: blocking of goal-tracking

In Experiment 3, we evaluated whether acquisition of a sign-tracking lever response would interfere with subsequent acquisition of a goal-tracking response to an auditory CS presented in combination with the initially conditioned lever CS. In the first phase of training rats were trained to discriminate between Lev1+ that was paired with pellet delivery and a Lev2− that was never paired with pellets. Acquisition of discriminative lever contact (sign-track) responding was rapid, as in Experiment 2. By the end of preliminary acquisition these discriminative responses were stable and averaged over the last four sessions of the preliminary training phase rats contacted Lev1+ significantly more than Lev2− (Fig. 5A sessions 13–16), as revealed by a significant two-tailed paired t-test, t₍₁₅₎ = 6.28, P < 0.00002. Concomitantly, magazine responding during Lev1+ presentations was decreased across the preliminary training phase relative to Lev2− and the pre-CS period (Fig. 5B). Averaged over the final four sessions, magazine entries during Lev2− presentations were significantly greater than during Lev1+ presentations (Fig. 5A sessions 13–16: two-tailed paired t-test: t₍₁₅₎ = 4.83, P = 0.0002). This suppression in goal-tracking between the Lev1+ versus Lev2− seems to reflect highly focused sign-tracking to Lev1+ which thereby reduced goal-tracking magazine responses below baseline rates.

Figure 5.

Experiment 3, Pavlovian conditioning and blocking. (A) Sign-tracking to presentations of the reinforced lever (Lev1+; empty squares) develops during initial conditioning and remains high during the blocking phase. Sign-tracking never develops to the presentations of the nonreinforced lever (Lev2−; solid squares). During the blocking phase compound presentation of the reinforced lever Lev1 with auditory S1 (Lev1 and S1+; empty circles) sustains sign-tracking at rates similar to presentations of Lev1+ alone. Compound presentations of the nonreinforced lever Lev2− with auditory S2 (Lev2 and S2+; solid circles) does not elicit sign-tracking and rates are similar to presentations of Lev2− alone. (B) Goal-tracking in response to presentations of the reinforced lever (Lev1+; empty squares) is reduced below pre-CS levels during initial acquisition and during blocking phase. No change from pre-CS rates in goal-tracking is observed in response to presentations of the nonreinforced lever Lev2 during initial acquisition or during blocking phase (Lev2−; solid squares). Compound presentations of the reinforced lever Lev1 with S1 (Lev1 and S1+; empty circles) during the blocking phase results in a reduction in goal-tracking below pre-CS rates which is similar to presentations of Lev1+ alone. Whereas compound presentation of the nonreinforced lever Lev2 with S2 (Lev2 and S2+; solid circles) elicits robust goal tracking which emerges across the blocking phase.

In this experiment, all but one of the rats displayed more lever contacts to Lev1+ than to Lev2− and none of the animals showed more goal-tracking responses to Lev1+ than to Lev2−, once again showing that the far majority of animals were sign-trackers.

After 16 sessions of initial conditioning, rats entered the blocking phase of training in which new auditory stimuli were paired with the lever CSs from initial training. Compound presentations of the levers and their paired auditory CSs were rewarded. Overall, sign-tracking was higher to the CS+, Lev1+, than to the CS−, Lev2− and this discriminative responding was also maintained when the lever CSs were presented in compound with the auditory CSs (Fig. 5A Blocking Phase). A one-way ANOVA was performed across the four trial types averaged across the seven training sessions in this phase (means (±SEM) for Lev1+, Lev2−, Lev1 and S1+, Lev2 and S2+, respectively, were 0.30 (±0.04), 0.05 (±0.01), 0.39 (±0.06), and 0.01 (±0.004) rps). This analysis revealed significant differences among the four trial types, F_(3,45) = 33.91, P < 0.05, MSE = 160.445. Post-hoc tests (Rodger 1975) revealed somewhat greater sign-tracking responding to Lev1+ when compounded with S1 than when presented alone, and that responding on both of these trial types was greater than responding to Lev2− alone. Further, responding to Lev2− when compounded with auditory S2 was slightly less than to the Lev2− alone (Ps < 0.05). Similar analyses were performed on the goal-tracking magazine responding for these four trial types. Overall, goal-tracking CRs developed on compound trials consisting of Lev2 and auditory S2. In addition, goal-tracking CRs were suppressed below pre-CS levels on both trial types involving presentations of Lev1+ (see Fig. 5B Blocking Phase). A one-way ANOVA was performed on these data (as with the lever contact data) and this revealed significant differences across the four trial types, F_(3,45) = 66.56, P < 0.05, MSE = 48.946. Post-hoc tests revealed that responding was greatest on Lev2 and S2 compound trials (mean (±SEM) = 0.18 (±0.03) rps) and lowest on the two trial types in which the Lev1 was presented (means for Lev1+ and Lev1 and S1+, respectively, were −0.11 (±0.02) and −0.13 (±0.02) rps). Responding to Lev2 alone (mean (±SEM) = 0.04 (±0.02) was intermediate between these extremes (Ps < 0.05).

After seven sessions in the blocking phase, rats were tested for conditioned responding under extinction conditions in which the auditory CSs, S1 (blocked) and S2 (control) were presented alone. Goal-tracking responses are shown for the CS and post-CS periods as a change from pre-CS rates, as in the first two experiments (Fig. 6). It is clear that responding to the blocked auditory stimulus, S1, is lower than to the control stimulus, S2 both during the CS as well as in the post-CS period (Fig. 6). These data were analyzed by performing a one-way ANOVA across the two stimuli and periods (during and post-CS). The analysis revealed reliable differences, F_(3,45) = 14.34, P < 0.05, MSE = 112.287. Post-hoc tests revealed that responding was greater during the control CS, S2, than to the blocked CS, S1, in both the CS and post-CS periods, and that responding overall was greater during the CS than in the post-CS period (Ps < 0.05). These data show that prior conditioning of sign-tracking to a lever CS interferes with subsequent learning of goal-tracking CRs to an auditory CS conditioned in compound with that lever CS.

Figure 6.

Experiment 3, test for Kamin blocking effect. Mean goal-tracking (magazine) response to presentation of the blocked CS (S1; empty bars) is significantly lower than responding to the control CS (S2; solid bars) during both the CS (left) and post-CS (right) period. Data collapsed across three tests.

Experiment 4: Blocking of sign-tracking

In Experiment 4, we tested whether acquisition of a goal-tracking response to an auditory CS would interfere with subsequent acquisition of a sign-tracking response to a lever CS presented in combination with the initially conditioned auditory CS. During the initial acquisition phase, rats discriminated between the reinforced, S1+, and nonreinforced, S2−, auditory stimuli. Over the last four training sessions magazine responding to S1+ was significantly higher than to S2− (Fig. 7A Acquisition Phase, two-tailed paired t-test: t₍₁₅₎ = 5.25, P < 0.0001).

Figure 7.

Experiment 4, Pavlovian conditioning and blocking. (A) Goal-tracking (magazine) in response to presentations of the reinforced auditory CS (S1+; empty squares) increases across initial conditioning and remains high during the blocking phase. Goal-tracking does not develop to presentations of the nonreinforced auditory CS during initial conditioning or the blocking phase (S2−; solid squares). Compound presentations of the reinforced auditory CS, S1 with Lev 1 (S1 and Lev1+; empty circles) during the blocking phase does not elicit significant goal tracking. Similarly compound presentations of the nonreinforced auditory CS, S2 with Lev 2 (S2 and Lev2+; solid circles) does not elicit significant goal-tracking. (B) During the blocking phase sign-tracking to Lev2 which was presented in compound with the nonreinforced auditory CS, S2 (S2 and Lev2+; solid circles) is greater than sign-tracking to Lev1, which was presented in compound with the reinforced CS, S1 (S1 and Lev1+; empty circles).

In the blocking phase, rats were presented with the original stimuli from training and with compound presentations of the auditory stimuli paired with new lever stimuli. During this blocking phase, rats continued to display discriminative goal-tracking, preferentially entering the magazine more on S1+ than S2− trials. However, the presence of the lever eliminated goal-tracking responses during compound trials where S1 was combined with Lev1+ (Fig. 7A Blocking Phase). A one-way ANOVA was performed on the data from the four trial types collapsed across sessions during the blocking phase, and it revealed significant differences between the four trial types, F_(3,45) = 29.64, P < 0.05, MSE = 0.058. Post-hoc tests (Rodger 1975) revealed greater goal-tracking during S1+ trials (mean (±SEM) = 0.65 (±0.11) rps) than on any other trial type with little difference among them means (SEM) for S2−, S1 and Lev1+, S2 and Lev2+, respectively, = 0.03 (±0.01), 0.02 (±0.04), −0.04 (±0.02) rps (Ps < 0.05). Interestingly, acquisition of sign-tracking lever contacts during compound trials in which the lever CSs were combined with the auditory CSs was also observed during the compound training phase (Fig. 7B). Importantly, fewer sign-tracking lever responses were observed during compound trials of S1 (the previously established CS+) and Lev1 than during compound trials of S2 (the previously established CS−) and Lev2. These data were analyzed by averaging over sessions (means (±SEM), respectively, were 0.47 (±0.06) and 0.59 (±0.07)) and conducting a two-tailed paired t-test between the two trial types, t₍₁₅₎ = 2.74, P = 0.015.

After the blocking phase rats were tested under extinction conditions for conditioned responding to the blocked (Lev1) and control (Lev2) levers. During testing presentation of the levers did not elicit significant goal-tracking magazine responses to either lever above the pre-CS baseline, however following withdrawal of both levers magazine responses rapidly increased in the post-CS period (Fig. 8A). A t-test compared magazine responding between the two levers averaged across the 8-sec period of CS presentation and found no significant difference (mean CS-pre score (±SEM) for Lev1 and Lev2, respectively, was 0.00 (±0.02) rps and −0.03 (±0.02) rps). In contrast, sign-tracking lever contacts were significantly greater on presentation of the control lever, Lev2, compared to presentations of the blocked lever, Lev1, across their 8-sec duration (Fig. 8B). A two-tailed paired t-test examined contact responses to Lev1 and Lev2 averaged across the entire 8-sec CS (means (±SEM) for Lev1 and Lev2, respectively, were 0.27 (±0.05) and 0.41 (±0.06)), and revealed higher response levels to Lev2, t₍₁₅₎ = 2.30, P = 0.037. This indicates that prior conditioning of goal-tracking CRs to an auditory CS was capable of interfering with acquisition of sign-tracking CRs to a lever CS conditioned in its presence.

Figure 8.

Experiment 4, test for Kamin blocking effect. (A) Goal-tracking (magazine) during and after presentations of the blocked (Lev1; empty symbols) versus the control (Lev2; solid symbols) lever CSs are similar. (B) Sign-tracking (lever contacts) to the control lever CS (Lev2; solid symbols) is greater than sign-tracking to the blocked lever CS (Lev1; empty symbols).

Discussion

Pavlovian learning promotes the development of conditioned responding. This responding can be directed toward the site of reward delivery (goal-tracking) or toward the conditioned stimulus itself (sign-tracking). Research exploring the neural circuitry of these distinct forms of conditioned responding suggests that both the acquisition and expression of these responses depend upon different neural networks. This raises the question of whether these distinct conditioned behavioral phenotypes are governed by different psychological mechanisms. In the current studies, we took a behavioral approach to exploring this possibility.

First, it has been proposed that sign-tracking arises independent of cortical control, whereas goal-tracking relies more heavily on cortical pathways. Given that expectancy-mediated behaviors have been suggested to depend upon cortical nuclei (Flagel et al. 2009, 2011a; Meyer et al. 2012) and that the US devaluation effect has been shown to depend upon OFC and BLA-OFC interactions (e.g., Delamater and Oakeshott 2007; Lichtenberg et al. 2017), the assertion that sign-tracking manifests independent of cortical control sets up the testable hypothesis that sign-tracking should be insensitive to manipulations that alter the value of an expected outcome. We addressed this directly, using outcome-devaluation procedures using both within-subject (Experiment 1) and between-subject designs (Experiment 2). In both experiments, our results clearly demonstrated that sign-tracking was reduced following outcome-devaluation (Figs. 2A, 4A), arguing against the claim that the sign-tracking conditioned response arises independently of expectancy-driven cortical input. These data are consistent with the results of Davey and Cleland (1982), who were the first to demonstrate that sign-tracking was indeed sensitive to outcome-devaluation. Our data are also in line with a recent study by Robinson and Berridge (2013) demonstrating spontaneous expression of sign-tracking to a lever CS following an outcome “revaluation” procedure. In this experiment female rats identified as sign-trackers were initially trained to associate one lever CS with an intraoral infusion of an unpalatable high concentration of salt water and another lever CS with infusion of palatable sucrose. During conditioning, rats developed two distinct conditioned responses; rats approached and engaged the sucrose-paired lever, but they actively avoided contact with the salt-paired lever. Following conditioning rats were subjected to a salt-depletion procedure and then tested under extinction conditions for conditioned responding to the lever CSs. Testing revealed robust spontaneous sign-tracking to the salt-paired lever following this motivational shift. Even though rats had previously avoided contact with the salt-paired lever during training, following a shift in the value of the hypertonic salt outcome they vigorously engaged the lever in spite of the fact that salt was not presented during this test. In our studies, the value of the outcome was selectively reduced following training, whereas, in this study the value was selectively increased. In both cases, the data demonstrate that sign-tracking is indeed an expectancy-mediated behavior.

However, as noted in the introduction, two recent studies found no effect of outcome-devaluation (Morrison et al. 2015; Patitucci et al. 2016) on sign-tracking. So how do we reconcile our findings with these recent studies?

Morrison et al. (2015) trained rats to associate a lever CS with delivery of liquid sucrose. Following training, half the rats were given a single pairing of home cage intake of sucrose with LiCl. They were subsequently tested under extinction conditions for conditioned responding to the CS, followed by a home-cage sucrose consumption test. They found that a subset of their rats displayed a primarily goal-tracking phenotype (i.e., rats who displayed a “PCA index” below −0.47). Such rats displayed a reduction in goal-tracking responses relative to a nondevalued control group, together with a concomitant increase in sign-tracking behaviors. Other rats were identified as “sign-trackers” during training (i.e., with a PCA index greater than −0.47), though these animals would ordinarily be classified as “intermediates” using a strict interpretation of the PCA index (Meyer et al. 2012). These rats showed no change, relative to nondevalued controls, in either sign- or goal-tracking following outcome-devaluation.

It is not clear why our results differed from Morrison et al. (2015), but one important procedural difference was the amount of devaluation training given in our two studies. The conditioned taste aversion protocol used in the Morrison et al. (2015) study produced a relatively weak aversion. The results of their post-devaluation sucrose consumption test revealed that rats in the devalued group reduced consumption relative to controls, but, nevertheless, consumed a substantial amount of sucrose (∼5 mL). In contrast, in the present studies we gave five devaluation cycles to obtain a strong, yet, selective food aversion. If the primary aversion is weak, then this means that the US has not effectively been devalued. Consequently, only very modest US devaluation effects on conditioned responding would be expected. It seems possible that the animals displaying a devaluation effect in the Morrison et al. (2015) study had stronger sucrose aversions than the animals not displaying such an effect. This possibility is made more plausible by the fact that goal-tracking rats display increased palatability of liquid sucrose than sign-tracking rats (Patitucci et al. 2016), suggesting that goal-tracking rats may process the sensory aspects of sucrose more effectively. Data relevant to this possibility were not reported by Morrison et al. (2015).

Another potentially important procedural difference between our study and Morrison et al. (2015) was our use of a more complex training procedure than that of Morrison et al. (2015). In Experiment 1 we trained rats with 2 CS–US associations and in Experiment 2, we trained rats using a CS+/CS− discrimination task. In contrast, Morrison et al. (2015) trained rats with a single CS–US combination. We explicitly chose to use this more complex design in order to increase our ability to carefully control for any nonspecific effects in a devaluation task. In addition, we believe that our procedures more accurately reflect the complexities of real world situations that animal models are attempting to model. However, it seems possible that the added complexity of our training design (Stanhope 1989; Blundell et al. 2013; Robinson and Berridge 2013) may have facilitated sensory-specific learning thereby enhancing expression of a devaluation effect. This possibility may apply to situations involving multiple US types, but it is not immediately obvious that different rules should apply when a single US is used in a lever discrimination task (Experiment 2 here). If anything, that procedure may be expected to help define the relevant dimension of the predictive stimuli, rather than dimensions of the outcome. Future work will be needed to further clarify these different results, but the main conclusion from our studies, nonetheless, suggest that sign-tracking can indeed be mediated by a representation of its associated outcome.

A similar failure to observe an effect of outcome-devaluation on sign-tracking was shown by Patitucci et al. (2016). In this study, following conditioning with a lever CS paired with liquid sucrose, rats were given 15 min of free access to either water or sucrose and then tested for conditioned responding under extinction conditions. They found that goal-tracking, but not sign-tracking was reduced following sucrose versus water satiation. We are not sure how to interpret these results in light of our findings. However, we note several issues could be important. First, we may regard the “devaluation” treatment used in this study—15 min of sucrose satiation—to be a relatively weak one, when compared to our five cycles of LiCl aversion training. Second, in our Experiment 1 we examined the impact of an outcome-selective devaluation procedure on sign-tracking, whereas the Patitucci et al. (2016) task can be described as a more general devaluation procedure. Because their animals were food restricted it is likely they consumed very little water during the control condition (15 min exposure to plain water) and, therefore, were likely still relatively more “hungry” at the time of test than after sucrose satiation. In contrast, an outcome-selective procedure would have examined the impact of satiating the animals on the outcome associated with the lever (i.e., sucrose) versus another palatable outcome relevant to hunger (e.g., maltodextrin). Thus, it is not so clear whether we and Patitucci et al. (2016) are studying the same sort of devaluation effect. In our case, we can be sure that our results reflect the animals having learned an association between the levers and the specific foods with which they were paired and is not related to the fact that animals may have experienced different general levels of food motivation at the time of testing. A third consideration is that Patitucci et al. (2016) also showed a strong positive correlation between the tendency of their rats to goal-track and to display positive hedonic responses to sucrose (as assessed with a detailed lick rate analysis). This interesting finding presents a potential challenge for interpreting their data because a difference in palatability may have resulted in differences across individual animals in the amount of sucrose consumed during the 15 min satiation period. Although these authors presented group test data, it is possible that the subset of animals classified as goal-trackers actually drank more sucrose, and were more satiated, than those classified as sign-trackers. If these different subsets of animals contributed differentially to the sign- and goal-tracking measures, then it would be important to examine their data separately to have a clearer idea of whether individual differences in satiation may have contributed to the findings.

Regardless of how we interpret the differences between our devaluation results and those of Morrison et al. (2015) and Patitucci et al. (2016), we would agree with these authors that it is possible that sign-tracking CR may be somewhat less sensitive to some outcome-devaluation treatments than is goal-tracking. Indeed, Holland and Straub (1979) observed that components of a conditioned response sequence that were more proximal to reward delivery, such as magazine entry, were more sensitive to a LiCl-based outcome-devaluation treatment than response components that occurred earlier in the sequence, such as orienting responses at cue onset. Within a sign-tracking animal, lever contact responses necessarily precede magazine responses in terms of proximity to the reward. The greater sensitivity of magazine responses than lever contact responses to US devaluation seen in both of the Morrison et al. (2015) and Patitucci et al. (2016) studies, may, therefore, be related to the difference that Holland and Straub (1979) observed in their study. In particular, one might expect sign-tracking to be less affected by a devaluation treatment, especially if a relatively weak outcome devaluation treatment is used. In our study, the use of a powerful devaluation treatment likely enhanced the spread of the devaluation effect up the chain of responses to include behaviors less proximal to reward, such as sign-tracking.

Caution should be used, however, in interpreting this pattern of results to mean that the two response types are differentially controlled by a cortically driven expectancy process. First, if more powerful outcome-devaluation methods are used, as in our studies, the outcome-devaluation effect clearly spreads back to more distal sign-tracking responses. It is noteworthy that we observed a positive correlation between the magnitude of our selective outcome-devaluation effect and the extent to which animals were classified as sign-tracking rats (Fig. 2C). Thus, more extreme sign-trackers were “more” sensitive to outcome-devaluation than less extreme sign-trackers. Second, Holland and Straub (1979) also observed that if rapid rotation, rather than LiCl, was used to devalue the outcome, then the more distal response components were more sensitive to the devaluation treatment than was magazine responding. In other words, rotation devaluation caused a reduction in earlier components of the CR chain, but left responses more proximal to the US intact. This somewhat puzzling result at the very least suggests that the differential sensitivity of responses at different points in the CR chain should not be taken as evidence that the behavior, in question, is more or less based on a reward expectancy process (e.g., Flagel et al. 2009, 2011a; Meyer et al. 2012). Another aspect of our data support this conclusion by also showing that our mostly sign-tracking rats displayed outcome-devaluation effects on both sign-tracking CRs as well as on post-CS goal-tracking CRs. In other words, with strong US devaluation methods we were able to reveal that both behaviors appeared to be controlled by a detailed representation of the outcome.

One potential caveat worth considering here is that our result may be limited to sign-tracking populations, as the vast majority of rats in our study developed strong sign-tracking responses to the exclusion of goal-tracking during CS presentations. In contrast, the populations studied by both Morrison et al. (2015) and Patitucci et al. (2016) consisted of a more heterogeneous distribution of phenotypes. These distributions were more skewed toward the goal-tracking end of the spectrum, but also contained a significant portion of mixed profile respondents. Thus it is possible that our results may reflect attributes specific to sign-tracking rats. Nevertheless, our data strongly support the finding that sign-tracking within this population is expectancy-mediated.

In contrast to our findings, however, Nasser et al. (2015) recently provided some evidence to suggest that animals classified as sign-trackers were less likely than non-sign-trackers to behave in a “flexible” manner. In this study, rats initially underwent Pavlovian conditioning with a diffuse visual CS paired with pellet delivery, a procedure that resulted in the emergence of conditioned magazine responding to the light CS. One subgroup then underwent US devaluation training (two pellet-LiCl pairings) whereas another did not. Following a nonreinforced test session with the light CS, the rats underwent a second round of Pavlovian conditioning, but this time with a lever CS paired with a liquid sucrose US. They then used the PCA index during this lever training phase to classify animals as sign- or non-sign-trackers and reexamined their devaluation data during the previous light extinction test. The results from this study initially showed a nonsignificant US devaluation effect (comparing devalued to nondevalued animals) on magazine responding during the test with the light CS. However, a more detailed analysis of the scores revealed that animals that were subsequently classified as non-sign-trackers showed reduced magazine approach responding if the pellet US had been devalued, whereas the sign-trackers failed to show this effect. Thus, it appears that animals that later developed a sign-tracking phenotype had earlier been less flexible in their goal-track responding during the US devaluation test. We note that these findings do not directly conflict with our own since we assessed the impact of US devaluation on sign-track responding, whereas this was not examined in the Nasser et al. (2015) study. Moreover, we gave more extensive US devaluation training than Nasser et al. (2015). Another issue, however, is that the critical data reported by Nasser et al. (2015; Fig. 4) is difficult to interpret because pre-CS responding is not presented, and this raises concerns about the significant two-way interaction that is reported. If the same pattern of responding was seen during the pre-CS and CS periods, as suggested by the absence of a three-way interaction, then this complicates the analysis. We are, thus, not sure what to make of these findings but do note that our data along with those of Robinson and Berridge (2013) both show that sign-tracking animals are clearly highly flexible in their sign-track conditioned responding following US revaluation providing that powerful US revaluation procedures are used.

In order to more fully explore implications of the claim that sign- and goal-tracking rely on distinct psychological processes, we asked if sign- and goal-tracking rely on the same or different underlying prediction error learning processes. To address this, we used a variant of the Kamin blocking procedure (Kamin 1968) to determine if stimuli that only promote sign-tracking could block new learning to goal-track, and vice versa. In Experiment 3, we observed that a lever CS that produced primarily sign-tracking CRs blocked new learning of goal-tracking to an auditory stimulus paired with food in its presence. Furthermore, in Experiment 4 we observed that an auditory CS that produced goal-tracking CRs blocked acquisition of new sign-tracking CRs to a lever CS paired with food it its presence. These experiments demonstrate some interdependence of sign- and goal-tracking systems, and the data suggest that the acquisition of both types of responses rely on a common underlying prediction error mechanism.

Our finding of symmetric blocking effects between sign- and goal-tracking stimuli is not completely consistent with recent data reported by Holland et al. (2014). In their study, Holland et al. (2014) found, with rats, that whereas a sign-tracking lever CS could block acquisition of goal-tracking to an auditory CS, they failed to observe an auditory goal-tracking CS to block learning to sign-track to a lever CS. Their findings are inconsistent with earlier autoshaping studies conducted with pigeons (Blanchard and Honig 1976; Tomie 1976; Leyland and Mackintosh 1978; Khallad and Moore 1996), where it was shown that diffuse auditory stimuli interfered with the development of sign-tracking to a discrete visual stimulus during subsequent autoshaping training. The inconsistent findings may relate to procedural differences between our study and Holland et al. (2014). In our study, during the blocking phase in addition to presenting the previously trained CSs in compound with new stimuli, we continued to differentially reinforce the blocking and control stimuli when presented alone. This design is akin to a relative validity manipulation (see Wagner et al. 1968) and may have resulted in a stronger blocking effect. In contrast, Holland et al. (2014) differentially reinforced two auditory CSs, as we had, in phase 1, but in phase 2 of their study these stimuli were combined with levers and the rats only received training with these two stimulus compounds. We know from other studies that the magnitude of blocking is reduced when a relatively less “salient” CS is used to block conditioning to a more salient stimulus in phase 2 of Kamin's procedure (e.g., LoLordo et al. 1982). In Holland et al. (2014), the lever CSs appeared to be more salient than the pretrained auditory stimuli; this was apparent in that not only were they not blocked by the auditory stimuli, but conditioning to the lever CSs actually diminished goal-tracking CRs displayed to the auditory CSs. Holland et al. (2014) referred to this effect as a “vampire” effect. In our case, this “vampire” effect was also observed, to some extent, as magazine responding to the auditory CS was greatly reduced by the presence of the lever on compound trials during the blocking phase (Fig. 7A). However, since we continued to differentially reinforce the auditory CSs when presented by themselves during the blocking phase, this, presumably, enhanced the ability of that stimulus to interfere with the development of sign-tracking. Our procedure, therefore, may be construed as a more powerful method of assessing the blocking effect, even when somewhat less salient CSs are used to block more salient ones. Thus, the asymmetry reported by Holland et al. (2014) may have less to do with general prediction error mechanisms and more to do with different saliences of auditory and lever CSs.

We may express some caution here. While we have suggested that our blocking effect reflects the operation of a general prediction error mechanism, our experiments were not designed to distinguish among various explanations of the Kamin blocking effect. Rather, we focused on the question of symmetry between blocking in sign- and goal-tracking systems. We do note, however, that whether reduced CS (e.g., Mackintosh 1975; Pearce and Hall 1980) or US (e.g., Rescorla and Wagner 1972) processing mechanisms, ultimately, explain our blocking effects, all of these theories, in one way or another, depend upon the general notion of a prediction error driving changes in learning (be it changes in CS or US processing). Thus, our symmetrical blocking results suggest that a common prediction error system underlies both sign- and goal-tracking, but the precise nature of that system remains to be elucidated. One objection to this line of reasoning, however, is that some entirely different mechanism could explain our findings. Suppose, for instance, that rats in Experiment 4 approached and entered the food magazine during the auditory S1 stimulus and, as a result, simply did not see the actual lever on those stimulus compound trials. Poor learning would result from this. We think this explanation is unlikely because Figure 7A shows that while the auditory S1+ stimulus itself evoked strong levels of magazine approach responding, the introduction of the lever on S1 and Lev1 reinforced trials dramatically eliminated this strong magazine approach response. Thus, the lever was clearly processed on these trials, and whether it was processed less than on S2 and Lev2 trials is highly speculative. Furthermore, this account would not apply to our blocking results obtained in Experiment 3 where an added auditory stimulus was blocked by prior training of a lever CS. For these reasons we think our findings more likely reflect blocking by some prediction error mechanism.

Some have suggested that the tendency to sign- versus goal-track reflect different degrees of influence between two learning mechanisms: model-based and feature-model-free (Lesaint et al. 2014, 2015). It has been argued that model-based learning mechanisms produce goal-tracking CRs whereas, feature-model-free mechanisms drive sign-tracking. In this framework, model-based learning and the subsequent goal-tracking involve the formation of an explicit representation of the stimulus-outcome relationship. Whereas, feature-model-free mechanisms and the resultant sign-tracking are thought to arise from prediction error learning mechanisms that drives responding via a stimulus-response association whose strength is determined by the value of the outcome at the time of learning. Within this framework, sign-tracking should be insensitive to post-conditioning outcome-devaluation because it is presumed to be based on a stimulus-response association. The data in Experiments 1 and 2 challenge this model by demonstrating that sign-tracking is highly sensitive to outcome devaluation which is therefore an expectancy-mediated phenomenon. Furthermore, our data from Experiments 3 and 4 suggest that both sign- and goal-tracking similarly rely on shared prediction error mechanisms. It may be argued, however, that sign-tracking could entail a combination of both model-based (stimulus-outcome associations) and feature-model-free learning (stimulus-response associations), whereas goal-tracking is predominantly model-based. For instance, approach to the lever may arise via a stimulus-outcome association, whereas once in the presence of the lever the actual lever contact response may be driven by a stimulus-response association. If so, that could render sign-tracking CRs somewhat less sensitive to outcome devaluation manipulations, especially when they are relatively weak, while retaining some degree of sensitivity when devaluation manipulations are strong. Further work would be required to determine whether this mixture-type of approach has any merit.

All together our data suggest that sign- and goal-tracking are mediated by similar or, at least, highly overlapping psychological processes. We have shown that both types of CRs are partly controlled by an expectancy of the outcome, and that each form of learning depends upon a common underlying prediction error computation. Collectively, these data present a challenge to the idea that sign-tracking is a unique form of conditioned responding that once manifested is not driven by outcome expectancies or by standard error-prediction learning mechanisms. The outstanding question remains as to what causes one response to manifest over the other. Some researchers have suggested that these responses reflect “personality” traits and which response occurs is a reflection of individual differences in these intrinsic traits. One challenge to this idea was put forth by Patitucci et al. (2016) who suggested that the expression of sign- or goal-tracking is related to the perceived hedonic value of the outcome rather than to any intrinsic trait variable, with less palatable foods promoting sign-tracking and more palatable foods promoting goal-tracking. They provided strong evidence against the trait idea by showing that sign-tracking to one lever CS paired with one reward was uncorrelated with sign-tracking to a second lever CS paired with a second, qualitatively distinct, reward within the same individual. This within-subject correlation should have been highly positive if the trait model were correct.

In addition to food hedonics, other important variables include the CS–US interval and reward probability or uncertainty. Timberlake et al. (1982) demonstrated that rats were more likely to sign-track (to a moving ball CS) with longer and goal-track with shorter CS–US intervals. Robinson et al. (2014; 2015) reported increased levels of sign-tracking in animals trained with uncertain reward. These effects show that purely behavioral variables can determine whether rats develop sign- or goal-track tendencies, and this is another reason to express caution regarding the trait model. Perhaps one way to integrate some of these findings is with the possibility, noted above, that the tendency to sign- or goal-track reflects individual differences in the hedonic value of reward. If this, in turn, could influence perception of the CS–US interval, then, perhaps, some of the results above could be explained. When the US is less desirable the perceived CS–US interval may be inflated compared to when it is more desirable, and this could promote sign-tracking in the former case and goal-tracking in the latter. There is some precedent for thinking that the rat's estimate of time is affected in this way by reward value (e.g., Galtress et al. 2012; Kirkpatrick 2014; but see Delamater et al. 2018), but it is less clear that this sort of explanation would apply to a reward uncertainty manipulation. That would require that the hedonic value of less certain rewards is lower than for certain rewards.

A common contemporary interpretation of the psychological divergence in sign- and goal-tracking is provided by incentive salience models. In this framework, sign-tracking is mediated largely by an affective process where the CS, itself, is imbued with heightened affective significance (“incentive salience”), whereas goal-tracking is mediated mostly by an underlying cognitive outcome-expectancy process (Meyer et al. 2012; Huys et al. 2014; Flagel and Robinson 2017). Our data question the sharpness of this distinction because they show that sign-tracking CRs, as well as animals that are almost exclusively classified as “sign-trackers,” are highly sensitive to outcome-devaluation and error prediction processes. It may still be true that the sign-tracking phenotype is somewhat less sensitive than the goal-tracking phenotype to outcome-devaluation (Morrison et al. 2015; Patitucci et al. 2016), but our data suggest that there is a lot of overlap in the systems controlling sign- and goal-tracking.

What remains is a way to characterize the nature of the learning that does underlie these two systems. Following Konorski (1967) we suggest that two types of associations, at least, may be formed between the CS and the US. These associations depend upon how the US is encoded, and there is much reason to suspect that appetitive USs can be encoded both in terms of their general affective/motivational significance (e.g., whether it is “good” or “bad”) and also in terms of their more specific sensory characteristics (Balleine and Killcross 2006; Delamater 2012). If the brain encodes these aspects differently, then perhaps the CS independently associates with these distinct motivational and sensory US components. Accordingly, when we speak of “incentive salience” attribution, perhaps this refers to an association having been established between the lever CS and the general motivational characteristics of reward. An “expectancy process” could refer to an association having been established between the CS and the highly specific sensory qualities of the US. Sensitivity to outcome-devaluation clearly shows control by the latter type of association, and, therefore, we would conclude that sign-tracking CRs and sign-tracking animals have learned in this manner. Nonetheless, the relative balance between these two forms of associations may vary and that variation could account for some of the divergences observed in the literature when US devaluation effects have been studied for sign- and goal-tracking subjects. Further, our studies only begin to address the issue of the degree to which the same or different prediction error mechanisms might underlie these two forms of learning. Our data suggest there is much in common, but, clearly, more work on this problem is needed.

In summary, our data show clear sensitivity of sign-tracking to outcome-devaluation procedures. This implies that sign-tracking, itself is a phenotype that can be highly flexible and, therefore, “cognitive” in its appearance. Furthermore, our data suggest that a common prediction error mechanism underlies sign- and goal-tracking as the two forms of learning appear to mutually compete with one another in a Kamin blocking design. Further work is needed to more fully understand how the neural circuitry underlying these two forms of learning both overlap and diverge.

Materials and Methods

Experiment 1: within-group US devaluation effect

Subjects

Sixteen naïve male Long Evans rats approximately 10 wk old were procured from Charles River Laboratories. Rats were pair housed in standard plastic tubs in a colony room on a 14:10 light: dark cycle. Throughout training and testing rats had constant access to water and were held at 85% of their ad libitum free-feeding weights (85% weights ranged between 323–391 g).

Apparatus

Eight identical Med Associates conditioning chambers (ENV-008) were used, and each was housed in a Med Associates sound- and light-resistant shell. The conditioning chambers measured 30.5 cm × 24.1 cm × 21.0 cm. The two end walls were constructed of aluminum, and the sidewalls and ceiling were clear Plexiglas. The floor consisted of 0.48-cm diameter stainless steel rods spaced 1.6 cm apart. In the center of one end wall 2.54 cm above the grid floor was a recessed dual pellet/liquid magazine (ENV-202RMA) measuring 5.7 × 5.7 (length × width). The USs consisted of a single 45-mg food pellet supplied by TestDiet (MLab rodent pellets) and BioServ (Purified rodent pellets), and, when scheduled, these were dropped onto the magazine floor (pellet side). These pellets were chosen because the caloric profile is very similar and because prior work in our laboratory has established that rats can readily discriminate between their sensory properties (Delamater and Nicolas 2015; Delamater et al. 2017). The magazine included an infrared detector and emitter (Med Associates ENV-303HDA) enabling recording of head movements inside the magazine. These were located 1.0 cm above the magazine floor and 1.0 cm recessed from the front wall. Located 8.9 cm to the right and left of the magazine (center to center) and 6.4 cm above the floor were two retractable response levers (ENV-112CM). The levers only extended into the chamber during stimulus presentations. A 28-v house light was centrally positioned at the top of the side wall opposite the food magazine. This house light was on during the duration of the session. A speaker was mounted next to the house light, and was used to present clicker and white noise stimuli through a Med Associates audio generator (ANL-926). The white noise measured 4 dB, and the clicker 5 dB above a background level of 79 dB (C weighting, Realistic Sound Meter placed in the middle of the chamber with the door closed). A fan attached to the outer shell provided cross-ventilation within the chamber and produced background noise. All experimental events were controlled and recorded automatically by a computer running MedPC software located in the same room.

Magazine training

Once the animals had reached their target weights they underwent 2 d of magazine training. Each rat received two separate training sessions per day for each of the two pellet types used in this study (45 mg BioServ Dustless Purified Pellets and 45 mg 5TUM TestDiet Grain-Based Pellets). For magazine training, animals were placed into the conditioning chambers for 20 min during which 20 pellets of one type were delivered into the food magazine on a variable time (VT) 60 sec schedule. The order of sessions with the two outcome types was counterbalanced across animals and days. Head entries into the magazine were recorded during each session.

Pavlovian training

In the next phase animals underwent 12 d of Pavlovian training using a delay conditioning paradigm, wherein animals were taught to associate an 8-sec presentation of one lever (CS1) with delivery of one BioServ pellet (US1) and another lever (CS2) with delivery of a TestDiet pellet (US2). These pairings were counterbalanced across animals. Animals underwent two separate daily sessions—one with each lever-outcome pair—and these two sessions were separated by approximately 2 h. The order of training was counterbalanced across days using a double alternating scheme. Each training session consisted of 20 CS–US pairings separated by a mean inter-trial-interval (ITI) of 1 min in duration. Head entries into the magazine were recorded 8-sec prior to and 8-sec during CS presentations. Lever contacts during the CS were also recorded. The latency to contact the lever and to approach the magazine was recorded separately on each trial.

US devaluation training

Following Pavlovian training all animals were given 5 US devaluation training cycles to ensure that intake of the devalued outcome was completely suppressed in the conditioning chambers. On Day 1 of each cycle, the animals were placed in the experimental chambers for 20 min and 20 pellets of one type were delivered randomly in time. Immediately following this session, the rats were administered a 1% bodyweight intraperitoneal (IP) injection of 0.3 M lithium chloride. On Day 2 of each cycle, the animals were placed back in the experimental chambers for another 20-min session during which time 20 pellets of the other type (not given the day before) were presented, but this session was not followed by any injections. The animals were split into two counterbalanced subgroups that were matched in their lever contact and magazine performance to the two levers across training. Which specific outcome was devalued (and lever-outcome combination) was counterbalanced across animals. During each session head entries were recorded, but no levers were presented.

Testing

On the day following the fifth devaluation cycle, rats were given three extinction test sessions conducted on successive days to determine whether the US devaluation treatment had an impact on both magazine responding and lever contacts to either CS. In each test session, each lever CS was presented 10 times spaced by ITIs with a mean duration of 1 min. Presentations were randomly ordered across the session. No food was delivered into the magazine during these sessions. Magazine entries 8 sec prior to, during, and post the CS presentations were recorded. Lever contacts during the CS were also recorded. The latencies to approach the magazine or contact the CS following CS onset were recorded separately. Animals were not given any retraining between tests.

Experiment 2: between-group US devaluation effect

Subjects

Nine naïve male and three naïve female Long Evans rats were used in the study. These animals were bred at Brooklyn College of Charles River descent. They were housed and maintained as in Experiment 1. The 85% weights ranged between 383–449 g for males, and between 221–238 g for females.

Apparatus

The same apparatus was used as in Experiment 1.

Magazine training

The same procedure was used as in Experiment 1.

Pavlovian training

The procedure was similar in most respects to that used in Experiment 1 with the following exceptions. One lever CS was paired with a pellet US (TestDiet) (10 trials per session) and the other lever CS was nonreinforced (10 trials per session). Both trial types occurred randomly within the same session with an ITI of a mean of 1 min. This training continued for 16 d.

US devaluation training

The rats were segregated into 2 groups, Groups Devaluation and No Devaluation, and these were matched for their lever and magazine responding during Pavlovian training. Both groups were given five devaluation cycles similar to that used in Experiment 1. However, Group Devaluation rats were given pellet—LiCl pairings on Day 1 of each cycle and were placed in the chamber for 20 min without any pellets or injection on Day 2 of each cycle. Group No Devaluation was given a LiCl injection following simple exposure to the chamber on Day 1 of each cycle but free pellets without injections on Day 2. Thus, this group experienced pellets and LiCl unpaired.

Testing

Three nonreinforced test sessions were conducted as in Experiment 1.

Experiment 3: blocking of goal-tracking

Subjects

Eight naïve male and eight naïve female Long Evans rats, bred at Brooklyn College, were used in the study. The males 85% weights ranged between 300–338 g and the females ranged between 207–238 g.

Apparatus

The same apparatus was used as in Experiment 1.

Magazine training

The same procedure was used as in Experiment 1.

Pavlovian training

Training consisted of two phases. The first phase was conducted as in Experiment 2 where one lever CS was paired with a pellet US (TestDiet) and the other was nonreinforced over 16 sessions. The second “blocking” phase consisted of six trials with each of the reinforced and nonreinforced levers (as in phase 1), but, in addition, there were four trials in which lever 1 was combined with auditory CS1 (white noise, clicker, counterbalanced) and four trials in which lever 2 was combined with auditory CS2 (clicker, white noise, counterbalanced). Each of these compound stimuli was paired with reinforcement. The four trial types were randomly interspersed with a mean ITI of 1 min (as in the pretraining phase). It was expected that since lever 1 was trained as a strong predictor of the US it could “block” new learning about auditory CS1. Auditory CS2 served as a control stimulus since the pellet US was unexpected at the time CS2 was paired with it; thus, normal learning should proceed to auditory CS2.

Testing

There were three test sessions that were conducted as in the blocking phase with the exception that five nonreinforced test presentations each of auditory CS1 and CS2 were presented in an ABBABAABAB sequence during the second half of the test sessions. The first half of these sessions was just like the blocking phase sessions (but with fewer total trials). Data from these nonreinforced auditory CS1 and CS2 test trials constituted the main data from the study.

Experiment 4: blocking of sign-tracking

Subjects

Eight naïve male and eight naïve female Long Evans rats, bred at Brooklyn College, were used in the study. The males 85% weights ranged between 415–513 g and the females ranged between 243–295 g.

Apparatus

The same apparatus was used as in Experiment 1.

Magazine training

The same procedure was used as in Experiment 1.

Pavlovian training

The same procedures were used as in Experiment 3, except that the roles of lever 1 and lever 2 were switched with auditory CS1 and CS2. Thus, we initially gave differential reinforcement of the two auditory CSs and asked if they could differentially impact learning about the different levers during compound training. Note that the inclusion of auditory CS1+ and CS2− trials throughout these sessions should maintain CS1's ability to block conditioning to the accompanying lever on compound training trials. Holland et al. (2014) pretrained with the auditory CSs and then gave compound trials without the inclusion of additional reinforced auditory CS1+ trials. This may be construed as a weaker blocking manipulation than the procedure used here.

Testing

This was also conducted as in Experiment 3 except that nonreinforced lever alone trials occurred in the second half of the test sessions.