Isolating the incentive salience of reward-associated stimuli: value, choice, and persistence

Abstract

Sign- and goal-tracking are differentially associated with drug abuse-related behavior. Recently, it has been hypothesized that sign- and goal-tracking behavior are mediated by different neurobehavioral valuation systems, including differential incentive salience attribution. Herein, we used different conditioned stimuli to preferentially elicit different response types to study the different incentive valuation characteristics of stimuli associated with sign- and goal-tracking within individuals. The results demonstrate that all stimuli used were equally effective conditioned stimuli; however, only a lever stimulus associated with sign-tracking behavior served as a robust conditioned reinforcer and was preferred over a tone associated with goal-tracking. Moreover, the incentive value attributed to the lever stimulus was capable of promoting suboptimal choice, leading to a significant reduction in reinforcers (food) earned. Furthermore, sign-tracking to a lever was more persistent than goal-tracking to a tone under omission and extinction contingencies. Finally, a conditional discrimination procedure demonstrated that sign-tracking to a lever and goal-tracking to a tone were dependent on learned stimulus–reinforcer relations. Collectively, these results suggest that the different neurobehavioral valuation processes proposed to govern sign- and goal-tracking behavior are independent but parallel processes within individuals. Examining these systems within individuals will provide a better understanding of how one system comes to dominate stimulus–reward learning, thus leading to the differential role these systems play in abuse-related behavior.

Through associative learning, stimuli paired with biologically significant events (e.g., food or drugs of abuse) can come to influence behavior on their own and contribute to the development and persistence of psychopathologies, including addiction (Hogarth et al. 2010). Interestingly, not all reward-paired stimuli function equivalently; some stimuli are more likely to be attributed with value, and there are individual differences in the likelihood that a stimulus will attain value during stimulus–reward learning (Tomie et al. 2008; Robinson et al. 2014). Furthermore, the propensity to attribute a reward-predictive stimulus with incentive value has been demonstrated to predict drug reward in a variety of preclinical models of substance abuse (Saunders and Robinson 2010, 2011; Anderson and Spear 2011; Beckmann et al. 2011; Peters and De Vries 2014). Thus, understanding the neurobehavioral mechanisms that underlie the propensity to attribute reward-related stimuli with incentive value may shed light on the mechanisms that mediate substance abuse and reveal novel neurobehavioral targets for substance abuse therapies.

An increasing number of studies have used a Pavlovian conditioned approach (PCA) task, where a lever located next to a food receptacle reliably predicts a noncontingent food reward, to elicit sign- or goal-tracking responses from a rat or a mouse (Boakes 1977; Tomie et al. 2012; Robinson et al. 2014). Sign-tracking responses to the lever (approach and interaction with the lever) are theorized to exemplify “incentive salience” or value that has been attributed to the lever stimulus, above and beyond the predictive relationship shared between the lever and food (Robinson et al. 2014). On the other hand, goal-tracking responses to the food receptacle (approach to the location of forthcoming reward in the presence of the lever) are theorized to be more representative of a simple predictive relationship between the lever stimulus and reward, without value attribution to the predictive lever stimulus (Robinson et al. 2014). The incentive value associated with sign-tracking to a reward-predictive lever stimulus is further exemplified by the ability of the lever to serve as a more robust conditioned reinforcer, relative to the same stimulus associated with goal-tracking behavior (Robinson and Flagel 2009). Furthermore, it has recently been hypothesized that different valuation systems that operate as independent but parallel processes within an individual may drive sign- and goal-tracking and their associated neurobehavioral repertoires (Clark et al. 2012). This is supported by the differential conditioned reinforcing value of reward-predictive stimuli associated with each response type (sign- versus goal-tracking). Additionally, although the neurobiological mechanisms that govern sign- and goal-tracking responses are largely unknown, there is evidence that sign- and goal-tracking responses differentially engage dopaminergic signaling, where sign-tracking appears to be more sensitive to changes in dopaminergic signaling than and goal-tracking (Flagel et al. 2011; Saunders and Robinson 2012, 2013). Collectively, the above results indicate that different neurobehavioral systems may govern sign- and goal-tracking, and the propensity of an individual to preferentially exhibit either of these response types may be related to functional differences in the accompanying neurobehavioral systems or the tendency for one system to dominate stimulus–reward learning across individuals. Thus, the incentive salience or value of a stimulus may be determined by what neurobehavioral system it engages. However, the functional aspects of the proposed systems that may govern sign- and goal-tracking and their subsequent repertoires are not well characterized.

Much of the work to date investigating the differential processes governing sign- and goal-tracking has relied on a between-subjects cohort approach (Robinson et al. 2014), where depending on performance within a single-lever PCA task, animals are classified as either sign- or goal-trackers in relation to a particular performance threshold (Meyer et al. 2012). Interestingly, there is also a large literature demonstrating that pairing different stimuli (e.g., levers or tones) with reward can differentially induce sign- and goal-tracking. For example, pairing a tone with food results in primarily goal-tracking behavior (Holland 1977; Harrison 1979; Cleland and Davey 1983; Holland et al. 2014; Meyer et al. 2014). Furthermore, Meyer et al. (2014) demonstrated that a reward-predictive tone that elicits goal-tracking behavior maintains the same amount of conditioned reinforcement in animals prescreened as sign-trackers or goal-trackers, while the lever stimulus associated with sign-tracking was the most robust conditioned reinforcer. Importantly, Meyer et al. (2014) also demonstrated that the conditioned reinforcement maintained by the tone in sign- and goal-trackers was equivalent to that maintained by a lever in animals prescreened as goal-trackers. Collectively, these results indicate that a tone associated with goal-tracking and a lever associated with goal-tracking are of equivalent value, maintaining identical conditioned reinforcement regardless of whether an animal is labeled as a sign-tracker or goal-tracker. The functional differences in response type and corresponding conditioned reinforcement elicited by different stimuli suggest the possibility of isolating the functional characteristics of sign- and goal-tracking through the use of different stimuli “within” an individual (highlighting the potentially independent-but-parallel nature of the proposed systems), rather than different groups of animals (sign versus goal trackers).

To determine the functional properties of stimuli associated with sign- and goal-tracking, the current experiments took advantage of the fact that different stimuli elicit different conditioned responses. First, we investigated whether or not sign-tracking (defined as physical interaction with the stimulus) was dependent on visual modality and location (distance from food receptacle); specifically, we paired an auditory tone or two different visual stimuli (nosepoke or lever) that were equidistant from the food receptacle with noncontingent food delivery. If sign-tracking is dependent upon the ability of an animal to localize a visual stimulus, we reasoned that, like a lever stimulus, we would elicit sign-tracking (nosepokes into the nosepoke apparatus) toward a lit nosepoke apparatus. Next, we designed a two-conditioned stimulus PCA (2-CS PCA) task to promote sign-tracking responses to a lever and goal-tracking responses to a tone within a single animal. Given that the two proposed neurobehavioral systems thought to govern sign- and goal-tracking response types are considered independent but parallel processes within individuals (Dayan et al. 2006; Clark et al. 2012; Dayan and Berridge 2014; Huys et al. 2014), we reasoned that specifically engaging these two systems within a single individual would better isolate and compare the functional disparities between each. This 2-CS PCA task would also be useful for studying the development of the proposed arbitration process thought to determine the relative dominance of one system over the other (Dayan et al. 2006; Huys et al. 2014; Lesaint et al. 2014), a process that would be particularly difficult to assess using between-subject cohort designs.

Following training on the 2-CS PCA task, we then conducted a series of experiments to determine the relative value of the stimuli associated with sign- and goal-tracking by determining the ability of each stimulus to maintain novel operant responding in a conditioned reinforcement test and a novel choice task designed to measure the relative value of reward-associated stimuli. Just as a lever is a more robust conditioned reinforcer in sign-trackers than goal-trackers using the between-subject cohort model (Robinson and Flagel 2009), we hypothesized that the lever associated with sign-tracking in 2-CS PCA would serve as a more robust conditioned reinforcer, relative to the tone associated with goal-tracking within animals. Relatedly, we determined the relative value of each stimulus prior to 2-CS PCA as verification that any differences between the stimuli following 2-CS PCA training were not due to some preexisting difference in stimulus salience or associability. Additionally, if the lever associated with sign-tracking has more value relative to the tone associated with goal-tracking, we hypothesized that animals should choose a lever that predicts a food pellet over a tone that predicts the same pellet. Furthermore, to determine if sign-tracking to a lever and goal-tracking to a tone are differentially representative of stimulus–response versus response–outcome relationships (Williams and Williams 1969; Stiers and Silberberg 1974), following initial 2-CS PCA training, we subsequently trained animals on extinction or omission contingencies. Finally, to demonstrate that the acquisition of sign-tracking to a lever and goal-tracking to a tone are dependent on a learned stimulus–food relationship, we trained animals on a within-subject four-conditioned stimulus (4-CS) PCA discrimination task, with an additional lever and tone that were never followed by food. We hypothesized that if sign-tracking to a lever and goal-tracking to a tone are dependent upon a stimulus–food relationship, then both response types should only be observed to the lever and tone that were predictive of food.

Results

ST and GT responses to a lever, nosepoke, or tone CS during PCA

Figure 1 illustrates the acquisition of ST and GT response rates to the lever, nosepoke, and tone conditioned stimuli over the 10-d training period. Because there were no sign-tracking responses to the tone, sign-tracking responses directed to the lever and nosepoke were analyzed alone. Figure 1A illustrates the increase in sign-tracking responses to the lever CS only over the 10-d training period. Linear mixed effects analysis indicated that the increase in sign-tracking responses to the lever over training session was significant [F_(1,10) = 8.41, P < 0.05]; there was also a significant main effect of stimulus [F_(1,10) = 19.65, P < 0.05], indicating that sign-tracking responses were significantly higher to the lever, relative to the nosepoke; and, linear mixed effects analysis indicated a significant training session × stimulus interaction [F_(2,15) = 15.82, P < 0.05], indicating that the trajectory for sign-tracking responses over the 10-d training period was different for the lever and the nosepoke, namely sign-tracking increased over sessions for the lever and did not develop over sessions for the nosepoke. Additionally, Figure 1B illustrates the acquisition of goal-tracking response rates to the lever, nosepoke, and tone over the course of the 10-d training period. Linear mixed effects analysis indicated that there was a main effect of training session [F_(1,15) = 21.21, P < 0.05], indicating that goal tracking changed over training sessions; there was a main effect of stimulus [F_(2,15) = 5.91, P < 0.05], indicating that goal-tracking responses were different for the lever, nosepoke, and tone conditioned stimuli; and, there was a training session × stimulus interaction [F_(2,15) = 8.21, P < 0.05], indicating that the trajectory for goal-tracking responses over the 10-d training period was different for the lever, nosepoke, and tone conditioned stimuli, namely goal-tracking decreased over sessions for the lever and increased over sessions for both the nosepoke and tone. Finally, Figure 1C illustrates the changes in the average probability difference score over the course of the 10-d training period for the lever, nosepoke, and tone conditioned stimuli. Linear mixed effects modeling indicated a significant main effect of training session [F_(1,15) = 5.27, P < 0.05], indicating that probability difference scores changed over training sessions; there was a significant main effect of stimulus [F_(2,15) = 43.75, P < 0.05], suggesting that the probability difference scores diverged for the lever, nosepoke, and tone conditioned stimuli. Linear mixed effects modeling also indicated a significant training session × stimulus interaction [F_(2,15) = 15.82, P < 0.05], indicating that the trajectory for changes in probability difference scores were significantly different for the lever, nosepoke, and tone conditioned stimuli, namely probability difference scores increased toward 1.0 for the lever and decreased toward −1.0 for both the nosepoke and tone. Collectively, the above results indicate that only the lever CS was associated with predominant sign-tracking responses, while the both the nosepoke and tone CS were associated with predominant GT responses.

Figure 1.

When conditioned alone between groups, a lever elicits predominant sign-tracking, and both a nosepoke and a tone elicit predominant goal-tracking. (A) Mean (±SEM) response rate (responses/second; r/sec) for sign-tracking (ST) responses to either a lever, nosepoke, or tone CS (B) Mean (±SEM) response rate (responses/second; r/sec) for goal-tracking (GT) responses to either a lever, nosepoke, or tone CS. (C) Mean (±SEM) difference in response probability (ST probability − GT probability) for either a lever, nosepoke, or tone CS. 1.00 indicates exclusive sign-tracking and −1.00 indicates exclusive goal-tracking.

2-CS PCA and conditioned reinforcement

Figure 2 illustrates the acquisition of sign- and goal-tracking response rates to the lever and tone conditioned stimuli over the 14-d training period. Because there were no sign-tracking responses to the tone, sign-tracking responses directed to the lever were analyzed alone. Figure 2A illustrates the increase in sign-tracking responses to the lever CS over the 14-d training period, and linear mixed effects analysis indicated that the increase in sign-tracking responses over training session was significant [F_(1,11) = 13.55, P < 0.05]. Additionally, Figure 2B illustrates the acquisition of goal-tracking response rates to both the lever and tone over the course of the 14-d training period. Linear mixed effects analysis indicated that there was a main effect of training session [F_(1,11) = 45.27, P < 0.05], indicating that goal tracking changed over training sessions; there was a main effect of stimulus [F_(1,11) = 19.57, P < 0.05], indicating that goal-tracking responses were different for the lever and tone conditioned stimuli; and, there was a training session × stimulus interaction [F_(1,11) = 32.99, P < 0.05], indicating that the trajectory for goal-tracking responses over the 14-d training period was different for the lever and tone conditioned stimuli, namely goal-tracking decreased over sessions for the lever and increased over sessions for the tone. Finally, Figure 2C illustrates the changes in the average probability difference score over the course of the 14-d training period for both the lever and tone conditioned stimuli. Linear mixed effects modeling indicated a significant main effect of stimulus [F_(1,11) = 54.01, P < 0.05], suggesting that the probability difference scores diverged for the lever and tone conditioned stimuli. Linear mixed effects modeling also indicated a significant training session × stimulus interaction [F_(1,11) = 28.85, P < 0.05], indicating that the trajectory for changes in probability difference scores were significantly different for the lever and tone conditioned stimuli, namely probability difference scores increased toward 1.0 for the lever and decreased toward −1.0 for the tone. Collectively, the above results indicate that the lever CS was associated with predominant sign-tracking responses, while the tone CS was associated with predominant goal-tracking responses.

Figure 2.

A lever elicits predominant sign-tracking and a tone elicits predominant goal-tracking in within-subject 2-CS PCA training. (A) Mean (±SEM) response rate (responses/second; r/sec) for sign-tracking (ST) responses to either a tone or lever CS during 2-CS PCA training. (B) Mean (±SEM) response rate (responses/second; r/sec) for goal-tracking (GT) responses to either a tone or lever CS during 2-CS PCA training. (C) Mean (±SEM) difference in response probability (ST probability − GT probability) for a tone or lever CS during 2-CS PCA training. 1.00 indicates exclusive sign-tracking and −1.00 indicates exclusive goal-tracking.

Figure 3 illustrates the number of operant nosepoke responses maintained by both the lever and tone conditioned stimuli during conditioned reinforcement testing, following 2-CS PCA training. Figure 3A illustrates the average number of nosepokes for both the lever and tone conditioned stimuli following 2-CS PCA training. Mixed effects modeling indicated that there was a significant main effect of stimulus [F_(1,11) = 4.71, P < 0.05], a significant main effect of response type [F_(1,11) = 53.12, P < 0.05], and a significant stimulus × response type interaction [F_(1,11) = 8.92, P < 0.05], suggesting that both the lever and tone conditioned stimuli maintained greater nosepoke responding on the active nosepoke, but that active nosepoke responding was greater for the lever CS. Post hoc tests indicated that active nosepokes for the lever CS were greater than all other comparisons. Additionally, Figure 3B illustrates the number of active and inactive nosepokes over each 30-min conditioned reinforcement test in 5-min blocks. Linear mixed effects modeling indicated a significant main effect of block [F_(1,11) = 49.23, P < 0.05], a main effect of stimulus [F_(1,11) = 5.97, P < 0.05], and a significant block × stimulus interaction [F_(1,11) = 7.14, P < 0.05], suggesting that active responses decreased over block, active responses were generally higher for both stimuli, and that active responses decreased at a slower rate across block for the lever stimulus. Collectively, the above results indicate that the lever CS was attributed with greater value than the tone CS.

Figure 3.

A lever CS associated with sign-tracking is a more robust conditioned reinforcer, relative to a tone CS associated with goal-tracking, in a conditioned reinforcement test following within-subject 2-CS PCA training, and this effect is not due to a preexisting difference in salience between the two stimuli. (A) Mean (±SEM) number of nosepokes made to earn access to the lever CS or tone CS during the conditioned reinforcement test following 2-CS PCA training. (B) Mean (±SEM) number of nosepokes made to earn access to the lever CS or tone CS during 5-min blocks across each 30-min conditioned reinforcement test following 2-CS PCA training. (C) Mean (±SEM) number of nosepokes made to earn access to the lever or tone stimuli during the conditioned reinforcement test prior to any 2-CS PCA conditioning. (D) Mean (±SEM) number of nosepokes made to earn access to the lever or tone stimuli during the conditioned reinforcement test following 2-CS PCA conditioning. (*) significant difference between active and inactive; (ˆ) significant difference between active responses for lever and active responses for tone; (^★) significant difference between active responses for lever during pre- versus post-conditioning test.

Figure 3C illustrates the average number of nosepokes for both the lever and tone absent any conditioning to either stimulus, and Figure 3D illustrates the average number of nosepokes for both the lever and tone from the same animals following 14 sessions of 2-CS PCA training. Mixed effects modeling revealed a main effect of response type [F_(1,5) = 44.13, P < 0.05], a main effect of pre- versus post-conditioning test session [F_(1,5) = 7.36, P < 0.05], a significant stimulus × response type interaction [F_(1,5) = 7.28, P < 0.05], a significant stimulus × pre- versus post-conditioning test session interaction [F_(1,5) = 23.15, P < 0.05], a significant response type × pre- versus post-conditioning test session interaction [F_(1,5) = 10.05, P < 0.05], and a significant stimulus × response type × pre- versus post-conditioning test session three-way interaction [F_(1,5) = 10.05, P < 0.05]. Post hoc tests revealed that there were no differences in responding during the preconditioning tests, active responses for the lever and tone were higher than inactive responses during the post-conditioning tests, and active responses for the lever stimulus following 2-CS PCA training were higher than all other responses, indicating that both the lever and tone maintained the same amount of responding during the preconditioning tests but the lever maintained the greatest amount of responding during the post-conditioning tests. Collectively, the above results indicate that the lever and tone stimuli were identical in salience/value, when neither of them was conditioned and the lever attained more value after both stimuli were conditioned during 2-CS PCA training.

2-CS choice

Figure 4A illustrates the average sign- and goal-tracking response rates from the final session of 2-CS PCA training. As above, the lever CS produced significantly greater sign-tracking rates relative to goal-tracking rates [F_(1,11) = 19.94, P < 0.05] , and the tone CS was associated with only goal-tracking behavior (no analysis was performed for the tone because of the absence of sign-tracking behavior). Figure 4B illustrates a preference for the lever CS, when given an option between the lever and the tone, each followed by one food pellet across five training blocks. Thus, even though each stimulus (lever and tone) led to the same food pellet, animals preferred the option that led to the 8-sec lever followed by the food pellet. Figure 4C illustrates that when the odds against the lever predicting the food pellet were systematically increased over the five training blocks, preference for the lever CS decreased according to a hyperbolic function (lines drawn in Fig. 4C). NLME modeling indicated a main effect of condition (lever and tone, tone removed, and lever removed) on the intercept parameter (S), indicative of subjective lever value at equal food probability, [F_(2,163) = 163.28, P < 0.05] and a significant main effect of condition on the slope parameter (k), indicative of the rate of discounting of the lever CS value with increasing odds against the lever predicting food, [F_(2,163) = 3.45, P < 0.05]. Post hoc analyses indicated that both main effects were due to a large shift toward tone preference when the lever was removed from the terminal link, as indicated by a significant decrease in the S parameter and a significant increase in the k parameter relative to all other conditions.

Figure 4.

A lever CS associated with sign-tracking is preferred over a tone CS associated with goal-tracking in a choice test following within-subject 2-CS PCA training. (A) Mean (±SEM) of sign- and goal-tracking response rates to a lever CS and tone CS during the 14th session from the 2-CS PCA training period prior to choice training. (B) Mean (±SEM) percent (%) choice of the lever CS when the probability of reinforcement was equivalent (1.0) across both alternatives. (C) Mean (±SEM)% choice for the lever CS as a function of increasing odds against [(1 − p)/p] food delivery following the lever CS. Solid lines are choice functions defined by s/(1 + k(odds against)), (s) sensitivity to lever CS value; (k) discounting rate of lever CS value.

2-CS PCA omission and extinction

Figure 5 illustrates the rates of responding, relative to baseline response rates, over the course of the 10 sessions of training on the omission contingency or extinction for both the lever and tone conditioned stimuli. Figure 5A,B illustrates the baseline (BL) sign- and goal-tracking response rates for both the lever and tone conditioned stimuli. As above, during the last day of 2-CS PCA training (BL) both groups demonstrated predominant sign-tracking responses to the lever and goal-tracking responses to the tone. Additionally, Figure 5A,B illustrate the rate of decline for both sign- and goal-tracking responses for both the lever and the tone over the 10-session omission-training and extinction-training periods, respectively. NLME modeling indicated no significant effect of stimulus on the intercept parameter (a; indicative of response rates at baseline) for either the omission or extinction group, suggesting that baseline sign-tracking rates to the lever and goal-tracking rates to the tone were similar for the omission group and the extinction group. NLME modeling also indicated that the rate of response decline (b) was higher for goal-tracking to the tone, relative to sign tracking to the lever for the omission group [F_(1,207) = 2.57, P < 0.05] and the extinction group [F_(1,249) = 2.57, P < 0.05] over the course of the 10-d training. Thus, changing the contingency between the stimulus and food through extinction and the response and food through omission resulted in a faster decline of goal-tracking to the tone, relative to sign-tracking to the lever. Note that, in general, rates of response decline were lower under the omission contingency, as indicated by the lower rate parameter estimates (b) under the omission contingency. Finally, because there was no goal-tracking to the lever at baseline to undergo decline, we analyzed the goal-tracking to the lever in both the omission and extinction groups with linear mixed-effects modeling, rather than NLME; the results revealed no significant effect of session on goal-tracking to the lever, suggesting that the absence of goal-tracking to the lever at baseline persisted throughout the 10 training days of omission or extinction training.

Figure 5.

Sign-tracking is more persistent under omission or extinction contingencies following within-subject 2-CS PCA training. (A) Mean (±SEM) of sign- and goal-tracking response rates to lever CS and tone CS during omission. (B) Mean (±SEM) of sign- and goal-tracking response rates to lever CS and tone CS during extinction contingencies. (C) Mean (±SEM) of sign- and goal-tracking response rates to lever CS and tone CS during first session of omission. (D) Mean (±SEM) of sign- and goal-tracking response rates to lever CS and tone CS during first session of extinction. Lines are exponential decay functions defined by a × e^−b×s, (a) baseline ST and GT response rate or rate during first 4-trial block from first session, and (b) decay rate of ST and GT responding over extinction sessions or over 4-trial blocks from first session. (BL) baseline responding from day 14 of 2-CS training.

To more closely examine the differential sensitivity of sign-tracking to a lever and goal-tracking to a tone under omission and extinction contingencies, Figure 5C,D illustrate the rate of decline of sign- and goal-tracking responses for both the lever and the tone over the 16 blocks of training (binned in four-trial blocks) for each stimulus from the first session of omission and extinction, respectively. Consistent with the above results, NLME modeling indicated that the rate of response decline (b) was higher for goal-tracking to the tone, relative to sign tracking to the lever for the omission group [F_(1,67) = 5.47, P < 0.05] and the extinction group [F_(1,81) = 12.05, P < 0.05]. Thus, even within the first session of omission and extinction, goal-tracking to the tone was significantly more sensitive to the change in contingency than sign-tracking to the lever.

4-CS PCA discrimination

Figure 6A illustrates the acquisition of sign-tracking to the lever that predicted food (Lever+) and the lever that did not predict food (Lever−). As above, because there are no sign-tracking responses to the tone, sign-tracking rates were only analyzed for the lever stimuli (Lever+ and Lever−). Linear mixed-effects modeling revealed a main effect of stimulus [F_(1,5) = 24.21, P < 0.05] and a significant stimulus × session interaction [F_(1,5) = 7.17, P < 0.05], indicating that sign-tracking rates were generally higher for the Lever+ stimulus and that these rates increased for Lever+ and decreased for Lever− over training sessions. Additionally, Figure 6B illustrates the acquisition of goal-tracking response rates to both the Lever+, Lever−, Tone+, and Tone− stimuli over the course of the 28-d training period. Linear mixed-effects modeling revealed a significant main effect of stimulus [F_(3,15) = 54.85, P < 0.05] and a significant stimulus × session interaction [F_(3,15) = 9.14, P < 0.05], suggesting that goal-tracking rates were generally higher for the Tone+ and that goal-tracking rates decreased for all other stimuli over training sessions. Figure 6C illustrates the changes in the average probability difference score over the course of the 28-d training period for all stimuli. Linear mixed effects modeling indicated a significant main effect of stimulus [F_(3,15) = 678.82, P < 0.05], suggesting that the probability difference scores diverged for all conditioned stimuli. Linear mixed effects modeling also indicated a significant training stimulus × session interaction [F_(3,15) = 10.15, P < 0.05], indicating that the trajectory for changes in probability difference scores were significantly different for the lever and tone conditioned stimuli, with Lever+ approaching 1.0, Tone+ approach −1.0, and Lever− and Tone− both approaching 0. Finally, Figure 6D illustrates the changes in average lever and tone discrimination ratio (sign-tracking or goal-tracking response rate to Lever+ or Tone+ divided by sign-tracking or goal-tracking response rate to Lever+ or Tone+ plus sign-tracking or goal-tracking response rate to Lever− and Tone−). Linear mixed effects modeling revealed only a significant main effect of session on discrimination ratio [F_(1,5) = 86.58, P < 0.05], suggesting that the Lever+/Lever− and the Tone+/Tone− discriminations were acquired at equal rates over the 28 training sessions.

Figure 6.

The dependence of sign- and goal-tracking on a stimulus–reward relationship during 4-CS PCA discrimination training (A) Mean (±SEM) response rate (responses/second; r/sec) for sign-tracking (ST), to a lever paired with food (Lever+) and a lever not paired with food (Lever−) (B) Mean (±SEM) response rate (responses/second; r/sec) for goal-tracking (GT), to a Lever+, a Lever−, a tone paired with food (Tone+) and a tone not paired with food (Tone−). (C) Mean (±SEM) difference in response probability (ST probability – GT probability) for Lever+, Lever−, Tone+, and Tone−. 1.00 indicates exclusive sign-tracking and −1.00 indicates exclusive goal-tracking. (D) Mean (±SEM) discrimination ratio values (sign-tracking or goal-tracking responding to Lever+ or Tone+ divided by responding to Lever+ or Tone+ plus responding to Lever− or Tone−) for lever and tone response rates as a function of 4-CS PCA discrimination training session.

Discussion

The results reported here reveal a number of novel aspects of the attribution of incentive salience or value to reward-associated stimuli. First, localized, visual aspects of stimuli close to the food receptacle are necessary but not sufficient to elicit sign-tracking behavior, as a lever and nosepoke were located in the identical position, but the lever elicited predominant sign-tracking and the nosepoke elicited predominant goal-tracking. Second, consistent with the incentive salience hypothesis (Robinson and Berridge 2008), different types of stimuli that are equally effective predictors of the same reinforcer (acting as equivalently effective conditioned stimuli) can be differentially attributed with incentive value; the lever and the tone used here in the 2-CS PCA task each effectively and preferentially elicited their associated response types (sign-tracking to the lever and goal-tracking to the tone), but the lever served as a more robust conditioned reinforcer, indicating it was attributed with greater incentive value than the tone. Third, also consistent with the incentive salience hypothesis, stimuli attributed with incentive value are preferred over stimuli that are just as effective conditioned stimuli but not attributed with incentive value; furthermore, stimuli attributed with incentive value can promote suboptimal choice, leading to a significant loss in primary reinforcement. Fourth, sign-tracking responses associated with incentive salience attribution are more persistent, as these responses are less sensitive to changes in the both the response–reinforcer relation (omission) and the stimulus–reinforcer relation (extinction). Finally, the fact that these different functional relations were measured within-subjects suggests that sign- and goal-tracking are mediated by independent but parallel neurobehavioral processes that are cue-dependent; thus, an individual can attribute incentive salience/value to certain reward-associated stimuli and not others, even though all stimuli may be equally predictive of reward.

Like others (Holland 1977; Cleland and Davey 1983; Holland et al. 2014; Meyer et al. 2014) the present results demonstrate clearly that different conditioned stimuli can preferentially elicit sign- and goal-tracking, and these stimuli can be differentially attributed with incentive salience or value. However, the current within-subject method is a departure from the predominant method used to study sign- and goal-tracking behavior in the past. To date, research on the different neurobehavioral mechanisms that underlie value attribution to reward-related stimuli in sign- and goal-tracking behavior has relied almost exclusively on the behavioral screening of large groups of animals using a single-lever PCA task (Robinson et al. 2014). More specifically, large groups of animals are trained on the single-lever PCA task for a short duration (e.g., 5–7 d). Based on a PCA performance index, animals exhibiting predominant sign-tracking responses over the training period are added to the “sign-trackers” cohort; animals exhibiting predominant goal-tracking behavior over the training period are added to the “goal-trackers” cohort; and animals that exhibit both sign- and goal-tracking behavior over the training period are added to the “intermediate” cohort (Meyer et al. 2012). The cohort approach has provided a myriad of important details regarding the neurobehavioral mechanisms that underlie value attribution to reward-related stimuli and has served well as a model of individual differences in substance abuse vulnerability (Saunders et al. 2013; Robinson et al. 2014); however, it is not without difficulties. For example, this approach requires the use of large animal numbers to isolate the cohorts associated with sign- and goal-tracking behavior. Additionally, response distributions on the single-lever PCA task are often not uniform (with a heavy bias toward sign-tracking responses; cf. Holland et al. 2014), an issue that may be partially related to differences in animal vendors or even breeding colonies within vendors (Fitzpatrick et al. 2013). Furthermore, because identification of the response cohorts is based on PCA performance, study of the processes involved in the acquisition of sign- and goal-tracking behavior is problematic; although the use of a genetic model has been used for the study of sign- and goal-tracking response acquisition (Flagel et al. 2011), selective breeding comes with its own set of problems (e.g., inadvertent neurobehavioral changes not intentionally selected). Perhaps more importantly, because the different systems thought to govern sign- and goal-tracking are thought to be independent but parallel processes within individuals (Clark et al. 2012), isolating these systems across individuals makes investigation into the arbitration process that may determine the predominance of one system over the other within an individual more difficult and resource-intensive, while also complicating investigation into the acquisition of stimulus–reward learning system biases modulated via the arbitration process.

Necessarily, the results we obtained with the within-subject 2-CS PCA procedure mirror what is known regarding incentive salience attribution using the cohort model described above. For example, just as we demonstrated here that a lever associated with sign-tracking serves as a more robust conditioned reinforcer (relative to a stimulus, tone, associated with goal-tracking) with both a conditioned reinforcement test and a novel choice procedure, Robinson and Flagel (2009) demonstrated that a lever CS was a more robust conditioned reinforcer in animals identified as sign-trackers, relative to animals that goal-tracked to the lever. Furthermore, Meyer et al. (2014) demonstrated that a reward-predictive tone that elicits goal-tracking behavior promotes the same conditioned reinforcement in animals prescreened as sign-trackers or goal-trackers. Additionally, Meyer et al. (2014) demonstrated that the conditioned reinforcement maintained by a tone in sign- and goal-trackers was equivalent to the conditioned reinforcement maintained by a lever in animals prescreened as goal-trackers, indicating that stimuli associated with goal-tracking are not as robust conditioned reinforcers as stimuli associated with sign-tracking, regardless of cohort phenotype. Finally, the results of the present 4-CS PCA procedure parallel those using unpaired controls in cohort studies, indicating that both sign- and goal-tracking responses are dependent on stimulus–reward learning. Overall, the overlap between the results demonstrated here with the within-subject design and those from the between-subject cohort studies suggest that the two methods are accessing the same functional relationships.

In addition to demonstrating a functional overlap with the between-subject cohort method, the 2-CS PCA task may offer some advantages in the study of the neurobehavioral processes associated with incentive value attribution. First, because the 2-CS procedure uses two stimuli equally predictive of reward, one associated with incentive value and the other not, it allows for the dissociation of incentive value attribution to reward-associated stimuli from more general associative learning processes. That is, manipulations thought to affect only incentive value attribution processes should only affect responding to a stimulus associated with incentive value attribution, leaving responding to the other reward-associated stimulus less or unaffected. Second, the procedure allows for the study of acquisition of incentive value attribution to reward-associated stimuli, while the additional stimulus (without incentive value attribution) again serves as a control for more general associative learning processes. Unlike the cohort model where animals are identified as sign- or goal-trackers following acquisition, the present procedure allows for the investigation of the specific processes involved in the attribution of incentive value, while controlling for more general stimulus–reward learning via the accompanying stimulus. Third, use of the choice procedure herein offers a novel method for studying conditioned reinforcement (without relying on extinction conditions) and how reward-related stimuli contribute to suboptimal choice behavior, as choices for the lever stimulus are inversely related to the number of primary reinforcers earned. Finally, the choice procedure offers a method to study the arbitration process thought to underlie differential attribution of incentive value to reward-related stimuli (Clark et al. 2012); using the present choice procedure to simultaneously engage and pit the different neurobehavioral systems that underlie sign- and goal-tracking behavior against one another will allow for study of the relative propensity of these different neurobehavioral systems to dominate stimulus–reward learning. Furthermore, the choice procedure used here produced individual differences in the relative value of a reward-associated stimulus to a primary food reinforcer; indeed; the rate of sign-tracking to the lever during the initial 14-d acquisition period was significantly correlated with subsequent choices for the lever (r = 0.33, P < 0.01). Thus, using the choice procedure described herein to place the incentive value of a reward-associated stimulus in direct competition with a primary reinforcer allows the experimenter to investigate the individual differences in the relative magnitude of value attribution to a reward-associated stimulus and the relative vulnerability to stimulus-driven suboptimal choice.

One limitation to the present results is the lack of video recording to capture conditioned responses in addition to those measured by lever pressing and magazine entries. For example, others that have measured multiple response types to the tone with video, including rearing/orienting and head-jerk responses, but these studies have repeatedly demonstrated that a tone CS preferentially elicits goal-tracking behavior, regardless of the relative location of the tone source to the food magazine (Holland 1977; Harrison 1979; Cleland and Davey 1983; Holland et al. 2014; Meyer et al. 2014); moreover this goal-tracking to a tone is not due to an inability of animals to localize the tone source, as sign-tracking responses to a tone are subject to reinforcement (Harrison 1979; Cleland and Davey 1983). Furthermore, under the present conditions, neither localizability nor stimulus modality of the reward-associated stimulus alone appear to be defining features necessary for sign-tracking, as a nosepoke stimulus that shared the visual modality and location of the lever produced predominant goal-tracking behavior (see Fig. 1). However, like the tone, the absence of video recording also precluded our ability to measure multiple conditioned responses to the nosepoke CS. In addition to goal-tracking responses, rearing/orienting responses to light stimuli have also been reported in the literature (Holland 1977, 1979; Olshavsky et al. 2013, 2014). Future research should attempt to measure multiple response types with video during 2-CS PCA, as these stimulus-dependent response topographies may be relevant to the ability of the stimulus to acquire incentive value. Additionally, measuring multiple response types may help to better parse out mechanisms underlying treatment effects, as in the mixture of both sign- and goal-tracking to a nosepoke with repeated nicotine treatment (Dion et al. 2011).

Another potential drawback of the present approach is the seeming lack of individual differences observed in sign-tracking to a lever. As stated above, much of the previous work on the differential neurobehavioral mechanisms involved in sign- and goal-tracking behavior has utilized different cohorts that demonstrate sign- or goal-tracking behavior, and this work has been carried out primarily in relation to individual differences in substance abuse vulnerability, where, in general, individuals that are more likely to sign-track are also those that are more susceptible to the rewarding effects of drugs of abuse (Tomie et al. 2008; Beckmann et al. 2011; Robinson et al. 2014). As with the cohort approach, all animals tested herein were exposed to PCA training with a lever, leading to the question of why there was so little goal-tracking observed to the lever in the present experiments. Others (Boakes 1977; Davey et al. 1982; Chang et al. 2012; Chang and Holland 2013; Chang 2014; Holland et al. 2014) have observed a similar bias toward sign-tracking to a lever as that observed herein when training took place in conjunction with an additional control lever that was never followed by food (Lever−); consistent with this hypothesis, both sign-tracking rates to the lever and goal-tracking rates to the tone were higher in the 4-CS PCA experiment, where both stimuli were trained with a never-reinforced counterpart (Lever− and Tone−). Additionally, Other than the single between-subject experiment presented herein (although no goal-trackers were identified in that experiment either), all animals were trained with at least one additional stimulus, leaving open the possibility that including any element of discrimination in training may promote sign-tracking to the food-associated lever (cf. Holland et al. 2014). However, unlike the experiments reported herein and those that utilize the cohort approach where animals were on an ad libitum diet throughout experimentation, each of these previous examples used food-restricted animals during experimentation, and variation in satiety is known to modulate sign-tracking, with greater sign-tracking observed under food-restricted conditions (Cleland and Davey 1983). Furthermore, the homogeneity of the sample used here could also be a contributing factor; although we know not in what colony rooms our animals were bred, they were all from Harlan Laboratories (Indianapolis, IN). As stated in the introduction, Fitzpatrick et al. (2013) observed variation in the uniformity of sign-tracking behavior to a lever between cohorts of rats acquired from Harlan Laboratories and Charles River Laboratories (Wilmington, MA), where rats from Harlan Laboratories showed a sign-tracking bias that was modulated by breeding colony room. Finally, an additional factor that could be of interest is the type of lever stimulus used. In the present experiments we used a standard Med Associates retractable lever, while much of the previous cohort-based work has been done with a Med Associates retractable lever that is illuminated from behind. Given the present result that an illuminated nosepoke can elicit predominant goal-tracking, perhaps the illuminated lever used in the cohort designs is picking up differential selective associations made between the different aspects of the illuminated lever (e.g., haptic versus visual) and food, leading to a differential expression of sign- and goal-tracking behavior across individuals. Although the seemingly state-dependent nature of sign- and goal-tracking argues against incentive salience attribution as a trait variable, clearly more research is needed to help parse out these issues.

Because the sign- and goal-tracking measured here was associated with different stimulus types, one potential caveat to the present approach could be that different preexisting attributes (perceptual, associability, motivational, etc.) of the two stimuli are responsible for the subsequent response and value differences reported herein. However, there are results in the literature and in the present report that argue against this possibility. First, although it is possible that the multi-modal (auditory, visual, and haptic) nature of the lever stimulus associated with sign-tracking could produce greater preexisting value relative to the tone, there were no differences between the lever and the tone in their ability to maintain novel operant responding when each stimulus was tested prior to conditioning with food (see Fig. 3). Second, it is also possible that the multimodal nature of the lever could be more salient and could have created an associability advantage (cf. Pearce and Hall 1982) for the lever, making it a “better” conditioned stimulus and thus more valuable and persistent in all subsequent testing, but we found no evidence to suggest this was the case; in all experiments, including the 4-CS PCA experiment (see Fig. 6), similar to what Holland et al. (2014) reported, there was no evidence of significantly speeded response acquisition to the lever, as would be the primary prediction of the differential preexisting salience or associability hypothesis (Rescorla and Wagner 1972; Mackintosh 1975; Pearce and Hall 1980, 1982). Collectively, the results herein indicate that both the lever and tone were equally effective conditioned stimuli but differed significantly in their ability to accrue value during conditioning, a difference that does not appear to be related to preexisting differences in salience or associability.

Though there is no evidence that the lever and tone are inherently different in regard to their ability to serve as conditioned stimuli, there were clear differences in their incentive value post-conditioning. More specifically, even though each stimulus-food pairing was learned equally well and at equal rates, the large discrepancies in stimulus value, preference, and response persistence between the lever and tone presented here suggest that the two stimuli engage different types or modes of learning. From a more ecological perspective, as proposed by Timberlake's behavior systems theory (Timberlake et al. 1982; Timberlake 1994), the greater value, preference, and behavioral persistence maintained by the lever, relative to the tone, might suggest that the two stimuli engage different appetitive systems related to gathering food. Following Timberlake (1994), the sign-tracking responses toward the lever observed here are in form with an appetitive mode similar to focal search (with a possible slight generalization into the handle/consume mode), resulting in grabbing, biting, gnawing, and licking the lever, while goal-tracking during the tone may be more representative of general search (with a possible generalization into the focal search mode). Thus, as a stimulus preferentially engages appetitive systems that are closer to the consumatory system, it might be attributed greater value. However, there is evidence that goal-tracking shares some topographical similarity (e.g., sniffing and nibbling) with sign-tracking to a lever, although directed toward the magazine (e.g., Mahler and Berridge 2009). Thus, at least within the present paradigm, topography of the response types themselves may not be very telling of the functional differences observed between them.

Timberlake (1994) also suggested that utilizing various Pavlovian and instrumental methods could help to parse out the functional characteristics of each appetitive module. For example, omission schedules have long been used to dissociate stimulus–response (S–R) or Pavlovian relationships from action–outcome (A–O) or instrumental relationships (Schwartz and Williams 1972; Timberlake et al. 1982; Dayan et al. 2006; Yin and Knowlton 2006), and they have even been applied to PCA procedures using a lever conditioned stimulus in rats for this purpose (e.g., Stiers and Silberberg 1974). In general, responding that persists on an omission schedule suggests insensitivity to subsequent outcomes, implicating an S–R relationship (responses continue to be elicited despite their negative effect on subsequent food delivery), while responding that decreases under omission schedules suggests sensitivity to the change in the response–reinforcer contingency indicative of an A–O relationship (Dayan et al. 2006; Yin and Knowlton 2006). Consistent with this literature, using a reinforcement learning framework (Sutton and Barto 1998), Dayan et al. (2006) formally modeled responding under the PCA omission contingency by modulating the relative influence of Pavlovian value (ω) on the instrumental advantage of the two response alternatives under the omission contingency, namely respond (R) and not respond (NR); ω within in this framework can be interpreted as a quantification of the arbitration process alluded to above. Under the omission schedule, NR is the optimal response alternative, and within the Dayan et al. (2006) framework, the propensity to choose the optimal option is gauged by the relative Pavlovian value associated with the R alternative. In other words, the Dayan et al. (2006) framework assumes that the R option is the Pavlovian option and the value of the Pavlovian R option over the optimal instrumental NR option is directly reflected in the value of ω, where larger values of ω are associated with greater Pavlovian influence and results in persistent responding during omission training; thus, the model suggests that persistence under PCA omission schedules is a reflection of direct competition between Pavlovian (S–R) and instrumental (A–O) learning processes. Following the framework laid out by Dayan et al. (2006), the large differences in the rate of response decay for sign-tracking to the lever and goal-tracking to the tone under the omission schedule studied herein suggests that these two stimuli are preferentially tapping into Pavlovian (sign-tracking to lever) versus instrumental (goal-tracking to tone) processes, and the rate of response decay during omission training and the discounting rate of lever choices studied herein can be interpreted as empirical quantitative measures of ω, where slower decay and discounting rates are indicative of greater ω values. From this point of view, individual differences in rate of response decay under omission schedules and rate of discounting during lever choice express the relative dominance of S–R learning over A–O learning. Indeed, this approach has recently been extended to individual differences in PCA performance using the cohort method, where sign-trackers have larger ω values and goal-trackers have lower ω values, leading to behavior governed by model-free and model-based systems, respectively (Lesaint et al. 2014). Collectively, the data suggest that sign-tracking in response to a lever is controlled by Pavlovian S–R relationships, while goal-tracking in response to a tone is controlled by instrumental A–O relationships, making the function of the tone in reference to goal-tracking more akin to a discriminative stimulus that modulates A–O contingencies, rather than a Pavlovian CS. Interestingly, the notion that goal-tracking may be a reflection of A–O contingencies that are modulated by discriminative stimuli is in accord with recent results demonstrating that goal-trackers are more sensitive to reinstatement of cocaine seeking by noncontingent discriminative contextual cues (Saunders et al. 2014). Overall, future use of the omission schedule and lever choice paradigm described herein might serve as a good empirical test bed to study the assumptions of these models and may help in empirically isolating the neurobehavioral mechanisms that underlie individual differences in PCA performance, including the cohort model.

In conclusion, the results reported here offer a novel method to investigate the neurobehavioral processes that underlie sign-tracking, goal-tracking, and the differential incentive value attribution to stimuli associated with each response type. Importantly, the results obtained here using the within-subject model coincide with those of previous research using the between-subject cohort approach, while also offering some novel insight into the processes involved in incentive salience attribution to reward-associated stimuli. Future use of these novel methods may aid in isolating and dissociating the neurobehavioral mechanisms that underlie incentive value attribution to reward-associated stimuli from those that govern more general stimulus–reward learning. Finally, using the present method to measure the balance of the neurobehavioral systems associated with sign- and goal-tracking behavior within an individual may provide insight into how individual differences in this arbitration process contribute to substance abuse vulnerability.

Materials and Methods

Animals

Eighty-eight adult male Sprague-Dawley rats (Harlan Inc.; Indianapolis, IN, USA), weighing ∼250–275 g at the beginning of experimentation, were used. Rats were individually housed in a temperature-controlled environment with a 12:12 h light:dark cycle, with lights on at 0600 h. All rats were first acclimated to the colony environment and handled daily for 1 wk prior to experimentation. All experimentation was conducted during the light phase. All rats had ad libitum access to food and water in their home cage throughout experimentation. All experimental protocols were conducted according to the 2010 NIH Guide for the Care and Use of Laboratory Animals (8th edition) and were approved by the Institutional Animal Care and Use Committee at the University of Kentucky.

Apparatus

Experiments were conducted in operant conditioning chambers (ENV-008, MED Associates) that were enclosed within sound-attenuating compartments (ENV-018M, MED Associates). Each chamber was connected to a personal computer interface (SG-502, MED Associates), and all chambers were operated using MED-PC. Within each operant chamber, a 5.1 × 5.1 cm recessed food receptacle (ENV-200R2MA) outfitted with a head-entry detector (ENV-254-CB) was located on the front response panel of the chamber, two retractable response levers were mounted on either side of the food receptacle (ENV-122CM; 6 cm above metal rod floor), two white cue lights (ENV-221M) were mounted at 4.1 and 8.2 cm above each response lever, and a Sonalert tone (ENV-223 AM) was located above the top left cue light and a Sonalert tone (ENV-223 HAM) was located above the top right cue light. The back response panel was outfitted with a single retractable response lever (ENV-122CM; located directly opposite of the food receptacle); two nosepoke response receptacles (ENV-114BM; 6 cm above metal rod floor and directly opposite to front response levers) were mounted on either side of the retractable response lever, and a house-light (ENV-227M) was located 12 cm above the response lever. However, during nosepoke CS PCA training, the nosepoke response receptacles were located in place of the response levers on the front panel (6 cm above metal rod floor). Food pellets (45-mg Noyes Precision Pellets; Research Diets, Inc.) were delivered via a dispenser (ENV-203M-45).

Procedure

Lever, nosepoke, or tone CS PCA

Animals were first trained to retrieve food pellets from the food receptacle for two consecutive days; rats were placed in the operant chamber and given 40 min to retrieve and consume 16 food pellets, delivered on a fixed time 60-sec schedule. Following magazine training, three groups of animals (lever group; tone group; and nosepoke group; n = 6 per group) were trained on a PCA task using a single food-predictive CS. During training sessions for the lever group, a single response lever (counterbalanced for side) adjacent to the food receptacle was inserted into the chamber for 8 sec, followed by lever retraction and noncontingent food delivery into the food receptacle. During training sessions for the nosepoke group, a single nosepoke apparatus (counterbalanced for side) adjacent to the food receptacle (same relative location as lever location in lever group) was lit for 8 sec, followed by nosepoke offset and noncontingent food delivery into the food receptacle. During training sessions for the tone group, a single tone was turned on for 8 sec, followed by tone offset and noncontingent food delivery into the food receptacle. Stimulus-presentation trials in all groups were separated by a 90-sec variable time (VT) intertrial interval (ITI) that began immediately after pellet delivery. For all groups, each training session consisted of 25 trials, and all groups were trained for a total of 10 sessions. Each session lasted ∼40 min. Sign-tracking (ST) responses were recorded as lever presses, or nosepokes into the nosepoke apparatus during 8-sec trials, while goal-tracking (GT) responses were recorded as breaks of a photo beam within the food receptacle during stimulus (lever, nosepoke tone) presentation.

2-CS PCA

Given that the nosepoke and the lever were the only manipulanda available within the operant chambers, use of either of these for subsequent operant responding would complicate interpretation during conditioned reinforcement testing. Furthermore, the demonstration that nosepokes and tones elicit goal-tracking equivalently allowed the use of a tone to elicit goal-tracking during training, instead of a nosepoke. Thus, we trained a group of animals (n = 12) on a novel 2-CS PCA task that utilized a tone CS to elicit goal-tracking and a lever to elicit sign-tracking within a single individual. Magazine shaping was identical to the procedure described above. During each training session, either a single response lever adjacent to the food receptacle (counterbalanced for side) was inserted into the chamber for 8 sec or a tone was presented for 8 sec. Immediately after lever retraction or tone offset, a food pellet was noncontingently delivered into the receptacle. Each session consisted of 32 total trials, comprised 16 lever−insertion trials and 16 tone−presentations trials presented in a pseudorandom order, where no more than four presentations of the same stimulus could occur sequentially. A 90-sec VT ITI separated stimulus presentations. Each session lasted for ∼50 min. Because this procedure conditioned two stimuli within the same animal, training was extended to 14 sessions for all animals. ST responses were recorded as lever presses, while GT responses were recorded as breaks of a photo beam within the food receptacle during stimulus (lever or tone) presentation. Head entries into the food receptacle during the 8-sec period preceding stimulus presentation (8-sec pre-CS) were also recorded.

2-CS conditioned reinforcement test

To determine the relative value of a lever associated with sign tracking and a tone associated with goal-tracking, we initially trained a group of animals (n = 12) on the 2-CS PCA task (described above) for 14 d, followed by a conditioned reinforcement test over the next two subsequent days. The conditioned reinforcement test took place over two 30-min sessions during the 2 d that immediately followed the 2-CS PCA training described above. At the start of the first session (day 15), animals were presented with an illuminated nosepoke (counterbalanced for side), where a single response (break in photobeam within the nosepoke receptacle) resulted in the simultaneous offset of the nosepoke light and 8-sec presentation of either the lever or tone (counterbalanced across animals) that was used during initial 2-CS PCA training. After the 8-sec stimulus presentation (lever or tone), the nosepoke light turned on again. On the subsequent day (day 16), the opposite nosepoke light was illuminated, and a response produced either the lever or tone for 8 sec, opposite to what stimulus was presented during conditioned reinforcement test day 1. Nosepoke responses to the nonilluminated, inactive nosepoke were recorded during each test but had no consequence.

To determine the relative value of a lever and tone potentially associated with differential preexisting stimulus salience, we tested a separate group of rats (n = 6) on the conditioned reinforcement test prior to and following 14 sessions of 2-CS PCA training. The conditioned reinforcement tests took place over two 30-min sessions as described above, except two test sessions took place prior to 2-CS PCA training and two test sessions took place immediately following 2-CS PCA training; thus, any differences noted between the lever and tone during the tests prior to 2-CS PCA training would be indicative of some preexisting difference in stimulus salience between the lever and tone, and differences between the stimuli noted following training would be indicative of the relative conditioned reinforcement of each stimulus.

2-CS choice

To determine the relative preference for a lever associated with sign-tracking over a tone associated with goal-tracking, a group of animals (n = 12) was initially trained on the 2-CS PCA task (described above) for 14 d. Subsequently, the group was trained on a stimulus-choice procedure. The procedure was adapted from the probability discounting procedure reported in Yates et al. (2014), and the order of operations is described in Figure 7. Specifically, the choice procedure consisted of five blocks of 13 trials, separated by a dark 60-sec intertrial interval (ITI). Each trial began via illumination of the house-light, where a single head-entry orienting response into the magazine resulted in the offset of the house-light and illuminated a nosepoke (left, right, or both, depending on trial type) on the rear. During forced-choice trials (8/block; four lever and four tone), only a single option (either the left or right nosepoke was illuminated) was available, where a single response resulted in the previously conditioned 8-sec lever or tone stimulus (counterbalanced for nosepoke side), followed by a single pellet and initiation of the ITI. During free-choice trials (5/block), both options were available (both the left and right nosepokes were illuminated) and a single response to either nosepoke resulted in either the previously conditioned 8-sec lever or tone associated with its respective nosepoke side, a single food pellet, and initiation of the ITI. All trial types operated according to a 30-sec limited hold; if a nosepoke response did not occur within 30-sec from the onset of the nosepoke light, the trial was aborted, an omission was counted, and the ITI began. Following stability on the choice procedure, where both stimuli were always followed by food over the five 13-trial blocks, training then included a decrease in food probability (p) following the lever (100%, 50%, 25%, 12.5%, and 6.25%) in descending order over the five 12-trial blocks. Finally, following stable performance (no statistically significant changes in the discounting parameters discussed in the analysis section below across 3 d) on the probabilistic choice task with both stimuli present, either the lever or the tone was removed from the 8-sec period that preceded food delivery to determine the relative value of each stimulus; these stimulus-removal tests lasted seven sessions each.

Figure 7.

Diagram of the choice procedure, following 2-CS PCA training. The initial link began with the onset of the house-light. Following the onset of the house-light, an orienting response into the food magazine turned off the house-light and illuminated either the right, left, or both nosepoke lights, depending on trial type (Forced versus Choice trials). A response into an illuminated nosepoke produced the previously conditioned lever CS or tone CS. Failure to respond within 30 sec of house-light illumination was counted as an omission and led to the initiation of the intertrial interval (ITI).

2-CS PCA task omission

To determine the relative sensitivity to changes in response–food contingency for each stimulus, a group of animals (n = 10) was initially trained on the 2-CS PCA task (described above) for 14 d. Immediately following 2-CS PCA training, animals underwent omission training for an additional 10 d. During omission training, the 2-CS PCA task proceeded as described previously, except no food followed any stimulus presentation that included any response (sign- or goal-tracking) to the lever or tone. Thus, food only occurred if the animals withheld a response to either stimulus.

2-CS PCA task extinction

To determine the relative sensitivity to changes in stimulus–food contingency for each stimulus (lever versus tone), a group of animals (n = 12) was initially trained on the 2-CS PCA task (described above) for 14 d. Immediately following 2-CS PCA training, animals underwent extinction training for an additional 10 d, where the 2-CS PCA task proceeded as described above, except no food followed any stimulus presentations.

4-CS PCA discrimination

To demonstrate the dependence of sign-tracking to the lever and goal-tracking to the tone on a stimulus–food relationship during acquisition, we trained a group of animals (n = 6) on a 4-CS PCA discrimination task. The 4-CS PCA discrimination task was similar to the 2-CS PCA task as described above, except two additional stimuli were included in each training session. One 8-sec lever presentation (left or right, counterbalanced for side) was always followed by food (Lever+), while the other was never followed by food (Lever−). Additionally, one 8-sec tone presentation (2900 or 4500 Hz, counterbalanced for frequency) was always followed by food (Tone+), while the other was never followed by food (Tone−). Because the 4-CS PCA task required two simultaneous discriminations, we anticipated slower acquisition; thus, training consisted of a 28-d period, instead of the 14-d period used for 2-CS PCA.

Analysis

All data were analyzed using linear mixed-effects modeling (Gelman and Hill 2006). Lever, nosepoke, or tone CS PCA data were analyzed with session as a continuous, within-subject factor, group as a between-subject factor, and subject as a random variable. Because there were no sign-tracking responses to a tone, all sign-tracking response rate data during 2-CS PCA training were analyzed with session as a single continuous, within-subject factor and subject as a random variable, while the 4-CS PCA discrimination training analysis included stimulus (Lever+, Lever−, Tone+, and Tone−) as an additional within-subject factor. All goal-tracking response rate data, conditioned reinforcement data (number of responses), discrimination ratio data (sign-tracking or goal-tracking response rate on Lever+ or Tone+/(sign-tracking or goal-tracking response rate to Lever+ or Tone+ plus sign-tracking or goal-tracking response rate to Lever− and Tone−), and probability difference data (probability of making a sign-tracking response − probability of making a goal-tracking response) were analyzed with session as a continuous within-subject factor, stimulus (lever and tone for 2-CS PCA training and Lever+, Tone+, Lever−, and Tone− for 4-CS PCA discrimination training) as a nominal within-subject factor, and subject as a random variable. The average number of responses from the conditioned reinforcement tests was analyzed using two-way mixed-effects ANOVA, with stimulus (lever versus tone) and response type (active versus inactive) as nominal within-subject factors and subject as a random variable. The data from conditioned reinforcement tests pre- and post-2-CS PCA conditioning was analyzed using three-way mixed-effects ANOVA, with stimulus (lever versus tone), response type (active versus inactive), and session (pre- versus post-conditioning) as nominal within-subject factors and subject as a random factor. All post hoc tests were conducted with Tukey HSD.

For the 2-CS choice data, hyperbolic discount functions (Rachlin et al. 1986, 1991; Mazur 1987) of the form

$$V_{\text{Lever}}=\frac{s}{1+(k\times \text{O})}$$

were fit to the data, where V_Lever represents subjective value of the lever CS, s represents the sensitivity to stimulus value (lever versus tone) at equal food probability, k represents the discounting rate of lever value, and O represents odds against [(1 − p)/p]. For the omission data and the extinction data, exponential decay functions (Nevin and Grace 2000) of the form

$$R=a\times e^{-b\times s}$$

were fit to the data, where R represents response rate, a represents baseline response rate, b represents the rate of response decay, and s represents omission or extinction training session.

Each model (hyperbolic and exponential decay) was fit to the data via nonlinear mixed effects modeling (NLME; Beckmann and Young 2009; Young et al. 2009; Brooks et al. 2010) using the NLME package in the R statistical software (Pinheiro et al. 2007), with session defined as a continuous, fixed within-subject factor, stimulus condition (lever and tone, lever only, and tone only) defined as a nominal, fixed within-subject factor, and subject defined as a random factor. Briefly, like linear mixed effects modeling, NLME is a multilevel, hierarchical modeling technique that relies on maximum likelihood estimation (Myung 2003) to produce parameter estimates for a predefined function fit to data from various conditions within an experiment (using model fits from all individuals), while providing statistical tests for goodness of fit and whether or not parameter estimates are significantly different from one another. Unlike traditional repeated-measures ANOVA techniques, linear mixed effects modeling and NLME significantly increase power, reduce Type I error rates, and by using empirically defined functions, NLME aids in interpretation by bringing the researcher closer to the underlying relationship in the data (Young et al. 2009). For all tests, α was set at 0.05.