The sensory features of a food cue influence its ability to act as an incentive stimulus and evoke dopamine release in the nucleus accumbens core

Abstract

The sensory properties of a reward-paired cue (a conditioned stimulus; CS) may impact the motivational value attributed to the cue, and in turn influence the form of the conditioned response (CR) that develops. A cue with multiple sensory qualities, such as a moving lever-CS, may activate numerous neural pathways that process auditory and visual information, resulting in CRs that vary both within and between individuals. For example, CRs include approach to the lever-CS itself (rats that “sign-track”; ST), approach to the location of reward delivery (rats that “goal-track”; GT), or an “intermediate” combination of these behaviors. We found that the multimodal sensory features of the lever-CS were important to the development and expression of sign-tracking. When the lever-CS was covered, and thus could only be heard moving, STs not only continued to approach the lever location but also started to approach the food cup during the CS period. While still predictive of reward, the auditory component of the lever-CS was a much weaker conditioned reinforcer than the visible lever-CS. This plasticity in behavioral responding observed in STs closely resembled behaviors normally seen in rats classified as “intermediates.” Furthermore, the ability of both the lever-CS and the reward-delivery to evoke dopamine release in the nucleus accumbens was also altered by covering the lever—dopamine signaling in STs resembled neurotransmission observed in rats that normally only GT. These data suggest that while the visible lever-CS was attractive, wanted, and had incentive value for STs, when presented in isolation, the auditory component of the cue was simply predictive of reward, lacking incentive salience. Therefore, the specific sensory features of cues may differentially contribute to responding and ensure behavioral flexibility.

Learning Pavlovian associations allows for individuals to identify events (conditioned stimuli; CS) that consistently predict delivery of rewards or punishments (unconditioned stimuli; US). It is necessary for such associations to be flexible, as predictive events often change over time, promoting adaptation in the performance of specific responses (Holland 1984). Moreover, these conditioned responses (CRs) are not simply “reflex transfers” to predictive events (Holland 1984). Instead of being exact “copies” of unconditioned responses (URs), CRs may be substantially different from URs, providing the organism with preparatory or compensatory mechanisms in advance of reward or punishment. For example, while a single electric shock increases behavioral activity, during fear conditioning repeated pairings of shocks with a CS result in the development of freezing responses to the cue (Fanselow and Wassum 2015). Thus, as stated by Boakes (1977): “we need not only to understand how the animal learns to associate the two events but also what the behavioral consequences of such learning are.”

It has long been appreciated that Pavlovian learning involves the acquisition of many associations between the CS and the US, which can influence behavior in a variety of different ways (Rescorla 1988; Delamater 2012). Indeed, the form of a CR is dependent not only on the US but also on the qualities of the CS—tones, diffuse lights, and localized visual cues are all capable of evoking unique behavioral responses, despite being predictive of the same food reward in a single test environment (Holland 1977). One explanation for this variability in CRs is that Pavlovian CSs do not simply elicit responses originally evoked by the US, but can generate complex emotional and motivational states that can influence behavior in a variety of ways (Rescorla 1988). It is thus likely that various cues become differentially endowed with conditioned reinforcing, incentive motivational, and approach eliciting properties that serve to direct conditioned responses (Milton and Everitt 2010).

Even when the same CS and US are used, there can be considerable individual variation in the form of the CR that results (Zener 1937; Boakes 1977). For example, in rats, an auditory CS paired with a food reward evokes a “goal-tracking” (GT) response (approach to the location of reward delivery) in all animals (Holland 1977; Farwell and Ayres 1979; Cleland and Davey 1983). In contrast, if a retractable lever or visual stimulus is used as the CS some rats show a GT response, but others come to approach and interact with the CS itself (i.e., they develop a sign-tracking [ST] CR), and yet others vacillate between ST and GT CRs (Zener 1937; Boakes 1977; Jenkins et al. 1978; Flagel et al. 2007; Saunders and Robinson 2012). It is likely that changing the sensory qualities of the cue may alter the performance of sign- and goal-tracking, as the associative structure of the CS–US relationship adjusts (Holland et al. 2014; Meyer et al. 2014).

There are many possible reasons for individual variation in the form of Pavlovian CRs, even under constant training conditions. For example, we have suggested that animals may vary in the extent to which they attribute motivational value (incentive salience) to a lever-CS (Robinson et al. 2014). Relative to GTs, in STs the lever-CS is not only approached but is also a more effective conditioned reinforcer and better at reinstating reward-seeking behavior (Meyer et al. 2012; Saunders et al. 2013). Such variation in incentive salience attribution could potentially result if different animals attend to unique sensory features of the lever-CS. A lever-CS is a complex multimodal cue as its presentation involves movement and generates visual (the lever itself and its illumination), auditory (the sound when extended and retracted, as well as when pressed), and tactile sensory components. This may be important because a number of studies have suggested that auditory stimuli (tones) are attributed with less incentive salience than a lever-CS (Meyer et al. 2014; Beckmann and Chow 2015), even though rats can localize auditory stimuli (Harrison 1979; Cleland and Davey 1983). Thus, if an animal attended primarily to its auditory qualities, a lever-CS may generate a GT CR because of its predictive value, but the auditory component alone may not be sufficient to confer much incentive value to the lever-CS, reducing its potential to act as a potent incentive stimulus. It is possible that the ability of sensory features of the lever-CS to gain incentive value is not so sharply defined and dichotomous. Indeed, the motivational properties of the lever-CS could vary from moment to moment, which might be why some rats vacillate between making a ST versus GT CR (IN rats; Saunders and Robinson 2012). To explore the contribution of the auditory component of the lever-CS to conditioned responding in STs, GTs, and INs, we conducted studies in which the lever-CS was covered, so it could be heard but not seen.

There is considerable evidence that dopamine (DA) neurotransmission, especially in the core sub-region of the nucleus accumbens (NAc), plays an important role in the attribution of incentive salience to cues predictive of reward. For example, both the acquisition and performance of a ST (but not a GT) CR are DA dependent (Danna and Elmer 2010; Flagel et al. 2011; Saunders and Robinson 2012; Chow et al. 2016). We hypothesized, therefore, that if specific sensory features of the lever-CS differentially control behavior, then altering these qualities of the cue may promote flexibility in conditioned behavior and corresponding variation in DA signaling. Thus, we used fast-scan cyclic voltammetry (FSCV) to monitor DA release in the core of the NAc in STs, GTs, and INs and observed how this neurotransmission reflected the form of the CR evoked by a lever-CS that could be seen, heard, and touched, or only heard.

Results

Standard Pavlovian training

In rats, there is individual variability in the form of the CR evoked by a lever-CS paired with food delivery. As described previously (Flagel et al. 2007; Robinson et al. 2014), some rats (STs) increasingly contacted the lever, doing so with reduced latency with training (Fig. 1A,C,E; ST PCA index +1.0 to +0.4, ∼31% of rats trained). In contrast, other rats (GTs) did not approach the lever, but instead showed increased food cup contacts (Fig. 1B,D,F; GTs PCA index −0.4 to −1.0, ∼41% of rats trained). Furthermore, yet other rats displayed an “intermediate” behavioral phenotype (INs, neither ST nor GT, PCA score −0.4 to +0.4, ∼28% of rats trained), demonstrating variable lever- and food cup-directed behaviors across trials. In all conditioned rats, these behaviors were controlled by presentation of the lever-CS, as nonspecific entries into the food cup during inter-trial-intervals (ITIs) significantly decreased across training days (effect of training day, F_(4,328) = 41.14, P < 0.0001). Similar to previous reports (Meyer et al. 2012), when the lever-CS was unpaired from food delivery, rats did not develop conditioned responding (experiment 2, control rats used for tests of conditioned reinforcement).

Figure 1.

Standard Pavlovian training. Lever- (“sign-tracking,” A,C,E) and food-cup-directed (“goal-tracking,” B,D,F) conditioned response behaviors over 5 d of training. Graphs display the probability of approach to the lever or food cup (A,B), the number of contacts with the lever or food cup (C,D), and the latency to approach the lever or food cup (E,F). Means are shown as ±SEM.

The lever-CS used to identify variation in responding is a complex multimodal cue. This CS has several properties (to name a few: visual, auditory, spatial, temporal, tactile, and gustatory) to which rats may attend. Importantly, these specific characteristics of the cue may either individually or in combination serve to motivate a rat’s behavior. The following experiments isolated the auditory component of the lever-CS from other sensory features to identify their contributions to motivated behavior and DA neurotransmission.

Experiment 1: CS properties and the development of conditioned approach

We first sought to determine whether only the sound associated with lever movement could support the development of conditioned responding and, if so, did this stimulus component facilitate either sign- or goal-tracking. Therefore, a subset of rats was trained using our normal conditioning procedures (8 sec lever insertion), except the lever was covered—the lever-CS could be heard extending and retracting, but was not visible (Fig. 2; n = 12). These rats had never before seen the lever-CS; therefore, they had no knowledge of the lever’s existence behind the cover. Thus, rats likely perceived the sounds of lever extension (CS1) and lever retraction (CS2) as distal and proximal cues, respectively, predictive of reward (separated in time by 8 sec of silence). After this initial conditioning (sessions 1–5), the lever cover was removed and rats were retrained for another 5 d (sessions 6–10) with the lever available. This additional training (sessions 6–10) allowed for the characterization of rats as STs (n = 4; ∼33% of rats), GTs (n = 3; 25% of rats), and INs (n = 5; ∼42% of rats) according to their index of conditioned approach.

Figure 2.

The sound of lever extension results in the development of conditioned approach to the food cup in all rats. Rats were trained using standard conditioned procedures (days 1–5), but with the lever covered. During this initial training, all rats developed a conditioned approach response to the food cup during lever extension (A; days 1–5), with their latency to do so decreasing across days (C). Rats were then reconditioned with the lever cover removed (lever visible) for 5 d. With the lever available, rats developed food cup (A) and lever-directed (B) behaviors and were subsequently able to be characterized as STs, GTs, or INs. Accordingly, GTs continued to approach the food cup with short latency (C; days 6–10), while both IN and ST rats developed decreased latencies to approach the lever when it was visible (D). Green-shading and black-filled-in circles denote covered lever sessions. Means are shown as ±SEM.

Using this procedure, we found that all rats similarly developed the expected food cup approach response after hearing the extension of the covered lever, indicating that the lever sound itself supported the development of a GT CR equally well in STs, GTs, and INs (two-way repeated measures ANOVAs). Thus, when the lever was covered during training (sessions 1–5), there were no group differences in food cup contacts (Fig. 2A; training day effect, F_(4,36) = 10.52, P < 0.0001; group effect, F_(2,9) = 0.04, P = 0.96, ns; Training Day × Group Interaction, F_(8,36) = 2.05, P = 0.07, ns), latency to the first food cup contact (Fig. 2C; training day effect, F_(4,36) = 29.65, P < 0.0001; group effect, F_(2,9) = 0.13, P = 0.88, ns; Training Day × Group Interaction, F_(8,36) = 2.73, P = 0.02), or probability of food cup contact (data not shown; training day effect, F_(4,36) = 21.31, P < 0.0001; group effect, F_(2,9) = 0.05, P = 0.95, ns; Training Day × Group Interaction, F_(8,36) = 1.67, P = 0.14, ns). Importantly, there was also no difference in the latency of food cup approach for GTs between when the lever was covered or visible (Fig. 2C; t₍₂₎ = 0.35, P = 0.76; day 5 covered vs. day 10 visible), suggesting that they were not approaching the lever cover before goal-tracking. STs were also not likely approaching the lever location when it was covered, as their latency to respond did not differ between lever-covered and -visible procedures (Fig. 2C,D; t₍₃₎ = 0.95, P = 0.41; day 5 food cup approach vs. day 10 lever approach), as well as because lever-approach only developed in STs after several days of training once the lever was visible (Fig. 2B). Finally, there was only an effect of training day (sessions 1–5) on decreased food cup contacts during inter-trial-intervals (data not shown; training day effect, F_(4,36) = 4.34, P < 0.006; group effect, F_(2,9) = 0.32, P = 0.73, ns; Training Day × Group Interaction, F_(8,36) = 1.72, P = 0.13, ns), indicating that conditioned approach to the food cup was controlled by the predictive value of the lever sound, rather than a nonspecific increase in behavioral activity.

Since the audible cue supported the development of conditioned responding in all rats, we sought to remove this component of the lever-CS to evaluate its contribution to sign- and goal-tracking. We therefore altered an important facet the lever’s stimulus properties—its movement. By “fixing” the lever in its extended position and only having its 8 sec illumination serve as the predictive reward cue, we thereby investigated whether the lever’s visual (illumination and movement) or auditory (sound of extending and retracting) components influenced the development of sign- and goal-tracking. Interestingly, we found that illumination of the lever alone (illuminated from behind, similar to other experiments), without its extension and retraction, was insufficient to support the development of sign-tracking behavior (automated recording of lever contact) compared with normally trained controls that experienced movement of the CS (Fig. 3; n = 32, new set of rats that had no conditioning experience). Instead, under these conditions, all rats learned a goal-tracking CR (F_(4,124) = 41.06, P < 0.0001; one-way repeated measures ANOVA, across days) and consequently reduced inter-trial-interval entries into the food cup as training progressed (F_(4,124) = 11.12, P < 0.0001). There was also no difference in the latency to goal-track between rats trained using the fixed lever-CS and rats identified as GTs using standard retractable lever conditioning procedures (t₍₃₅₎ = 0.83, P = 0.41; day 5 of training; data not shown). The equivalent speeds of responding further suggest that rats did not approach the fixed illuminated lever before initiating a goal-tracking CR. Together, these findings suggest that while the sound of the lever movement was sufficient for the development of conditioning, multimodal sensory features of the cue (lever movement, sound, and illumination combined) needed to be presented in order for the lever-CS to become an attractive “motivational magnet.”

Figure 3.

Lever movement and sound are necessary for the development of sign-tracking. The development of conditioned approach depends upon the predictive value of the sound associated with the lever’s movement. All training occurred over 5 d with 25 conditioning trials/day. (A) Distribution of rat behavioral classifications following standard Pavlovian training (CS consists of 8 sec extension and illumination of the lever; cue is visible, audible, and manipulatable). (B) Distribution of rats (left) when during training the lever was “fixed” in place (CS consists of 8 sec illumination of the lever; cue is visual). Under this manipulation, all rats developed conditioned approach to the food cup (right, blue bars). As training progressed, rats decreased food cup entry during inter-trial-intervals (right, black circles). Data are displayed as entries per second to compare food cup contacts during the CS and inter-trial-intervals on the same scale. (***) P < 0.01–0.0001. Significant decrease in food cup contact during ITIs across training days, (###) P < 0.001. Means are shown as ±SEM.

Experiment 2: CS properties and the expression of conditioned approach

The development of sign-tracking appears to depend on not only seeing but also hearing, the lever’s movement. What remains unclear, however, is whether distinct and separable associative processes are responsible for encoding visual and auditory sensory information that supports the ongoing performance of sign- and goal-tracking. For example, once a rat has developed robust sign-tracking, does altering the component features of the cue impact approach to the lever-CS? To investigate this, we first trained rats (n = 30) using standard conditioning procedures (five daily sessions; reward-paired lever-CS visible and available), but on a sixth session, we covered the lever and assessed conditioned approach to (1) the food cup and (2) the lever cover (a sign-tracking approach CR, which includes both contact with the cover and head jerk movements into close proximity with the cover). Compared with conditioned responding when the lever was visible (training session 5), all rats (STs, INs, and GTs) readily adapted their behavior when the lever was covered, demonstrating flexibility in their approach.

Conditioned approach for ST rats after the lever was covered

Sign-tracking occurs when an animal approaches a CS. Importantly, interaction with a CS (i.e., lever-pressing) is not a requirement for classifying a behavior as sign-tracking (Hearst and Jenkins 1974). Thus, when the lever was visible and available, sign-tracking was measured as lever pressing (contacts). In contrast, once the lever was covered sign-tracking consisted of approach to the lever cover (video analysis). For STs, lever-directed behavior (contacts or approach) was only minimally altered by covering the lever on either the first trial (lever-visible vs. lever-covered: probability, 1, identical; latency t₍₁₀₎ = 1.15, P = 0.28; behaviors/sec, t₍₁₀₎ = 1.42, P = 0.19) or across the entire 25-trial session (lever-visible vs. lever-covered: probability, t₍₁₀₎ = 0.36, P = 0.72; latency, t₍₁₀₎ = 2.41, P = 0.04; behaviors/sec, t₍₁₀₎ = 2.32, P = 0.04). Given the consistency of their behavior, it was not surprising that STs continued to approach the lever more than GTs when the lever was covered (Fig. 4C, across all trials; approaches/sec, F_(2,12) = 4.02, P = 0.04; ST vs. GT, P < 0.05 post hoc Bonferroni). Yet, despite these results, across all trials STs only a trended toward increased probability (F_(2,12) = 3.71, P = 0.06) and decreased latency (F_(2,12) = 3.52, P = 0.06) of lever cover approach relative to IN and GT rats. These later findings suggest that STs may have been slower to interact with the lever cover compared with the visible lever-CS; this slight reduction in sign-tracking corresponded with increased goal-tracking in STs.

Figure 4.

CS properties and the expression of conditioned approach. Conditioned approach (A, probability; B, latency; C, contacts/sec) to either the lever location (upward red/black data points) or food cup (downward blue/black data points) is displayed across 25-trial sessions (bars) or during the first trial of a session (circles, lever approach; squares, food cup approach). Results are compared between the final standard Pavlovian conditioning session (lever-CS visible) and the subsequent session when the lever-CS was covered (not visible; green background shading, black cross-hatched bars and black filled-in circles/squares). Lever approach was defined as either a lever press (when visible) or contact with the lever cover (when not visible). While covering the lever had no effect on the probability (A) of lever (cover) approach in STs, STs decreased their latency (B; 25 trials; (*) P < 0.05) and rate of contact (C; 25 trials; (*) P < 0.05) with the lever(cover)-CS. Importantly, immediately after covering the lever, STs started to approach the food cup with identical probability (A), latency (B), and rate of contact (C) as IN and GT rats. After covering the lever, GTs began to approach the location of the lever-CS, but made contact with the lever cover significantly less than STs (C, 25 trials; (^) P < 0.05). In compensation, GTs reduced their probability of approaching the food cup when the lever was covered (A; 25 trials; (^#) P < 0.05). While IN rats did not significantly alter their lever-approach when the lever was covered, their interaction with the food cup was effected, approaching it quicker (B; 25 trial reduced approach latency; (++) P < 0.01). Means are shown as ±SEM.

After covering the lever, there were no longer significant differences between STs, INs, and GTs in their probability (Fig. 4A; F_(2,27) = 1.24, P = 0.30), latency (Fig. 4B; F_(2,27) = 1.19, P = 0.32), or contacts (Fig. 4C; F_(2,27) = 0.71, P = 0.50) with the food cup across all 25 trials (one-way ANOVAs; bars on Fig. 4 graphs). Remarkably, this effect was observed on the first trial of the covered lever session, indicating that rats were not learning a new conditioned response, but instead were still responding according to their cached representation of the CS properties (Fig. 4A–C; probability, F_(2,27) = 0.51, P = 0.60; latency, F_(2,27) = 0.57, P = 0.57; contacts, F_(2,27) = 2.10, P = 0.14; one-way ANOVAs, circles/squares on graphs to the right of corresponding bars). In combination with their continued approach to the lever-CS, these data suggest that after covering the lever, STs adopted a more “intermediate” behavioral phenotype.

Conditioned approach for IN rats after the lever was covered

While STs became “intermediate like,” the behavior of rats previously classified as INs changed very little when the lever was covered. After covering the lever, sign-tracking for IN rats was unchanged—this was the case for both the first trial (lever visible vs. covered; probability, t₍₁₄₎ = 1.15, P = 0.27; latency, t₍₁₄₎ = 1.28, P = 0.22; behaviors/sec, t₍₁₄₎ = 0.086, P = 0.93) and across the entire 25-trial session (probability, t₍₁₄₎ = 0.63, P = 0.54; latency t₍₁₄₎ = 1.47, P = 0.16; behaviors/sec, t₍₁₄₎ = 0.61, P = 0.55). While no significant changes in food-cup-directed behavior were observed for IN rats during the first trial of the covered lever session (Fig. 4, vs. lever visible; probability, t₍₁₈₎ = 0.87, P = 0.40; latency t₍₁₈₎ = 0.52, P = 0.61; contacts/sec, t₍₁₈₎ = 1.41, P = 0.17), across all 25 covered lever trials IN rats displayed a small increase in food-cup-directed conditioned responding. During the entire covered lever session (compared with the lever visible session), IN rats trended toward increased probability (t₍₁₈₎ = 1.89, P = 0.08) and rate at which they contacted the food cup (t₍₁₈₎ = 2.06, P = 0.05), while they significantly decreased their latency to approach the food cup (t₍₁₈₎ = 3.04, P = 0.0071). It is hypothesized that these data reflect enhanced vigor of conditioned responding during CS presentation, as neither sign-tracking toward the lever cover nor inter-trial-interval responding (described below) were altered for IN rats when the lever was covered.

Conditioned approach for GT rats after the lever was covered

While GTs adjusted their behavior and started sign-tracking when the lever was covered (similar latency and probability of lever cover approach to STs; see ST section above), they did so with less vigor than STs. In addition to GTs making less contact with the lever cover than STs (see ST section above), GTs were more likely to approach the lever cover only after first approaching the food cup. Accordingly, there was a significant group effect regarding the behavior that was performed first on each trial when the lever was covered (F_(2,12) = 6.340, P = 0.01; one-way ANOVAs, food cup contact; data not shown, video analysis). While GTs approached the food cup first 81% of the time, STs and INs approached the food cup first 55% and 47% of the time, respectively (GT vs. IN, P < 0.05; Bonferroni). As these data suggest that food-cup-directed behavior for GT rats remained remarkably constant before and after the lever was covered (despite also beginning to sign-track). According to comparisons of either the first trial or all 25 trials of lever-visible versus -covered sessions, there were no significant differences for the latency of GTs to approach the food cup (Fig. 4B; first trial, t₍₂₂₎ = 0.59, P = 0.56; all trials, t₍₂₂₎ = 1.28, P = 0.21) or the rate at which they contacted the food cup (Fig. 4C; first trial, t₍₂₂₎ = 0.97, P = 0.34; all trials, t₍₂₂₎ = 1.42, P = 0.17). While there was no significant difference in the probability of GTs approaching the food cup between the first trials of lever-visible and -covered sessions (Fig. 4A; t₍₂₂₎ = 0.41, P = 0.69), across the entire session of covered lever trials GTs significantly decreased their probability of approaching the food cup (Fig. 4A; t₍₂₂₎ = 2.59, P = 0.02). While this finding might be expected since GTs started making sign-tracking responses toward the lever cover during the session, it is nonetheless surprising given that other facets of their behavior did not change after the lever was covered. As will be discussed later, another explanation for reduced probability of food cup approach in GTs is that their perceived predictive value of the lever-CS sound was weakened when the lever was covered, and thus for them reward delivery was, to some degree, unexpected and surprising.

Orienting and inter-trial-interval behaviors for STs, INs, and GTs after the lever was covered

It seems that, after the lever was covered, all rats responded to the predictive value of the lever sound by approaching both the lever cover and food cup. Further supporting this idea, we found that (1) all rats learned a conditioned orienting response to the lever-CS sound (consistent with Yager et al. 2015; F_(2,13) = 0.52, P = 0.61, subset video analysis, data not shown), and (2) covering the lever did not increase food cup approach during inter-trial-intervals (group effect, F_(2,27) = 1.09, P = 0.35; lever availability effect, F_(1,27) = 3.94, P = 0.06; Group × Lever availability interaction, F_(2,27) = 0.12, P = 0.89; two-way between–within ANOVA). Thus, all rats responded to the predictive lever sound, even when the lever was not visible.

To varying degrees, we found that all rats sign- and goal-tracked when the lever was covered. Importantly, though, the way rats interacted with the lever cover (appetitive responses; nose taps) was substantially different from their mode of interaction with the extended visible lever (early consummatory responses; biting and nibbling; Timberlake 1994). Indeed, rats could not interact with the lever-CS sound cue in the same way they nibbled at the lever, and they did not transfer consummatory-like behavior to the lever cover. It is thus possible that unique modes of conditioned responding (general vs. focused behaviors) may differentially promote incentive motivation. Although it is not well studied, we propose that the biopsychological process that recruits early consummatory responses may also enhance the incentive salience of the cue that is imminently predictive of reward, turning the CS into a “motivational magnet.” Therefore, based on the present findings, we predicted that rats did not perform consummatory-like responses when the lever was covered (i.e., no opportunity to bite and nibble the lever-CS), then the incentive value of the cue may have also decreased. We thus used tests of conditioned reinforcement to help determine whether the visible lever-CS had similar incentive value compared with the covered lever-CS.

Conditioned reinforcement

Under normal training procedures, the lever-CS is a more effective conditioned reinforcer in STs than either GTs or INs (Robinson and Flagel 2009; Meyer et al. 2014), which provides additional evidence that the lever-CS is a potent incentive stimulus for STs (Robinson et al. 2014). When we covered the lever after 5 d of normal conditioning, we found that while STs continued to approach the lever vicinity (sign-tracking), they also started to approach the food cup (goal-tracking). We therefore hypothesized that when STs were unable to see or interact with the visible lever-CS, the cue may have lost some of its incentive value. To address this, we asked whether just the sound of the covered lever would act as an effective conditioned reinforcer (n = 39; new set of rats that had no experience with the covered lever). The present findings extend previous results (Meyer et al. 2014). First, when the lever was visible it acted as an effective conditioned reinforcer in STs, but not GTs or INs rats (Fig. 5A; two-way repeated measures ANOVA of active–inactive nose pokes; group effect, F_(3,35) = 3.08, P = 0.040; lever visibility effect, F_(1,35) = 8.35, P = 0.0066; interaction, F_(3,35) = 2.28, P = 0.097; lever visible STs vs. INs or GTs, P < 0.05 Bonferroni). STs also nose poked more for lever presentation than rats that were originally trained with the lever presentation unpaired from reward delivery (P < 0.01). In addition to the lever-CS being reinforcing, STs were also attracted to the lever and interacted with it more than any other group (Fig. 5B; one-way ANOVA, F_(3,35) = 15.55, P < 0.0001; STs vs. GTs or Us P < 0.0001, STs vs. INs P < 0.05, Bonferroni). In contrast, when the lever was covered, the sound of the lever extension was not an effective conditioned reinforcer in any group. Accordingly, when the lever was covered STs displayed a dramatic decrease in responding for lever presentation (P < 0.01, Bonferroni). In combination, these findings suggest that while the sound of the lever may be necessary for the development (experiment 1) and sufficient for the continued performance (experiment 2) of sign-tracking, the lever sound is not sufficient to elicit a robust incentive motivational state in trained rats. Thus, as later discussed, these experiments may illustrate a dissociation between the expression of sign-tracking-conditioned approach and conditioned reinforcement.

Figure 5.

Conditioned reinforcement. (A) When the lever was visible, STs displayed enhanced conditioned reinforcement (active—inactive nose pokes) for presentation of the lever-CS compared with IN ((#) P < 0.05), GT ((#) P < 0.05), or unpaired (U; (##) P < 0.01) rats. In contrast, when the lever was covered, conditioned reinforcement for lever presentation was drastically reduced in STs ((**) P < 0.05). Green-shaded regions indicate responding when the lever was covered (not visible). (B) Upon lever insertion following a nose poke into the active port, STs contacted the lever significantly more than any other group ((^#–###) P < 0.0001–0.05). Means are shown as ±SEM.

Experiment 3: DA release in the NAc core during the expression of conditioned approach

As STs develop lever-directed approach behavior, DA release in the NAc core shifts from the time of reward (US) delivery, to that of lever (CS) presentation (Flagel et al. 2011). This has been hypothesized to reflect the motivational value STs attribute to the CS (Flagel et al. 2011; Robinson et al. 2014). In contrast, GTs do not attribute incentive value to the lever-CS, and for them DA responses continue to occur following both CS and US (Flagel et al. 2011). We hypothesized, therefore, that if the incentive value of the lever was reduced by covering it (experiment 2, conditioned reinforcement), then we would observe equivalent CS- and US-evoked DA release in all rats. To investigate this, we used FSCV to measure conditioned DA release in well-trained rats that were characterized as STs, INs, and GTs (Fig. 6, representative color plots).

Figure 6.

Representative color plots. (A) Example FSCV recordings of NAc core DA signaling for a ST rat when the lever was visible (left; lever contact trial) or covered (right; food cup contact trial). (B) Representative recordings from a GT rat when the lever was visible (left; food cup contact trial) or covered (right; food cup contact trial). Green shading denotes covered lever trials. Color plots show current changes recorded on the electrode, plotted against the applied voltage (E_app) and trial time. The holding potential applied to the carbon-fiber electrode (−0.4 V) was ramped to +1.3 V and back to −0.4 V at a rate of 10 Hz.

Lever visible

Average [DA] traces for when the lever was visible and available are displayed for STs (Fig. 7A), INs (Fig. 7C), and GTs (Fig. 7E). While STs always contacted the lever and GTs always approached the food cup, IN rats varied in their behavior. Therefore, [DA] traces for IN rats (Fig. 7C) are plotted according to whether they sign-tracked (red), goal-tracked (blue), or performed both behaviors (purple) on averaged subsets of trials. Across animals, the pattern of DA release reflected these differences in behavioral performance (n = 28). During sign-tracking trials only the CS-evoked DA release, while on goal-tracking trials both the CS- and US-evoked DA release—this dual CS+US response resulted in a nearly twofold greater overall increase in [DA] measured during food cup approach trials than during lever-directed trials (DA area-under-the-curve calculation; t₍₂₆₎ = 2.33, P = 0.028).

Figure 7.

Modulating the incentive value of the CS alters dopamine. The average [DA] time-course for lever visible trials is displayed for ST (A), IN (C), and GT (E) rats. [DA] traces are color coded according to the behavior performed (red, sign-tracking; blue, goal-tracking; purple, both sign- and goal-tracking). Accordingly, when the lever was visible, lever contact trials were associated primarily with CS-evoked DA release, while food cup only trials displayed DA responses following both CS and US presentations ((*) P < 0.05, total [DA] released on food cup only trials vs. lever contact trials). Covering the lever such that only the auditory component of the cue predicted reward dramatically changed the pattern of DA release in the NAc core for STs (B; red-black line), but not significantly for either IN (D; purple-black line) or GT (F; blue-black line) rats. Yellow background shading denotes the time period of lever extension (regardless of visibility), while green background shading represents covered lever trials.

Figure 8 further compares the degree to which exposure to the CS or the US elicited DA release in the NAc core (subtraction of peak [DA] following the US from the peak [DA] response after the CS, [DA]_CS–US). Accordingly, there was a significant relationship between the actual behavior performed and the ability of the CS compared with the US to elicit DA release (lever visible, [DA]_CS–US, one-way ANOVA; F_(4,28) = 4.481, P = 0.0063; groups: ST (lever only), IN (lever only), IN (food cup only), IN (lever and food cup), GT (food cup only)). On trials when a rat contacted the lever, regardless of whether the rat also contacted the food cup on the same trial, the CS alone evoked DA release. As a result, [DA]_CS–US was similar between STs (Fig. 8A, red) and subsets of trials for INs (Fig. 8B; lever only trials, red; lever and food cup trials, purple; Tukey’s post hoc test). Interestingly, the opposite of these findings was also true—both the CS- and US-evoked DA release during trials that only included food cup approach. Thus, there was no difference in [DA]_CS–US between GTs (Fig. 8C, blue) and food-cup only trials performed by INs (Fig. 8B, blue; Tukey’s post hoc test). Given these results, it was not surprising that there were significant differences (P < 0.05, Tukey’s test) between pure sign-tracking and pure goal-tracking trials: [DA]_CS–US was significantly greater for STs (Fig. 8A, lever only, red) compared with either INs (Fig. 8B, food cup only, blue; P < 0.05) or GTs (Fig. 8C, food cup only, blue; P < 0.05). To summarize, CS-evoked [DA] was only significantly larger than US-evoked [DA] when a rat made a lever press (i.e., lever contact/deflection) during a trial, regardless of whether the rat was classified as an ST or IN.

Figure 8.

Peak [DA] responses vary as a function of lever visibility. The differential ability of the CS versus the US to elicit a DA response (peak CS minus US-evoked DA release; [DA]_CS–US) is quantified for STs (A), INs (B), and GTs (C). When the lever was visible, [DA]_CS–US for STs was significantly greater than [DA]_CS–US for either GTs ((*) P < 0.05) or INs ((^#) P < 0.05; food cup contact trials). In contrast, when the lever was covered (black bars), [DA]_CS–US did not differ between rats. Means are shown as ±SEM.

Lever covered

When the lever-CS was visible, it had the capacity to act as an incentive stimulus. Accordingly, when the lever elicited approach (contact), this sign-tracking response was associated with greater CS- than US-evoked DA release. In contrast, on trials during which rats goal-tracked, this response to the predictive (rather than incentive) value of the CS was associated with equivalent CS- and US-evoked DA release. Furthermore, we demonstrated that when the lever was covered, all rats (STs, INs, and GTs) responded to the predictive sound of the lever-CS (Fig. 4) and rats did not attribute much incentive value to this auditory cue (Fig. 5), regardless of whether they approached to the lever cover, food cup, or both. Given this reduction in incentive motivation, we hypothesized that conditioned behavioral responses observed after the lever was covered would be associated with equivalent CS- and US-evoked DA responses (n = 15). Thus, on a single day, we recorded DA levels in the NAc core during standard conditioning procedures (lever visible, 25 trials; Fig. 6, left, representative recording) and then subsequently after the lever was covered (another 25 trials; Fig. 6, right, representative recording).

When the lever-CS was visible, STs only contacted the lever and displayed greater CS- than US-elicited DA release (Fig. 7A). These responses changed when the lever was covered—STs also began to approach the food cup and displayed both CS- and US-evoked DA release (Fig. 7B). This new neurochemical response in STs resembled (1) the pattern of DA release seen in GTs when the lever was visible (Fig. 7E) and (2) the DA responses observed for both the IN (Fig. 7D) and GT (Fig. 7F) rats when the lever was covered. Accordingly, there were no longer significant differences in overall trial [DA] because both the CS and US increased NAc core [DA] in all rats (DA area-under-the-curve calculation, STs vs. IN + GTs; t₍₇₎ = 1.43, P = 0.20). Indeed, there was no significant difference in the ability of the CS or US to enhance DA levels between ST, IN, and GT rats when the lever was covered ([DA]_CS–US, Fig. 8, black bars; one-way ANOVA, F_(2,12) = 1.216, P = 0.33). These findings support the idea that the flexible performance of CRs, as observed after covering the lever, is associated with specific changes in patterns of DA release in the NAc core.

While not statistically significant, it appeared that after covering the lever for INs and GTs, US-evoked DA increased relative to CS-evoked DA release (Figs. 7D,F, 8B,C). While some individual variation among INs and GTs may account for this finding, it is also possible that, for some rats, increased US-evoked DA release was associated with a reduction in the perceived predictive value of the CS. Indeed, GTs decreased their probability of food cup approach when the lever was covered (Fig. 4A). Thus, similar to DA excitability and transmission following unpredictable reward early in conditioning (Schultz et al. 1997; Day et al. 2007), enhanced US-evoked DA release may potentially be indicative of unexpected reward in some IN and GT rats.

Finally, it is important to remember that, when the lever was covered, ST rats continued to approach to the lever location during the CS. If this sign-tracking behavior corresponded with the attribution of incentive salience to the sound of the lever-CS, then one might hypothesize that the CS would evoke greater DA release than the US. To determine if this was true, we compared the patterns of DA release during the covered-lever session between trials in which STs approached the lever cover first, or instead approached the food cup first (STs almost always approached both the lever cover and food cup during each trial). Regardless of which approach behavior was performed first, both the CS- and the US-evoked equivalent DA release in STs when the lever was covered ([DA]_CS–US, t₍₅₎ = 0.52, P = 0.63; data not shown). Therefore, the level of incentive value attributed to the lever-CS did not vary according to the order of approach behavior observed in STs when the lever was covered. This lack-of-effect is consistent with the observation that the covered lever was a weak conditioned reinforcer compared with the visible lever-CS, even in STs (Fig. 5A).

Discussion

We sought to determine the extent to which the sensory features of a CS paired with food reward contribute to individual variation in the form of resulting conditioned approach CRs, and in particular, the contribution of a multimodal stimulus (a lever-CS) to conditioned responding and associated DA neurotransmission in the core of the accumbens. An auditory stimulus (the sound of lever extension) resulted in the development of conditioned food cup approach (goal-tracking) in STs, GTs, and INs, consistent with a previous study (Meyer et al. 2014). In contrast, when the lever was inserted into the chamber, providing movement, visual, auditory, and tactile cues the development of sign-tracking was promoted in some rats. The visible movement, and associated sound, of the lever-CS were important for sign-tracking because only goal-tracking developed when the lever was stationary (fixed in place) and only its illumination served as the CS. This latter observation differs from a previous report in which sign-tracking was observed toward a stationary illuminated lever (Boakes 1977). However, in the Boakes (1977) study the food cup was occluded by a flap covering the food cup aperture, which may have enhanced the salience of the lever-CS despite its lack of movement. But a light CS does support the acquisition of sign-tracking in pigeons (Silva et al. 1992), and in rats when paired with a drug US (Yager and Robinson 2013; Yager et al. 2015).

Even after the development of sign- and goal-tracking, we found that the form of the CR was flexible and could change according to fluctuating circumstances. While STs continued to approach the lever location after the lever was covered (sign-tracking toward the lever-sound cue), they also started to approach the food cup during the same CS period (goal-tracking). However, in STs the covered lever-CS (sound alone) was a weaker conditioned reinforcer than the visible lever-CS, suggesting that it had reduced incentive value compared with the visible lever-CS.

DA neurotransmission in the NAc core is both necessary and sufficient for STs to attribute incentive value to a lever-CS (Flagel et al. 2011; Saunders and Robinson 2012; Singer et al. 2016). In particular, the development of sign-tracking is associated with a shift in DA release from reward delivery (US) to presentation of the lever-CS (Flagel et al. 2011). In contrast, Flagel et al. demonstrated that this change did not occur in GTs—both the CS- and US-evoked DA release, even after several days of Pavlovian training. In the present manuscript, we both replicate and extend these findings by also illustrating the relationship between variation in DA release and behavioral performance for IN rats. For IN rats, the CS primarily evoked DA release on lever-approach trials, while both the CS- and US-evoked DA release on food cup approach trials. Based on these findings, it is hypothesized that either (1) the incentive value of the lever may not be constant but instead change from trial to trial or (2) on different trials rats can alternate between attending to incentive or predictive properties of the CS. Both of these options may support behavioral flexibility, potentially allowing for individuals to update cognitive expectations and model their behavioral strategy (Dayan and Berridge 2014). Finally, the flexibility IN rats displayed in both behavior and DA signaling led us to question whether STs were also able to adapt their responding. Accordingly, we found that after covering the lever, not only did STs begin to approach the food cup, but they also displayed DA responses that were reflective of goal-tracking (i.e., equivalent CS- and US-evoked DA release).

Neural network models of associative conditioning may help to explain why the incentive value of a CS may be subject to change. As proposed by Delamater (2012), several sensory-specific input pathways (visual CS1, auditory CS2, movement CS3, etc.) may be connected via “hidden units” to various outputs (US1, US2, etc.). These hidden units may receive information from either single or multiple sensory modalities: “Thus, when a visual stimulus is presented, it will be permitted to activate hidden units not only from the visual pathway set of hidden units but also from the multimodal pathway set of units. An auditory stimulus, likewise, will activate auditory pathway hidden units, as well as multimodal hidden units” (Delamater 2012, p. 6). Accordingly, hidden units that receive information from convergent sensory pathways (instead of just a single sense) may be favored and lead to greater circuit output. In turn, similar to how the associative strengths from multiple stimuli summate (CS–US relationship strength; Thein et al. 2008), the component incentive properties of the cue may also combine, enhancing the attractiveness of the CS. We hypothesize that this is what occurs for rats (STs and INs) when they contact the lever-CS.

Importantly, changes in the incentive value of the CS do not necessarily correspond with the associated strength of the CS–US relationship (Anselme 2015). For example, in the present results and in previous reports, all rats learned the CS–US relationship equally well and developed a conditioned orienting response to the lever-CS (Yager et al. 2015; Robinson et al. 2014), regardless of whether or not they attributed incentive value to the cue. Furthermore, when a unimodal CS was used (experiment 1, fixed-illuminated lever), only a conditioned goal-tracking response developed, demonstrating learning of the CS–US relationship. The fixed-illuminated lever, however, neither elicited approach nor lever contact; the cue was not attributed with incentive value. Accordingly, we hypothesize that the unimodal fixed-illuminated lever-CS caused less “sensory excitement” than the moving/audible lever-CS, thereby resulting in reduced activation of the hidden unit of cognitive processing and a subsequent loss of incentive value, rather than associative strength. Interestingly, the absence of sign-tracking (contacts) to the fixed-illuminated lever seems to contradict previous research demonstrating how rats sign-track to a reward-predictive cue (white rectangles) shown on a visual display unit (VDU; Parkinson et al. 2000b). This discrepancy, however, serves as another example of how the brain processes different forms of stimuli in unique ways. For example, while the performance of sign-tracking to a lever requires activation of the basolateral amygdala (Chang et al. 2012), the expression of sign-tracking to a purely visual cue on a VDU instead requires the central nucleus of the amygdala (Parkinson et al. 2000b).

Unlike the VDU-cue, the ability to interact with the lever-CS may encourage the expression of consummatory behaviors, requiring the activation of specific neural circuitry (Parkinson et al. 2000a,b; Chang et al. 2012; Chang and Holland 2013). For example, a sign-tracking individual may enter a “focal search mode,” preparing the animal to handle food by promoting behaviors such as biting or gnawing (Timberlake 1994; Beckmann and Chow 2015). These CRs are routinely observed during sign-tracking interaction with the lever-CS. In contrast, goal-tracking may represent a “general search mode” in which conditioned responding has not yet entered a consummatory phase. While it is tempting to equate the “focal search mode” with the attribution of incentive value to the lever-CS in STs, recent work suggests that goal-tracking may also engage consummatory systems (Mahler and Berridge 2009). Therefore, it is possible that the performance of consummatory conditioned response behaviors may be an indication of the strength of the CS–US relationship, rather than the incentive salience of the CS.

Similar to this ecological perspective of early consummatory-like behavior, computational models also may help to explain individual variation in conditioned approach. For example, it has been suggested that lever-approach behavior may be a consequence of model-free processing, while goal-tracking reflects activity in model-based systems that require learning the structure of the task in order to optimize chances of reward (Lesaint et al. 2014a,b). Accordingly, behavior may be flexible, resulting from ongoing calculations of both incentive salience and advantage functions for behavioral performance. Importantly, though, the degree to which model-free systems dictate sign-tracking behavior is questionable, as changes in the incentive value of the CS can have an immediate consequence on conditioned responding (Robinson and Berridge 2013; Dayan and Berridge 2014).

The aforementioned cognitive models of conditioned responding may also help explain the distinct patterns of DA release observed in the NAc core during sign- and goal-tracking trials. It is possible, for example, that on lever-contact trials multimodal sensory activation of hidden units may be at its maximum. In this situation, the neural networks may only have enough resources to allow for the CS to evoke DA release. This does not mean that there is insufficient DA available to be released, but instead suggests possible inhibition or reduced excitation of VTA DA neuron firing (Danna et al. 2013), or altered terminal control of DA release and uptake in the NAc (Singer et al. 2016). If this was the case, then reduced hidden unit processing and DA neurotransmission may limit behavioral control by model-based cognitive strategies.

There are other alternative explanations for the data that also deserve consideration, either independently or in concert with the previous interpretations. For example, the variability in the pattern of DA release and behavior across trials may be a consequence of ongoing computation of motivation and reward value (Hamid et al. 2016). This form of control could be modulated by other brain regions or neurotransmitter systems. In particular, it is possible that directing attention toward specific cue components, or alternatively the lack of paying attention to them, may significantly impact responding. This idea is especially relevant considering that STs and GTs have different attentional function as dictated via cholinergic signaling (Paolone et al. 2013). While different systems may be involved, cholinergic neurotransmission can modulate DA signaling in the ventral striatum (Threlfell et al. 2012), and this could in turn impact the patterns of DA release observed during approach behavior.

When the lever was covered, all rats began to both approach the lever cover and the food cup during the CS period. In addition, during this manipulation both the CS- and US-evoked DA release. This finding may be construed as surprising, as we have hypothesized that sign-tracking and incentive value attribution are associated with the CS alone evoking DA release. Regardless of this assertion, at least two major differences may explain the covered lever results. First, despite continuing to evoke sign-tracking, the covered lever was not attributed with much incentive value because it was a weak conditioned reinforcer. The covered lever did, however, have predictive value, as all rats oriented toward its location upon hearing the sound associated with its movement. Second, the form of the sign-tracking CR was different when the lever was covered, consisting of only a “general search mode” of preparatory appetitive behavior (Timberlake 1994). In contrast, when the lever was visible, sign-tracking consisted of both appetitive and early consummatory (e.g., biting, nibbling the lever) modes of responding. It is therefore possible that the combination of various forms sign-tracking was essential to the attribution of incentive value to the lever-CS.

As a whole, the results of the present experiments illustrate functional roles of multisensory cues in mediating conditioned responding. Depending upon the sensory experience of cues, individual variability in behavior and neurochemistry emerges. Importantly, under changing situations, the meaning or value of CS may be altered, modifying patterns of neurotransmission, and ensuring behavioral flexibility. The use of complex stimuli, such as the lever, in animal models of conditioning is important given that in humans the sensory modality of cues influences their ability to alter brain activation (Yalachkov et al. 2012) and subsequently motivate food- and drug-seeking behavior (Johnson et al. 1998; Shadel et al. 2001; Reid et al. 2006).

Materials and Methods

Subjects

Outbred male Sprague-Dawley rats (from both Harlan and Charles River Laboratories) were individually housed in a temperature- and humidity-controlled vivarium with a reverse light cycle. Food and water were available ad libitum (no food deprivation) and all procedures began 1-wk after arrival during the dark phase of the light cycle. All procedures were conducted according to a protocol approved by the University of Michigan Committee on Use and Care of Animals (UCUCA).

Standard Pavlovian training

Behavioral conditioning used test chambers and procedures described previously (Saunders and Robinson 2012). Briefly, rats were first introduced to the reward (banana-flavored pellets; US) in their home cages and then subsequently acclimated to the test chambers for 1 d while they were trained to retrieve 25 pellets from the food cup (variable time 30 sec schedule). Then, over the course of 5 training days, rats experienced 25 CS–US pairings per day. The CS (8 sec extension of a lever that was illuminated by an LED from behind) was presented on a variable time 90 sec schedule. Following lever retraction, one banana-flavored pellet (US) was delivered to food cup located either to the left or right of the CS. Red house lights illuminated the chambers for the entire sessions. As described previously (Meyer et al. 2012; Saunders and Robinson 2012), each rat’s behavior was quantified according to their (1) response probability [P_(lever) − P_{(food cup)}], (2) response bias [(lever deflections − food cup entries)/(lever deflections + food cup entries)], and (3) the latency [(lever deflection latency − food cup entry latency)/8] for making either a lever-directed or food-cup-directed CR during the CS. Subsequently, a Pavlovian-conditioned approach (PCA) index was calculated for each rat, averaging their days 4 and 5 training data according to the following formula: [(probability difference + response bias + latency score)/3]. Based on this index, STs were defined as animals that had a score ranging from +0.4 to 1.0. In contrast, GTs had a score ranging from −1.0 to −0.4. Rats with intermediate scores (INs) were also classified (PCA score −0.4 to +0.4).

Experiment 1: CS properties and the development of conditioned approach

Standard procedures for Pavlovian conditioning were followed as described above (n = 46). In addition, in separate experiments using different rats, the following modifications to the conditioning procedure were tested.

Covered lever during training

The lever was covered during the 5 d of conditioning (n = 12) with either the plastic lid of a box designed to hold microscope slides (14 cm tall, 8.5 cm wide, 1.5 cm deep; secured with tape) or the top half of a plastic soap dish (11 cm tall, 8 cm wide, 3 cm deep; secured with tape). Each rat only experienced one type of lever cover across training and there appeared to be no qualitative differences between rats in their interactions with the lever covers. Furthermore, the latency for rats to contact the food cup was not different for the various lever covers (t₍₁₀₎ = 1.09, P = 0.30), suggesting that approach behavior was similar across conditions. During these training sessions, the lever could be heard extending and retracting, but the lever itself was not visible. Following this procedure, rats were trained for 5 additional days, but with the lever uncovered and thus available as described in the Standard Pavlovian Training procedures above.

Fixed illuminated lever during training

The lever remained extended into the test chambers for the entire session and did not move (n = 32). Only the 8 sec illumination of the lever, provided by an LED mounted within the lever casing behind the test chamber wall, served as the CS predictive of reward delivery. Similar to standard conditioning procedures, a red house light illuminated the chambers for the entire conditioning session.

Experiment 2: CS properties and the expression of conditioned approach

Rats underwent 5 d of Standard Pavlovian Training as described above and were characterized as STs, INs, and GTs. Afterward, the following behavioral tests were performed:

Covered lever after training

After the initial 5 d of training, rats were returned to the behavioral chambers and underwent a sixth conditioning session with the lever covered (ST, n = 8; IN, n = 10; GT, n = 12). On this last test only the sound of the lever’s movement served as the CS. The cover for the lever consisted of a slide box lid (14 cm tall, 8.5 cm wide, 1.5 cm deep; secured with tape), a soap dish lid (11 cm tall, 8 cm wide, 3 cm deep; secured with tape), or an in-house manufactured anti-static fiberglass chamber insert (6 cm tall, 7 cm wide, 2 cm deep; slides into chamber, no tape required) that was designed to reduce noise artifacts during collection of FSCV data (experiment 3). Across experiments, there were no instances of rats removing the lever covers. Of course, it was not possible to automatically measure lever deflections because the lever was not available to contact, but, in a subset of rats (n = 15; rats used in FSCV experiments with CS-extension time-stamps) video analysis allowed for the assessment of approach behavior to the lever cover. Contact with the lever cover was recorded if a rat touched the cover with its nose or paw. Time-stamps for the calculation of latency to approach the lever cover were recorded using Open-Source software (CowLog; Hänninen and Pastell 2009).

Conditioned reinforcement after training

In a separate subset of rats that had never experienced covered-lever procedures, tests of conditioned reinforcement were performed for 2 d immediately following either standard PCA training (ST, n = 7; IN, n = 6; GT, n = 11) or control training procedures in which the lever-CS was not followed by reward (Unpaired, U, n = 15). During conditioned reinforcement tests, the food cup was removed from the chamber and the lever-CS was positioned in its place. Nose poke ports equipped with photo-beams to detect entry were positioned to the left and right of the lever. During the conditioned reinforcement session, entry into a nose poke port designated as “active” (reinforced) was followed by extension of the illuminated lever for 3 sec (Fixed Ratio 1 schedule). In contrast, entry into the “inactive” nose poke port did not result in lever extension and served as a control for nonspecific behavioral activity. The side of the “active” nose poke port was counter balanced across rats.

The two conditioned reinforcement sessions lasted for 40 min each. Counterbalanced across rats, during one reinforcement session the lever-CS was visible and available for the rats to interact with following active nose pokes. In contrast, during the other session the lever-CS was covered (using slide box lids as described above) such that when an active nose poke was performed rats could hear the sound of the lever extending, but could not see the lever. Thus, this manipulation examined the conditioned reinforcing properties of the auditory component of the lever cue. For these experiments, lever contacts were measured when the CS was visible and conditioned reinforcement was analyzed as active minus inactive nose poke entries.

Experiment 3: DA release in the NAc core during the expression of conditioned approach

Surgery

Rats (ST, n = 8; IN, n = 8, GT, n = 12) were first trained using standard 5-d conditioning procedures as described above. The next week, rats were anesthetized (ketamine, 100 mg/kg; xylazine, 10 mg/kg; IP) and underwent stereotaxic surgery for in vivo FSCV. As previously described (Vander Weele et al. 2014; Singer et al. 2016), a guide cannula (Bioanalytical Systems) was implanted dorsal to the NAc core (AP, +1.3; ML, ±1.3; DV, −2.5 mm relative to bregma) and an Ag/AgCl reference electrode was secured in the contralateral cortex (AP, −0.8; ML, ±4.0; DV, −2.0 mm relative to bregma). In addition, a bipolar-stimulating electrode (AP, −5.2; ML, ±0.8 mm relative to bregma; Plastics One) was lowered into the ventral tegmental area (VTA) until electrically evoked DA release was able to be measured in the striatum. DA release was recorded using glass-encased cylindrical carbon-fiber electrodes. Surgical screws and dental acrylic were used to secure cannula and electrodes in place. Before surgery and during recovery, rats were administered saline, the antibiotic cefazolin (100 mg/kg, SC), and the analgesic carprofen (5 mg/kg, SC).

Behavior

After at least 5 d of recovery, when rats had returned to their presurgical weight, animals were retrained on the standard conditioning task for 2 d to confirm stable behavioral responding. On the second day of retraining, rats were tethered from their head cap to a commutator in order to acclimate them to FSCV recording conditions. The following day, in vivo FSCV was performed during two 25-trial conditioning sessions. During the first of these sessions, conditioning occurred according to standard procedures described above. The second conditioning session either tested the rats again under standard conditions (n = 13) or instead tested the rats while the lever was covered (n = 15). As stated above, when the lever was covered it could not be seen but it could be heard when it was inserted and then retracted. Behavior and DA release were measured in response to the sound of the lever-CS.

Fast-scan cyclic voltammetry

During recordings, rats were tethered via a headstage (Scott Ng-Evans) and commutator (Crist Instruments) to allow free movement throughout the chamber. At the beginning of the test day, a calibrated carbon fiber electrode loaded into a micromanipulator was lowered into the NAc core and tested for its ability to detect both naturally occurring and electrically evoked (via VTA stimulation) DA release. As described previously (Vander Weele et al. 2014; Singer et al. 2016), FSCV relied on the oxidation and reduction of the analyte of interest (i.e., DA) in response to the application of a triangular waveform (oxidative scan, −0.4 to 1.3 V; reductive scan, 1.3 to −0.4 V; 400 V/sec). On testing days, this waveform was cycled for 20 min at 60 Hz, followed by another 20 min at 10 Hz.

The surface of the electrode detected rapid electrochemical changes, producing a highly stable “charging current.” The charging current was then background subtracted, causing any additional changes in current and pH to become apparent. Such current changes were plotted in false color (Fig. 6) in order to visualize the presence of DA within the NAc core. As any electrochemically active species in the carbon fiber electrode’s microenvironment results in a change in current, cyclic voltammograms (CVs) were obtained to identify specific analytes. CVs provide a qualitative basis for in vivo identification of DA, as well as acidic and basic pH fluctuations.

Current was translated into [DA] via principal component analysis obtained during each voltammetric recording. For each animal, electrical stimulation of DA cell bodies in the VTA was used to induce DA release and pH changes at the site of the recording electrode. From these recordings, a training set of 10 CVs containing a variety of changes in [DA] and pH was constructed for each rat. Principal component analysis was then conducted on these training sets, allowing for the conversion of current into nanomolar [DA], as well as the ability to discriminate between current resulting from [DA] and pH changes (Keithley and Wightman 2011). Principal component analysis determined that 99.5% of the variance in the data could be included with three to five principal components depending on animal. These sets were then used to evaluate CVs obtained during the performance of conditioned approach. All data were analyzed using High Definition Cyclic Voltammetry (HDCV) software provided by the Wightman laboratory at the University of North Carolina Chapel Hill (Bucher et al. 2013).

Histology

Following testing, animals were euthanized via overdose of Beuthanasia-D (50 mg/kg, IP; Schering-Plough). After electrolytic lesions were made at the recording site, animals were decapitated, brains extracted, and stored in formalin. At least 1-wk later, coronal brain slices (40 µm) were collected and stained with cresyl violet. Electrode placements were verified using light microscopy and referenced according to a brain atlas (Paxinos and Watson 2004). Only rats with electrodes located in the NAc core were used.

Statistical analyses

One-way ANOVAs were used for analysis of inter-trial-interval (ITI) food cup entries during initial conditioning, behavioral responding when the CS was covered (day 6), and for comparing the ability of the CS to evoke DA release compared with the US ([DA]_CS–US). According to the experiment, either one- or two-way repeated measures ANOVAs were used to assess whether rats learned the CS–US relationship across training days, as well as to analyze conditioned reinforcement data. T-tests were used to compare behavioral responding between session 5 of conditioning and session 6 when the lever was covered and not visible. All statistics were performed in either SPSS or GraphPad Prism.