Repeated cocaine exposure dysregulates cognitive control over cue-evoked reward-seeking behavior during Pavlovian-to-instrumental transfer

Abstract

Drug-paired cues acquire powerful motivational properties, but only lead to active drug-seeking behavior if they are potent enough to overwhelm the cognitive control processes that serve to suppress such urges. Studies using the Pavlovian-to-instrumental transfer (PIT) task have shown that rats pretreated with cocaine or amphetamine exhibit heightened levels of cue-motivated food-seeking behavior, suggesting that exposure to these drugs sensitizes the incentive motivational system. However, the PIT testing protocol can also create conflict between two competing behavioral responses to the reward-paired cue: active reward seeking (e.g., lever pressing) and passive conditioned food-cup approach behavior. We therefore investigated whether repeated cocaine exposure alters the way in which rats use cue-based reward expectations to resolve such conflict. In-depth analysis of previously published and new research confirmed that when drug-naïve rats are given a cue that signals the timing of a delayed noncontingent reward, they adaptively transition from reward seeking to conditioned approach behavior, facilitating efficient collection of the predicted reward. In contrast, cocaine-exposed rats exhibit pronounced behavioral dysregulation, increasing, rather than suppressing, their reward-seeking behavior over time, disrupting their ability to passively collect reward. Such findings speak to the important and sometimes overlooked role that cognitive control plays in determining the motivational impact of cues associated with drug and nondrug rewards.

Individuals with a history of repeated cocaine abuse face many challenges when attempting to abstain from drug use. Although drug-paired cues can be powerful triggers of drug craving (Ehrman et al. 1992; Grant et al. 1996; Childress et al. 1999), there is much left to learn about the factors that determine whether this motivational influence ultimately translates into overt drug-seeking behavior and relapse (Sinha 2013). This problem may be viewed as an imbalance between incentive motivation and cognitive control, in that drug-paired cues are more likely to trigger relapse if they elicit an inordinate craving that exceeds one's ability to control, or inhibit, the urge to use drugs (Bolla et al. 1998; Jentsch and Taylor 1999; Metcalfe and Mischel 1999; Robbins and Everitt 1999; Bechara and Damasio 2002; Goldstein and Volkow 2002; Robinson and Berridge 2003; Jentsch and Pennington 2014). Factors that increase the strength of cue-induced drug cravings may lead to such an imbalance, and preclinical research indicates that repeated exposure to cocaine or other drugs dysregulates the neural systems supporting adaptive incentive motivation, ultimately leading to a state of hypersensitivity to reward-paired stimuli (Robinson and Berridge 1993, 2003). Interestingly, this upshift in motivational function is so fundamental that drug-exposed animals even exhibit heightened motivational responses to cues that signal palatable foods and other nondrug rewards (Harmer and Phillips 1998; Taylor and Jentsch 2001; Nocjar and Panksepp 2002; Nordquist et al. 2007; Afonso et al. 2009; Mendez et al. 2009; Ranaldi et al. 2009; Simon et al. 2009; Doremus-Fitzwater and Spear 2011; Shiflett 2012; Palmatier et al. 2013; McClory and Spear 2014; Robinson et al. 2015; Spoelder et al. 2015; Saddoris et al. 2016).

Some of the strongest evidence for drug-induced alteration in motivation comes from studies using the Pavlovian-to-instrumental transfer (PIT) task, which selectively assays the motivational properties of reward-paired cues (Rescorla and Solomon 1967; Holmes et al. 2010; Cartoni et al. 2016). In such experiments (e.g., Dickinson et al. 2000; Wassum et al. 2011, 2013), rats receive repeated Pavlovian pairings between a conditioned stimulus (CS+) and food reward, which is typically collected and consumed from a food cup. In a separate phase of the experiment, rats are trained to perform an uncued instrumental action, such as lever pressing, to earn food reward. At test, rats can freely lever press without earning reward (i.e., in extinction) while the CS+ is intermittently presented to determine the degree to which that cue is effective in motivating reward seeking (i.e., lever pressing). Multiple studies using the PIT task have shown that repeated preexposure to cocaine or amphetamine facilitates the expression of this measure of behavior, as do acute intracerebral injections of amphetamine into the ventral or dorsal striatum, suggesting that these treatments amplify the motivational potency of reward-paired cues.

Importantly, these studies have typically used PIT protocols that support little or no cue-motivated reward seeking in drug-naïve control groups (see Table 1). These suboptimal PIT protocols are noteworthy because they use CS+ stimuli that support strong reward expectancies, relative to the weak CS+ stimuli that are normally used in PIT studies (Dickinson et al. 2000). The temporal relationship between the CS+ and reward delivery is an important determinant of PIT and other forms of interaction between Pavlovian and instrumental behavioral systems (Konorski 1967; Bindra 1974; Timberlake et al. 1982). Specifically, cues that signal imminent, noncontingent food delivery tend to suppress, rather than invigorate, food seeking (Azrin and Hake 1969; Van Dyne 1971; Lovibond 1981), whereas long duration cues that are loosely associated with food reward are most effective in motivating such behavior (Estes 1943, 1948; Meltzer and Brahlek 1970), as discussed in recent reviews (Holmes et al. 2010; Cartoni et al. 2016). In fact, even putatively neutral cues (e.g., random control stimuli) (Rescorla 2000) and conditioned inhibitors (i.e., cues that signal the absence of imminent food delivery) (Konorski and Miller 1936, as described by Konorski 1967) can be effective in motivating instrumental food seeking, presumably because they serve as a general context for reward availability without triggering a strong expectation of impending, noncontingent reward delivery.

Table 1.

Psychomotor stimulant effects on Pavlovian-to-instrumental transfer

This inverted-U relationship between the strength of Pavlovian conditioning and the strength of PIT performance does not appear to reflect a reduction in the motivational properties of cues that signal imminent reward, but instead reflects their tendency to elicit incompatible conditioned approach behavior directed toward the site of food delivery (Holmes et al. 2010; Cartoni et al. 2016). This motivational influence can be unmasked by treatments that discourage or weaken competing conditioned approach responses (Baxter and Zamble 1982; Lovibond 1983; Holmes et al. 2010). Thus, rather than providing a simple readout of cue-evoked incentive motivation in all cases, it is more appropriate to view the PIT paradigm as a tool for gauging the influence of reward-predictive cues on the expression of two competing foraging strategies (or behavioral systems), with weak cues biasing animals toward active (exploratory) reward-seeking behavior and strong cues encouraging a more passive (focused) conditioned approach response directed toward the location of upcoming food delivery (cf. Timberlake et al. 1982). Importantly, competition between these behavioral strategies tends to be asymmetrical, in that strong cues elicit conditioned approach behavior even though they also retain a covert (or masked) motivational influence that would normally elicit instrumental reward seeking. The fact that cues signaling imminent reward do not simply increase both types of motivated behavior suggests that they are able to engage cognitive control processes to inhibit the competing impulse to engage in exploratory reward-seeking behavior, facilitating more efficient collection and consumption of the predicted reward.

This theoretical framework has important implications for interpreting the effects of drug preexposure on PIT performance. As noted above, most studies reporting potentiation of cue-motivated reward seeking in drug-exposed rats have used CS+ stimuli that (1) are relatively short in duration and (2) accurately signal the specific time of a delayed reward delivery (e.g., 30 sec after cue onset). This is important because rats can learn such information and use it to allocate their conditioned food-cup approach behavior over time during and after CS+ presentations (Holland 1980, 2000; Kirkpatrick and Church 2000; Delamater and Oakeshott 2007; Delamater and Holland 2008). As we have discussed, though relevant PIT data are limited, there is some indication that approach behavior controlled by cue-elicited reward expectancies compete with instrumental reward seeking, interfering with the expression of this behavior (Peciña et al. 2006; Holmes et al. 2010). Thus, rather than directly sensitizing the incentive motivational system that underlies expression of the PIT effect, drug exposure may indirectly facilitate PIT performance by disrupting the cognitive control processes that would normally suppress reward seeking when imminent reward is expected. Although these two accounts are difficult to disentangle, they make different predictions about the time course of PIT performance. If drug exposure primarily makes reward-paired cues more effective in triggering a state of motivational arousal, then PIT should be generally augmented, but there should still be evidence of cognitive control (suppression of reward seeking) late in the CS period when reward expectancy is greatest. In contrast, if drug exposure disrupts cognitive control over reward seeking, then PIT performance should also be augmented during periods in which there is a strong reward expectancy (i.e., when behavior would normally be biased toward the food cup). Evaluating this prediction is difficult because previous studies on this topic typically provide few details about whether drug-induced potentiation of cue motivated behavior varies with changes in reward expectancy, or how this treatment impacts the expression of concurrent food-cup approach behavior.

Here, we provide a reanalysis of a previously published PIT study in which a relatively strong CS+ was shown to be more effective in motivating reward-seeking behavior in cocaine-pretreated rats than in drug-naïve control rats (Ostlund et al. 2014). As in the majority of such studies, the CS+ signaled the specific timing of reward delivery (i.e., at CS offset), so that rats’ expectancy of reward should have grown over time during individual CS+ presentations. Because detailed information about the specific timing of behavioral events was recorded in this experiment, we were able to determine the effect of cocaine preexposure on the time course of CS+ evoked changes in lever pressing and food-cup approach behaviors. Thus, it was possible to investigate the impact of this drug treatment on rats’ ability to adaptively suppress reward seeking in order to retrieve the expected reward. We then describe a new PIT experiment designed to further investigate how variations in cue-elicited reward expectancies impact the balance between these two foraging strategies in drug-naïve rats.

Results and discussion

Experiment 1

Ostlund et al. (2014) investigated the effects of repeated cocaine preexposure on the expression of a food-motivated PIT task (see Table 2 for outline of Experiment 1). Briefly, rats in this study were initially given Pavlovian training, in which a 30-sec auditory CS+ signaled the delivery of three food pellets at cue offset. This provided a means for timing reward delivery, rendering the CS+ effective in eliciting conditioned food-cup approach behavior, relative to the control stimulus (CS−), which was unpaired with food. We confirmed that these rats had actually learned about the timing of reward delivery, in that their rate of food-cup approach increased over time during CS+ but not CS− trials (Fig. 1A,B; CS Type × CS Bin (3-sec periods), t₍₄₅₂₎ = 5.05, P < 0.001), and that there were no differences between the groups based on their planned drug treatments (main effect of and interactions involving group, Ps ≥ 0.597; full model output in Supplemental Table S1). Rats were then trained to lever press for food pellets in a separate phase of the experiment (no CSs were presented) before being split in two different drug treatment groups, which received 6 d (every other day) of i.p. injections of cocaine (15 mg/kg) or vehicle. On alternate days, all rats received vehicle injections. During subsequent PIT testing (after withdrawal), rats were free to perform the lever-press response in extinction, while the CS+ and CS− stimuli were noncontingently presented to assess their influence on reward-seeking behavior (lever pressing). While this experiment also assessed the influence of drug-paired contextual cues on expression of PIT, no such effect was observed (Ostlund et al. 2014), so we focus here on the unconditional (context-independent) effects of repeated cocaine exposure on cue-motivated behavior.

Table 2.

Timelines for Experiments 1 and 2

Figure 1.

The effects of cocaine sensitization on cue-motivated behavior (Experiment 1). (A) Change in conditioned approach behavior during the CS+ period in Pavlovian training relative to pre-CS+ baseline as a function of time. (B) Change in conditioned approach behavior during the CS− period in Pavlovian training relative to pre-CS− baseline as a function of time. (C) Change in reward-seeking behavior during the CS+ and post-CS+ periods in the PIT test relative to pre-CS+ baseline as a function of time. (D) Change in reward-seeking behavior during the CS− and post-CS− periods in the PIT test relative to pre-CS− baseline as a function of time. (E) Change in conditioned approach behavior during the CS+ and post-CS+ periods in the PIT test relative to pre-CS+ baseline as a function of time. (F) Change in conditioned approach behavior during the CS− and post-CS− periods in the PIT test relative to pre-CS− baseline as a function of time. (G) Response bias during the CS+ and post-CS+ periods in the PIT test relative to pre-CS+ baseline as a function of time. Response bias was calculated as the change in conditioned approach behavior (E) minus the change in reward-seeking (C) for each time period. (H) Response bias during the CS− and post-CS− periods in the PIT test relative to pre-CS− baseline as a function of time. All data points are group means. Error bars represent ±1 standard error of the estimated marginal means from the corresponding fitted linear mixed-effects models. (CS) conditioned stimulus.

In the original study, it was reported that the Cocaine Group exhibited a significantly larger CS+ elicited increase in reward seeking than the Vehicle Group, in line with previous studies (Table 1). We reanalyzed the data from this experiment to determine if rats in this study used CS+ elicited reward expectancies to adaptively shift from lever pressing to food-cup approach behavior and assess whether sensitivity to this influence of the CS+ was altered in cocaine-experienced rats. One challenge to this analysis is that rats approach the food cup when they expect reward, regardless of whether they think that reward is related to the cue delivery or the performance of the instrumental lever-press response. To distinguish between these two types of approach, we assessed the distribution of food-cup approaches surrounding individual lever presses during the PIT test (Fig. 2C). We found that the probability of food-cup approach was elevated for ∼2.5 sec following the lever-press response, in line with an attempt to collect response-contingent reward, even though no rewards were actually delivered at test. We categorized these behaviors as post-seeking approaches to distinguish them from conditioned approaches that were triggered by cue-elicited reward expectancies. Figure 2A shows representative PIT data from cocaine- and vehicle-treated rats to illustrate the temporal distribution between reward seeking and these two types of food-cup approach behaviors.

Figure 2.

Differentiation and categorization of behavior. (A,B) Raster plots of Experiment 1 (A) and Experiment 2 (B) for representative rats in each experiment. Data are split by CS type and all trials from one test from that rat are shown. (C,D) Probability of food cup approach as a function of time from each lever press for Experiment 1 (C) and Experiment 2 (D). Food cup approaches that occurred within 2.5 sec of a lever press (gray area) were designated as being the terminal behaviors of a bout of lever pressing (“post-seeking approaches”). All other food cup approaches were designated as “conditioned approaches.” (E,F) Probability of a post-seeking approach following each lever press. All data points are group means. Error bars represent ±1 standard error of the estimated marginal means from the corresponding fitted linear mixed-effects models. (CS) conditioned stimulus.

Figure 1C and D show the mean rates of reward seeking (lever pressing) for both groups during CS+ and CS− presentations, relative to baseline (30-sec pre-CS), plotted as a function of time. Analysis of these data using linear mixed-effects regression modeling revealed that the two groups exhibited significantly different rates of cue-motivated reward seeking, an effect that varied across CS Type (CS+ versus CS−), CS Period (CS versus Post-CS), and CS Time (10-sec periods) (Group × CS Type × CS Period × CS Time interaction, t₍₂₆₀₎ = 2.30, P = 0.022; for full model output, see Supplemental Table S2). Group differences were observed during CS+ trials (Group × CS Period × CS Time interaction, t₍₁₃₀₎ = 2.55, P = 0.012) but not during CS− trials (main effect of group and interactions involving group were not significant, Ps ≥ 0.298). Further analysis revealed that the two groups displayed opposite changes in seeking rates over time during CS+ presentations (Group × CS Time interaction, t₍₆₅₎ = 3.26, P = 0.002); specifically, the Vehicle Group reduced their rate of seeking as the expected time of reward delivery approached, while the rate of seeking in the Cocaine Group grew over time. This elevated reward-seeking rate in the Cocaine Group persisted into the post-CS period (main effect of group; t₍₆₅₎ = 2.21, P = 0.031), with no further modulation over time (Group × Time, P = 0.565).

In contrast to controls, cocaine-treated rats failed to inhibit, and instead increased, their reward-seeking behavior over time during CS+ trials, even though that cue had previously signaled (during Pavlovian conditioning) that food pellets would be noncontingently delivered into the food cup at CS+ offset. This persistent, escalating pattern of reward seeking during CS+ trials suggests that cocaine-treated rats experienced a greater motivational response to the cue or were unable to adaptively inhibit their reward-seeking behavior to allow a transition to the conditioned approach response when reward expectancy was greatest. To investigate this issue, we analyzed the time course of cue-related changes food-cup approach behavior during PIT testing. As described above, this analysis did not include food cup approaches that were performed within 2.5 sec of lever pressing. We found that the Cocaine Group exhibited significantly fewer conditioned approaches overall compared to the Vehicle Group (Fig. 1E,F) (t₍₂₆₀₎ = −3.78, P < 0.001), an effect that varied with CS Type, CS Period, and CS Time (four-way interaction, t₍₂₆₀₎ = −3.46, P = 0.001; for full model output, see Supplemental Table S3). Although both groups showed a similar pattern of conditioned approach behavior during CS+ trials, with rates increasing over time during that cue and dropping abruptly during the Post-CS+ period, this effect was much more modest in the Cocaine Group (Group × CS Period × CS Time interaction, t₍₁₃₀₎ = −3.64, P < 0.001). Specifically, conditioned approach rates were lower in the Cocaine group than in the Vehicle Group during both CS+ and Post-CS+ periods, Ps ≤ 0.009, and there were Group × CS Time interactions in both periods, Ps ≤ 0.029. While both groups increased their conditioned approach behavior during the CS+ (Fig. 1E), this increase was significant only in the Vehicle Group (t₍₃₄₎ = 4.28, P < 0.001; Cocaine Group, t₍₃₁₎ = 1.31, P = 0.200). No significant effects were observed during CS- trials (Ps ≥ 0.076). Notably, as seen in Figure 2E, there were no appreciable differences in how often the groups attempted to retrieve reward after a bout of reward seeking (post-seeking approaches; main effect of and interactions involving group were not significant, Ps ≥ 0.108; full model output in Supplemental Table S5), indicating that the drug-induced suppression of conditioned food-cup approach behavior described above was specific to approaches controlled by cue-elicited reward expectancies.

To more directly characterize the tendency of the CS+ to bias rats toward the food cup and away from the lever, we calculated a response bias measure [CS+ evoked change in conditioned approach behavior—CS+ evoked change in reward-seeking behavior]. While both groups behaved similarly during and after CS− trials (no main effects of or interactions involving group during CS− trials, Ps ≥ 0.268), they exhibited distinct biases during CS+ trials (main effect of Group, t₍₁₃₀₎ = −3.59, P < 0.001; Group × CS Period × CS Time interaction, t₍₁₃₀₎ = −3.77, P < 0.001; Fig. 1G). For rats in the Vehicle Group, the bias toward conditioned approach behavior grew over time during the CS+ and rapidly dropped thereafter (CS Period × CS Time interaction, t₍₆₈₎ = 5.71, P < 0.001). In contrast, rats in the Cocaine Group failed to show any evidence of a shift toward the conditioned approach response (CS Period × CS Time interaction, t₍₆₂₎ = 0.77, P = 0.447).

Experiment 2

The results of Experiment 1 show that repeated cocaine exposure markedly potentiated CS+ elicited reward seeking, but also disrupted rats’ tendency to transition to conditioned approach behavior later in the cue period, when noncontingent reward delivery should have been expected. Although vehicle-treated rats in that study seemed to use information about the timing of reward to shift between these behaviors, similar studies looking at the relationship between instrumental reward seeking and food-cup approach behavior during PIT suggest that these behaviors may often co-occur, peaking around the time of expected reward delivery (Delamater and Oakeshott 2007). We therefore conducted an experiment to further investigate the influence of cue-based reward timing on expression of reward seeking and conditioned approach behavior in normal, drug-naïve rats (Table 2). We used a similar PIT protocol to the one used in Experiment 1 and related studies examining the impact of repeated drug exposure on cue-motivated behavior (Table 1). Thus, rats were trained with a 30-sec auditory cue (Fixed CS+) that reliably signaled that three pellets would always be delivered at cue offset. To determine how variations in reward expectancy influence the way drug-naïve rats allocate their time between reward seeking and food-cup approach behavior, the current rats were also trained with two other 30-sec cues that had distinct relationships with reward. Although the average number of food pellets delivered per trial was the same for all cues (i.e., three pellets), the remaining two cues were unreliable in signaling either the specific timing or amount of reward delivery on any given trial. For one of these cues (Random Time CS+), individual pellets were randomly delivered over time during the cue presentation (every 10 sec, on average), including early after cue onset. For the other cue (Random Amount CS+), reward was always delayed until cue offset, but varied in terms of the number of pellets that were delivered per trial (0–6). This latter condition was included to control for uncertainty in reward magnitude (but not timing). To facilitate discrimination between cues, rats in the current study were given more extensive Pavlovian training than those in Experiment 1 (Table 2). Figure 3A shows conditioned approach behavior across the final five sessions of Pavlovian training as a function of CS type and time within the CS (3-sec bins). For the two delayed-reward CS+ cues (Fixed, Random Amount), rats increased their rate of conditioned approach behavior over time, whereas their rate of approach during the Random Time CS+ reached asymptote earlier and was relatively flat across time (CS Type × CS Time interaction, F_(2,890) = 11.88, P < 0.001; full model output in Supplemental Table S6).

Figure 3.

The effects of different CS+ types on cue-motivated behavior (Experiment 2). (A) Change in conditioned approach behavior during each CS+ and post-CS+ period in Pavlovian training relative to the pre-CS+ baseline period of each CS+ as a function of time. (B) Change in reward-seeking behavior during each CS+ and post-CS+ period in the initial set of PIT tests relative to the pre-CS+ baseline period of each CS+ as a function of time. (C) Change in conditioned approach behavior during each CS+ and post-CS+ period in the initial set of PIT tests relative to the pre-CS+ baseline period of each CS+ as a function of time. (D) Response bias during each CS+ and post-CS+ period in the initial set of PIT tests relative to the pre-CS+ baseline period of each CS+ as a function of time. Response bias was calculated as in Figure 1. All data points are group means. Error bars represent ±1 standard error of the estimated marginal means from the corresponding fitted linear mixed-effects models. (CS) conditioned stimulus.

As above, we analyzed reward seeking and conditioned approach behavior during PIT testing, after excluding post-seeking food cup approaches (Fig. 2D). Figure 2B shows representative data illustrating the temporal organization of behavior during CS trials. Our initial assessment of the effects of these cues was carried out over two PIT tests to ensure that our sampling was sufficient to discern cue-specific differences in the time course of reward seeking (CS Type × CS Period × CS Time interaction, F_(2,546) = 11.36, P < 0.001; Fig. 3B; full model output in Supplemental Table S7). Significant CS Type × CS Time interactions were detected for both CS (F_(2,273) = 8.68, P < 0.001) and Post-CS periods (F_(2,273) = 3.69, P = 0.026). Both cues that signaled delayed reward delivery (Fixed CS+ and Random Amount CS+) came to elicit a similar pattern of reward seeking, with seeking rates peaking early in the CS period and decreasing substantially before cue offset (Fixed CS+: t₍₉₁₎ = −4.12, P < 0.001; Random Amount CS+: t₍₉₁₎ = −3.86, P < 0.001). In contrast, the Random Time CS+, which signaled that reward could be delivered at any moment, had little effect on lever pressing (t₍₉₁₎ = 0.94, P = 0.351).

These cue-elicited effects on the temporal control over reward seeking were mirrored by opposing changes in the expression of conditioned approach behavior, suggesting that the rats were balancing their time between these competing behavioral strategies. Analysis of these data (Fig. 3C) revealed a significant CS Type × CS Period × CS Time interaction (F_(2,546) = 15.39, P < 0.001; full model output in Supplemental Table S8), which was driven by a considerably stronger CS Type × CS Time interaction during the CS (P < 0.001) than during the post-CS period (P = 0.022). Specifically, whereas rats ramped up their rate of performing the conditioned approach response over time during the Fixed CS+ (t₍₉₁₎ = 4.59, P < 0.001) and the Random Amount CS+ (t₍₉₁₎ = 5.10, P < 0.001), approach behavior was elevated in a persistent manner during the Random Time CS+ (t₍₉₁₎ = −1.36, P = 0.177). When PIT data were analyzed using the response bias measure (Fig. 3D), a significant CS Type × CS Period × CS Time interaction was detected (F_(2,546) = 19.61, P < 0.001; full model output in Supplemental Table S9), confirming that the cues were differentially effective in biasing rats toward the food cup over time, presumably in line with their growing anticipation of noncontingent reward.

The results of initial PIT testing confirmed that normal, drug-naïve rats tend to use information about the expected reward delivery to allocate their time between instrumental reward seeking and conditioned approach behavior. Interestingly, during the first third of the CS presentation, the Random Time CS+ was the least effective trigger of reward seeking and the most effective trigger of conditioned approach. Although all three cues signaled the same amount of reward, the Random Time CS+ differed from the other two cues in that its onset signaled the possibility of imminent, rather than delayed, reward delivery. We hypothesized that this stronger reward expectancy early in the cue period masked its incentive motivational properties, interrupting its ability to invigorate reward seeking. We attempted to unmask this motivational influence by giving the rats four sessions of extinction with the Random CS+ (12 trials per session; no rewards, no lever, and no other CSs). This treatment was effective in reducing that cue's ability to evoke conditioned-approach behavior (Fig. 4A; main effect of session, t₍₁₂₃₆₎ = −5.76, P < 0.001; full model output in Supplemental Table S11). During a subsequent PIT test, CS-evoked reward seeking was significantly increased relative to baseline (intercept: F_(1,546) = 35.53, P < 0.001), an effect that did not depend on CS type (main effect of and interactions involving CS type were not significant, Ps ≥ 0.093; full model output in Supplemental Table S12). Although overall conditioned approach rates were substantially lower for all cues (presumably due to lack of Pavlovian retraining between PIT tests), there was no longer a main effect of CS type (F_(2,546) = 1.71, P = 0.182) that there had been in initial PIT testing (F_(2,546) = 7.21, P = 0.001). The cues also differed in their influence on the timing of conditioned approach behavior during CS periods (CS Type × CS Period × CS Time interaction, F_(2,546) = 16.52, P < 0.001; full model output in Supplemental Table S13), which could account for the slightly different patterns of reward seeking during these trials. In line with this, analysis of the response bias also revealed a significant CS Type × CS Period × CS Time interaction (F_(2,546) = 16.73, P < 0.001; full model output in Supplemental Table S14), notably characterized by a significant increase in a conditioned approach bias during the Fixed and Random-Amount CS+ cues (Ps < 0.001) and a significant decrease during the Random-Time (RT) CS+ (P = 0.008). Altogether, these data show that the Random Time CS+ had, in fact, acquired incentive motivational properties that were comparable to those of the other two cues, but were masked during initial PIT testing due to that cue's tendency to elicit an immediate shift toward the incompatible conditioned approach response.

Figure 4.

The effects of different CS+ types on cue-motivated behavior following the extinction of the Random Time CS+ (Experiment 2). (A) Change in conditioned approach behavior during the Random Time CS+ for each of four extinction sessions (S) of that cue as a function of time. Following extinction, a final PIT test was conducted. (B) Change in reward-seeking behavior during each CS+ and post-CS+ period in the final PIT test relative to the pre-CS+ baseline period of each CS+ as a function of time. (C) Change in conditioned approach behavior during each CS+ and post-CS+ period in the final PIT test relative to the pre-CS+ baseline period of each CS+ as a function of time. (D) Response bias during each CS+ and post-CS+ period in the final PIT test relative to the pre-CS+ baseline period of each CS+ as a function of time. Response bias was calculated as in Figure 1. All data points are group means. Error bars represent ±1 standard error of the estimated marginal means from the corresponding fitted linear mixed-effects models. (CS) conditioned stimulus.

General discussion

Previous reports of drug-induced augmentation of PIT performance have typically used unconventional versions of this task that involve assessing the behavioral influence of relatively strong reward-predictive stimuli, such as cues that reliably signal the specific time of reward delivery (Table 1). We hypothesized that such cues create conflict between two competing responses: active reward seeking and passive conditioned food-cup approach behavior. The current study investigated this possibility and examined whether the impact of repeated cocaine exposure on PIT performance may reflect difficulty resolving such conflict, rather than a heightened state of motivational arousal.

Experiments 1 and 2 confirmed that presenting drug-naïve rats with a CS+ that signaled the timing of a delayed reward caused them to transition between these behaviors over time, shifting from reward seeking (early in the CS+) to the conditioned approach response (late in the CS+), presumably reflecting their mounting expectation of noncontingent reward delivery. In contrast, although rats preexposed to cocaine showed a larger overall increase in CS+ elicited reward seeking, these animals ramped up, rather than inhibited, their reward-seeking behavior over the course of the CS+. Thus, their level of reward-seeking behavior was greatest when this behavior was most maladaptive, in that it demonstrably interfered with their tendency to retrieve the predicted reward using a more efficient (passive) food-cup approach strategy. These findings suggest that cocaine preexposure altered PIT performance, at least in part, by disrupting the cognitive control processes that allow animals to flexibly alternate between incompatible foraging strategies based on changes in reward expectancy. Given previous reports that similar drug treatments can impair response inhibition (Jentsch and Pennington 2014), it is plausible that such an effect contributed to the dysregulation of cue-motivated reward-seeking behavior described here and in related studies. There is also evidence that repeated exposure to cocaine and other drugs can distort time perception (Gibbon et al. 1997; Matell et al. 2004; Teixeira et al. 2013), potentially weakening their ability to use cue-based reward expectancies to adaptively organize their reward-seeking and conditioned approach behavior. This is generally consistent with related findings that cocaine-exposed rats exhibit deficits in using cue-based reward expectations (Schoenbaum and Setlow 2004; Saddoris and Carelli 2014; Saddoris et al. 2016). Similarly, we suggest that the degree to which repeated drug exposure potentiates the expression of cue-motivated reward seeking will depend in large part on the strength of the cue-evoked reward expectancy, which would tend to suppress—not invigorate—reward seeking in drug-naïve rats. Thus, cues that signal the possibility of imminent food delivery, such as the Random Time (10-sec) CS+ used in Experiment 2, may be particularly effective in exposing the exaggerated PIT effect in cocaine-exposed rats.

Although the current findings suggest that repeated cocaine intake does not simply increase rats’ motivational responses to reward-predictive cues, there are various lines of other evidence that repeated drug exposure can sensitize the incentive motivational system (Robinson and Berridge 2003, 2008; Berridge and Robinson 2016). However, the current findings may have implications for how these other findings are interpreted. Specifically, when rats are given pairings between a discrete, localizable CS+, like a lever, and the delivery of reward into a food cup, some animals (sign-trackers) will consistently approach the lever-cue, and others (goal-trackers) will approach the food cup (Flagel et al. 2009; Saunders and Robinson 2010), similar to the way rats behaved in the current experiments. Several studies have shown that preexposing rats to psychostimulants (Doremus-Fitzwater and Spear 2011; Palmatier et al. 2013; Robinson et al. 2015; Saddoris et al. 2016) or alcohol (McClory and Spear 2014; Spoelder et al. 2015) increases their tendency to exhibit a bias toward sign-tracking versus goal-tracking behavior. This is interesting given that one prominent interpretation of sign- and goal-tracking behaviors is that they reflect distinct behavioral processes, with sign-tracking reflecting the tendency to attribute incentive salience to the CS+ (Flagel et al. 2009; Saunders and Robinson 2010), and goal-tracking representing an alternative behavioral strategy that relies on a cognitive expectancy of imminent reward or a stimulus-response habit (Flagel et al. 2009; Robinson et al. 2015).

Although this interpretation supports the conclusion that drug sensitization amplifies the motivational influence of reward-paired cues, it also acknowledges the inherent response conflict in this task. Thus, it is important to consider the possibility that drug preexposure biases rats toward sign-tracking because it disrupts their capacity to inhibit such behavior in order to more efficiently collect the predicted reward via goal-tracking. This is consistent with evidence that animals predisposed toward sign-tracking also show deficits in response inhibition (Flagel et al. 2010). As noted above, cue-evoked reward expectancies may also be weaker or delayed in animals preexposed to drugs (Schoenbaum and Setlow 2004; Saddoris and Carelli 2014; Saddoris et al. 2016). Importantly, just as weak cues are most effective in eliciting instrumental reward-seeking behavior in PIT studies, they are also more likely to elicit sign-tracking behavior (Boakes 1977; Gibbon et al. 1980; Davey et al. 1982; Collins et al. 1983; Anselme et al. 2013; Robinson et al. 2015). In fact, early studies found that the CS+ → reward interval was an important determinant of conditioned approach behavior, such that animals tend to goal-track when the CS+ signals immediate food delivery but are more likely to sign-track when food is expected after a sizable delay (Timberlake et al. 1982). Future research is therefore warranted to investigate whether the tendency for drugs to induce a bias toward sign-tracking reflects an impairment in forming and/or using cue-based reward expectancies.

It should also be emphasized that the current framework does not assume that conditioned food-cup approach behavior is disconnected from the incentive motivational processes that underlie PIT or sign-tracking behavior. In fact, there is considerable evidence that conditioned food-cup approach is highly sensitive to manipulations of relevant physiological need states such as hunger and thirst (Balleine 1994), and may under some conditions be modulated, at least in part, by neurochemical systems implicated in other less ambiguous forms of motivated behavior (Wassum et al. 2011; DiFeliceantonio and Berridge 2012). The view laid out here is that strong reward expectancies can create a state of response conflict between two fundamentally different forms of motivated behavior—exploratory reward seeking and focused reward retrieval—which is normally and adaptively resolved by shifting from the former to the latter. That said, there are a number of reports that animals preexposed to drugs exhibit exaggerated levels of behaviors motivated by natural rewards, including in situations where there is no obvious need for cognitive control to resolve response conflict (Harmer and Phillips 1998; Taylor and Jentsch 2001; Nordquist et al. 2007; Afonso et al. 2009; Mendez et al. 2009; Simon et al. 2009; Shiflett 2012). Such findings suggest that the effects of drug exposure on cognitive control, including the results reported here, might actually be secondary to a more fundamental change in the incentive motivational system. Specifically, the motivational arousal elicited by reward-paired cues may disrupt one's ability to inhibit their reward-seeking behavior, resulting in maladaptive decision making, in accordance with research on the influence of emotion on cognitive control (Verbruggen and De Houwer 2007; Herman et al. 2018). If drug-exposed animals (or humans suffering from drug addiction) exhibit a heightened motivational response to reward paired-cues, this would explain why they also have so much trouble controlling their reward-seeking behavior.

These findings have important implications for understanding the impact of repeated drug exposure on motivational function and how this process contributes to cue-precipitated drug relapse. We have argued that cues paired with natural or drug rewards will only be effective in triggering reward seeking if their motivational influence exceeds any attempts they make to restrain such behavior when it has become maladaptive. Those addicted to drugs regularly attempt to control their drug use because of the perceived (and realized) adverse consequences of this behavior but may lack sufficient control when experiencing intense drug cravings. Further research is needed to parse the specific behavioral and neural mechanisms that underlie such lapses in control, particularly given that both the attribution of incentive salience to drug-paired cues and the capacity to adaptively suppress drug cravings may represent distinct vulnerabilities to relapse.

Materials and Methods

Experiment 1

A full description of the methods for Experiment 1 can be found in the supplementary materials of Ostlund et al. (2014). Briefly, 24 adult male pair-housed Sprague-Dawley rats (Charles River) completed fourteen 30-min sessions of instrumental training (Table 2), in which the final 10 sessions involved response-contingent reward delivery on a random-interval (RI) 45-sec schedule of reinforcement. Subsequently, during 14 sessions of Pavlovian training, the rats learned to associate a 30-sec auditory cue with a 3-pellet reward, delivered at cue offset (CS+). Each session involved 10 pairings between the CS+ and reward. The final three sessions also included two presentations of a second auditory cue (CS−) that was not explicitly paired with reward. Then, for 12 d, the rats underwent a drug sensitization procedure. Half of the rats (Vehicle Group) were injected daily with sterile saline (1 mL/kg), while the other half (Cocaine Group) received alternating injections of cocaine (15 mg/kg) and saline (e.g., Day 1: Cocaine, Day 2: Saline, Day 3: Cocaine, etc.). Rats spent 45 min in the operant chamber following injection, and, for the Cocaine Group, the chamber was manipulated so that the rats were in different contexts following cocaine and saline injections. Ten days following the final injection, all rats received 3 d of instrumental retraining (RI-45) followed by an extinction session prior to the PIT test. Before test, each rat was injected with sterile saline. During the test session, the CS+ and CS− were presented four times each, separated by a 3.5-min interstimulus interval (ISI). For this PIT test, the operant chambers for half of the Cocaine Group were set to the context paired with cocaine during the drug sensitization phase, but, as reported by Ostlund et al. (2014), context did not significantly affect expression of lever pressing during the PIT test in the Cocaine Group. While Ostlund et al. (2014) gave all rats two PIT tests, only the data from the first test are reanalyzed here.

Experiment 2

Materials

Animals and apparatus

Thirty-one experimentally naïve male Long Evans rats (Charles River) were used in this experiment. They arrived at the facility (University of California, Irvine) at ∼12 wk of age. They were pair-housed in a colony room set to a standard 12:12 h light:dark schedule. The rats were tested during the light phase. Water was always provided ad libitum in the home cages. Rats were fed between 10 and 14 g of standard laboratory chow per day during the experiment to maintain them at ∼85% of their estimated free-feeding bodyweight. Husbandry and experimental procedures were approved by the UC Irvine Institutional Animal Care and Use Committee (IACUC) and were in accordance with the National Research Council Guide for the Care and Use of Laboratory Animals.

The experiment was conducted in 16 operant chambers (Med-Associates), each housed within sound-attenuating, ventilated boxes. Each chamber was equipped with a stainless-steel grid floor; two stainless steel walls (front and back); and a transparent polycarbonate side-wall, ceiling, and door. Two pellet dispensers, mounted on the outside of the operant chamber, were equipped to deliver 45-mg grain-based food pellets (Bio-Serv) to a recessed food cup centered on the lower section of the front wall. Head entries into the food receptacle were transduced by an infrared photobeam. A retractable lever was located to the left of the food cup, on the front wall. The chamber was also equipped with a house light centered at the top of the back wall. Auditory stimuli were presented to the animal via a speaker located on the back wall. Experimental events were controlled and recorded with 10-msec resolution by the software program MED-PC IV (Tatham and Zurn 1989).

Procedure

Magazine and instrumental training. All sessions of all phases began with the onset of the houselight. In each of two 30-min sessions of magazine training, food pellets were delivered on a RT 60-sec schedule of food deliveries. During lever-press training, rats were continuously reinforced for pressing the lever until 30 food pellets had been delivered. Initial lever-press training lasted for 2–3 sessions, until each rat had earned 30 food pellets in at least one session. During subsequent instrumental training sessions, food delivery was contingent on the first lever press after t seconds had elapsed since the previous food delivery. The value of t was reflective in the label of the RI schedule of reinforcement (i.e., RI t s), and was drawn from an exponential distribution with a mean of t. In order, instrumental training (total sessions) involved RI-5 (1), RI-15 (1), RI-30 (2), and RI-45 (14) schedules of reinforcement. Each session lasted for ∼30 min.

Pavlovian training. Pavlovian training involved exposure to three 30-sec conditioned stimuli (CS; 80-dB white noise, 3-kHz tone, 10-Hz clicker) paired with reward (food pellets). The Fixed CS+ reliably signaled that three food pellets would be delivered at the cue offset. The Random Amount CS+ signaled that a variable amount of reward (0–6 pellets) would be delivered at cue offset. The Random Time CS+ signaled that one pellet was delivered to the rat, on average, every 10 sec, according to an exponentially distributed random time-10 sec schedule.

In each session, a 60-sec interval preceded onset of the first CS. There was a 120-sec ISI between consecutive CS presentations (i.e., between previous CS offset and subsequent CS onset), and a 60-sec interval following that final CS presentation prior to the end of the session. Pavlovian training lasted for 25 sessions. The first nine sessions involved four presentations of each CS and lasted 30 min. To facilitate learning, the final 16 sessions involved eight presentations of each CS and lasted 60 min. In each session, the order of CS presentation was pseudo-random, such that each CS occurred once within each consecutive block of three CS presentations.

Extinction. To facilitate expression of PIT at test, rats were administered a 30-min session of lever-press extinction, which was identical to the instrumental training phase described above, except that reward was not delivered.

Pavlovian-to-instrumental transfer. During the initial round of testing, rats received two Pavlovian-to-instrumental (PIT) transfer test sessions. The left lever was inserted for the duration of each 32-min test session. The ISI was 150 sec, and a 6.25-min interval preceded onset of the first CS (i.e., 5 min plus one half of the ISI). There were three presentations of each CS, using the same random CS ordering used in Pavlovian conditioning sessions. Food was not delivered at test. Prior to the second test, the rats received two sessions of instrumental training, followed by one session of instrumental extinction. After initial testing, rats were given four sessions of extinction with the Random Time CS+. Each session consisted of 12 nonreinforced presentations of that cue (120-sec ISI). Rats were then given a final PIT test, as described above.

Data analysis

All summary measures were obtained from the raw data using MATLAB (The MathWorks) and analyzed with mixed-effects regression models (Pinheiro and Bates 2000), a powerful analytical framework that is both well established and highly recommended for behavioral research (Boisgontier and Cheval 2016). Mixed-effects models are comparable to repeated-measures regression analyses, and allow for parameter estimation per manipulation condition (fixed effects) and the individual (random effects) (Pinheiro and Bates 2000; Hoffman and Rovine 2007; Bolker et al. 2008; Schielzeth and Nakagawa 2013; Hoffman 2015). Mixed-effects regression models (1) effectively handle missing data and (2) permit the inclusion of categorical and continuous predictors in the same analysis, thus allowing detection of group-level changes across ordered data samples (i.e., continuous time points) while also accounting for corresponding individual differences. All relevant fixed-effects factors were included in each model, and model selection of random-effects terms was performed using the Akaike information criterion (AIC), in which the doubled negative log likelihood of the model is penalized by twice the number of estimated parameters (see Burnham and Anderson 1998). Categorical predictors were effects-coded (i.e., codes sum to 0), and continuous predictors were mean-centered (Kreft et al. 1995).

The primary dependent variables were lever presses and food-cup approaches. Lever pressing was used to operationally define “reward-seeking behavior.” Food-cup approaches (i.e., transduction of the infrared beam within the food cup) that occurred within 2.5 sec of a lever presses were regarded as “post-seeking approaches” (Fig. 2A–D) and were therefore excluded from our analysis of “conditioned food-cup approaches.” This 2.5-sec cutoff period is also appropriate for identifying the elevated rate of post-seeking food cup approaches during both reinforced and nonreinforced sessions of lever pressing (i.e., in the absence of explicit food-paired cues; data not shown).

One rat from the Cocaine Group of Experiment 1 was removed from all analyses due to excessive conditioned approaches during CS+ trials at test, which exceeded three scaled median absolute deviations from the median (see Leys et al. 2013). For Experiment 1, the final three sessions of Pavlovian training were used to assess conditioned approach behavior during CS+ and CS− trials relative to pre-CS baseline periods. For Experiment 2, the final five sessions of Pavlovian training were analyzed. Analysis of approach behavior during the Random Time CS+ in Pavlovian training only included the time within each trial prior to the first food reward delivery. Analysis of reward seeking, post-seeking approach, and conditioned approach in PIT testing included all trials. One PIT test made up the test data in Experiment 1. The initial round of PIT testing in Experiment 2 included two tests, and the second round of testing included one additional test. Analysis of the extinction phase of Experiment 2 included all trials from each session.

All statistical analyses were conducted in MATLAB (The MathWorks). The alpha level for all tests was 0.05. Effect size was represented by the unstandardized regression coefficient (Baguley 2009), reported as b in model output tables. The source of significant interactions was determined by secondary mixed-effects models identical to those described but split by the relevant factor of interest. For analyses in which a factor had more than two levels (i.e., CS type in Experiment 2), interactions were reported in-text as the results of ANOVA F-tests (i.e., whether the coefficients for each fixed effect were significantly different from 0).