Modulation of cue-triggered reward seeking by cholinergic signaling in the dorsomedial striatum

Abstract

The dorsomedial striatum (DMS) has been strongly implicated in flexible, outcome-based decision making, including the outcome-specific Pavlovian-to-instrumental transfer effect (PIT), which measures the tendency for a reward-predictive cue to preferentially motivate actions that have been associated with the predicted reward over actions associated with different rewards. Although the neurochemical underpinnings of this effect are not well understood, there is growing evidence that striatal acetylcholine signaling may play an important role. The current study investigated this hypothesis by assessing the effects of intra-DMS infusions of the nicotinic antagonist mecamylamine or the muscarinic antagonist scopolamine on expression of specific PIT in rats. These treatments produced dissociable behavioral effects. Mecamylamine infusions enhanced rats’ tendency to use specific cue-elicited outcome expectations to select whichever action was trained with the predicted outcome, relative to their performance when tested after vehicle infusions. In contrast, scopolamine infusions appeared to render instrumental performance insensitive to this motivational influence of reward-paired cues. These drug treatments had no detectable effect on conditioned food-cup approach behavior, indicating that they selectively perturbed cue-guided action selection without producing more wide-ranging alterations in behavioral control. Our findings reveal an important role for DMS acetylcholine signaling in modulating the impact of cue-evoked reward expectations on instrumental action selection.

The dorsomedial striatum (DMS) is strongly implicated in the acquisition and flexible control of reward-seeking behavior (Ragozzino, 2007; Yin et al., 2008; Hart et al., 2014; Goodman & Packard, 2016), particularly when this involves the use of detailed expectations of behavioral goals or outcomes (Yin et al., 2005a; Yin et al., 2005b; Corbit & Janak, 2007b; Lex & Hauber, 2010; Shiflett et al., 2010; Corbit et al., 2013; Li et al., 2016). For instance, disrupting normal DMS function in rats impairs their ability to learn about new action-outcome contingencies or use previously encoded associations when selecting actions based on expected outcome value (Yin et al., 2005a; Yin et al., 2008). The DMS also appears to mediate flexible action selection based on cue-evoked reward expectations (Corbit & Janak, 2007b), an aspect of motivated behavior which can be selectively assayed using the outcome-specific Pavlovian-to-instrumental transfer (PIT) task (Kruse et al., 1983). Outcome-specific PIT studies typically involve training rats during separate experimental phases with two different stimulus-outcome contingencies (S1—O1 & S2—O2) and two different action-outcome contingencies (A1—O1 & A2—O2). Subsequent test sessions are then used to assess the impact of noncontingent cue presentations on ongoing instrumental performance. Under normal conditions, presenting such a cue will selectively bias performance towards whichever action was trained with the same outcome as that cue (i.e., S1 will increase performance of A1 relative to A2). However, transiently inactivating the DMS during PIT testing has been shown to disrupt the outcome selectivity of this effect, resulting in a nonspecific increase in instrumental performance during cue presentations (Corbit & Janak, 2007b). Although studies such as these are typically conducted in rodents, recent findings suggest that expression of PIT in humans engages a homologous neural circuitry (Bray et al., 2008; Prevost et al., 2012). Interestingly, there is growing evidence that the processes underlying PIT contribute to drug seeking (Corbit & Janak, 2007a; Hogarth et al., 2007; LeBlanc et al., 2012; Garbusow et al., 2016) and may be compromised in certain maladaptive states such as heightened stress (Quail et al., 2016) and schizophrenia (Morris et al., 2015).

Recent findings also indicate that cholinergic activity within the striatum plays a crucial role in the expression of flexible reward-seeking behavior (Ragozzino, 2007). For instance, selective lesions of DMS cholinergic interneurons, the primary source of striatal acetylcholine, disrupt the expression of outcome-specific reinstatement (Matamales et al., 2016), which involves selecting actions based on the noncontingent presentation of actual rewards – as opposed to cue-evoked reward expectations (Ostlund & Balline, 2007). Although it was recently shown that systemic blockade of either muscarinic (mAChR) or nicotinic (nAChR) acetylcholine receptors prior to specific PIT testing disrupts the expression of this behavioral effect (Ostlund et al., 2014a), it remains unknown how acetylcholine signaling within the DMS contributes to action selection based on cue-evoked reward expectations. The current study investigated this issue by determining the effects of intra-DMS infusions of mecamylamine (selective mAChR antagonist) and scopolamine (selective nAChR antagonist) on expression of outcome-specific PIT.

Methods

Subjects

15 adult male Sprague-Dawley rats (300 – 425 g; Charles River Laboratories) were housed in a climate-controlled vivarium and tested during the light phase of a 12:12 h light:dark cycle. Rats were pair-housed up until surgery, after which they were individually housed for the remainder of the experiment. A food-restriction schedule (~10–14g/rat/day) was in place during training and testing to maintain rats at approximately 85% their free-feeding body weight. Ad libitum water was continuously provided in their home cages. All procedures were approved by the University of California, Los Angeles Institutional Animal Care and Use Committee, and were performed in accordance with the National Research Council’s Guide for the Care and Use of Laboratory Animals.

Apparatus

Behavioral testing was conducted in eight identical Med Associates (East Fairfield, VT) chambers, housed within light- and sound-attenuating cubicles. Each chamber contained two retractable levers, located on either side of a recessed food cup, into which 45-mg grain-based food pellets (Bioserv, Frenchtown, NJ) or 20% sucrose solution (0.1ml) could be delivered. A photobeam sensor detected head entries into the food cup. A house light (24V, 3W) provided continuous illumination during all sessions. White noise and clicker (10Hz) generators were used to deliver auditory stimuli (~70 dB).

Pavlovian Conditioning

Rats received eight once-daily sessions of Pavlovian conditioning in which each of two auditory-conditioned stimuli (CSs; noise or clicker; 2min each) was paired with a different food outcome (grain pellets or sucrose solution). Pavlovian stimulus-outcome contingencies were counterbalanced, such that half of the rats were given clicker-pellet and noise-sucrose pairings, and half were given clicker-sucrose and noise-pellet pairings. Each session lasted approximately 40min and consisted of four clicker and four noise trials, separated by a variable interval (mean=3.125min; range=2.25–4min). During each stimulus, the appropriate outcome was delivered on a random time 30-s schedule, resulting in an average of four outcome deliveries per trial.

Instrumental Conditioning

Rats then received 11 days of instrumental training. Each response (left vs. right lever press) was reinforced with a different outcome (pellet or sucrose). Responses were trained in separate sessions, such that rats received two sessions per day, with session order alternating over days. Action-outcome contingencies were counterbalanced with Pavlovian contingencies. Thus, half of the rats in each Pavlovian training condition were trained with left press-pellet and right press-sucrose contingencies, whereas the remaining half were trained with left press-sucrose and right press-pellet contingencies. Each session lasted until 30 rewards were earned or 30min had elapsed, whichever came first. Over days, the reinforcement schedule was gradually shifted from continuous reinforcement (2 days) to increasingly more effortful random ratio (RR) schedules (3 days each with RR-5, -10, and -20) to establish robust instrumental performance.

Surgery

After initial training, rats underwent asceptic stereotaxic surgery for bilateral guide cannula implantation under isoflurane anesthesia. Stainless steel guide cannulae (22-gauge, Plastics One, Roanoke, VA) were positioned such that their tips would be 1 mm above the intended infusion site in the DMS (AP: −0.4 mm from Bregma, ML: ± 2.6 mm from Bregma, DV: −4.2 mm from skull surface), following previous studies (Yin et al., 2005b; Shiflett et al., 2010). Rats were given 7 d to recover from surgery before further testing. They were given ad libitum home chow for 5 d after surgery, at which point they were returned to the food restriction regimen for the remainder of the experiment.

Pavlovian-to-instrumental transfer testing

After recovering from surgery, rats were given 1 d of Pavlovian retraining followed by 2 d of instrumental retraining (RR-10, then RR-20), as described above. Rats were then administered a 1 h extinction session consisting of continuous nonreinforced access to both levers prior to PIT testing. Each test session involved 30 min of non-reinforced access to both levers with intermittent presentations of the reward-paired cues (also nonreinforced). Each cue (clicker and noise) was presented twice in a noncontingent manner for 2 min using an alternating trial order (clicker-noise-clicker-noise), with the first trial beginning 4 min into the session and trials separated by a fixed 4-min interval. Before each test, rats were given bilateral intra-DMS injections (0.5 µl/site) of vehicle (artificial cerebrospinal fluid), scopolamine hydrochloride (10 µg/site; Tocris Bioscience), or mecamylamine hydrochloride (10 µg/site; Tocris Bioscience) using a 0.5 µl/min flow rate. Injectors were left in place for an additional 1 min to facilitate drug diffusion. Rats were then placed in the chamber and the test session was initiated 1 min later. Drug doses and injection-to-test intervals were based on previous studies to maximize potential to alter reward-motivated behavior and minimize gross motor effects (Pratt & Kelley, 2004; Tzavos et al., 2004; Collins et al., 2016). Each rat was administered 4 PIT tests to provide a fully within-subjects assessment of each drug’s effect. During the first pair of tests, half of the rats were tested on scopolamine and half were tested on mecamylamine. This drug treatment occurred prior to Test 1 or Test 2 (counterbalanced with drug type and training contingencies), with the alternate test serving as a vehicle control. The same procedure was repeated for the second pair of tests except that the drug conditions were reversed (e.g., a rat given mecamylamine in Test 1 and vehicle in Test 2 would be given scopolamine in Test 3 and vehicle in Test 4). Retraining began 48 h after each test to allow for drug clearance. Prior to Tests 2–4, rats were given 1 d of Pavlovian conditioning and 3 d of instrumental conditioning (RR5, RR10, RR20), followed by 1d of response extinction (60 min), as described above.

Histology

After testing, rats were given an overdose of sodium pentobarbital (100 mg/kg, i.p.). Their brains were removed, postfixed, and cryoprotected in a 30% sucrose–formalin solution, and cut into 50 µm coronal sections across the DMS. Sections were mounted on glass slides, stained with cresyl violet, and analyzed under a light microscope to determine injector placements. All injector placements were confirmed to be within the DMS (see Figure 1 Paxinos and Watson (2005)).

Figure 1.

Coronal sections (adapted from Paxinos and Watson, 2005) showing microinfusion cannula placements in the dorsomedial striatum (DMS). Circles represent estimated tip of injector tip. Numbers indicate distance (mm) from bregma.

Statistical analysis

Data were analyzed using repeated-measures ANOVAs or paired t-tests, as appropriate. We assessed the development of conditioned approach behavior by comparing the mean rate (responses/min) of food cup beam breaks during pre-CS and CS (prior to first reward delivery) periods across Pavlovian conditioning sessions (CS period × Session ANOVA). The acquisition of instrumental performance was assessed as the mean rate of lever pressing across conditioning sessions. Analysis of lever press rates during PIT testing focused on difference scores reflecting CS-induced changes in response rate (CS – pre-CS), which were calculated separately for each action based its relationship to the CS (Δ Same vs. Δ Different). Because rats were tested with concurrent access to both actions, a change in performance in one action may have impacted performance of the alternate action through response competition. Therefore, our response-specific difference scores are appropriate for evaluating how CS presentations influence action selection, as opposed to their ability to generally invigorate reward-seeking behavior (cf. Ostlund & Maidment, 2012). Analysis of these data included factors for Action and Drug treatment (Action × Drug ANOVA). Data were collapsed across the two vehicle tests after preliminary analysis confirmed that the specific PIT score was unaffected by this counterbalancing condition (F_1,14 = .0.71, p = 0.41). The rate of food cup entry during PIT testing was analyzed as a function of Cue period and Drug condition (CS period × Drug ANOVA). The source of significant omnibus interactions in 2 × 3 ANOVAs was determined through assessment of partial interactions involving each combination of Drug treatment (2 × 2 ANOVAs). Significant interactions in 2 × 2 ANOVAs were assessed further through multiple pairwise comparisons (paired t-test, two-tailed) as advised by Levin et al. (1994) based on a logical extension of Fisher’s protected least significant difference (PLSD) procedure for controlling familywise Type I error rates. Statistical significance was set at p < 0.05 for all analyses.

Results

Pre-training

Rats were first given Pavlovian conditioning with two distinct stimulus-outcome relationships (S1-O1 & S2-O2). During these sessions, the rats acquired conditioned anticipatory food-cup approach behavior (Figure 2a), indicating that the training regimen was effective. Analysis of these data detected a significant main effect of Session (F_7,105 = 29.71, p < 0.001) and CS period (F_1,15 = 51.00, p < 0.001), as well as a significant Session × CS period interaction (F_7,105 = 17.32, p < 0.001). In the next phase of the study, rats were trained to perform two different lever-press actions for different food rewards (R1-O1 & R2-O2). They readily acquired this behavior, steadily increasing their rate of lever pressing over sessions as the schedule of reinforcement increased from FR1 to RR20 (Figure 2b), as indicated by a significant main effect of Session (F_10,140 = 248.07, p < 0.001).

Figure 2.

A, Results of Pavlovian conditioning, plotted as the mean rate of food cup entries during CS and pre-CS (baseline) periods over training sessions. B, Results of instrumental conditioning, plotted as the mean rate of lever pressing over training sessions. Error bars represent +/− SEM.

Pavlovian-to-instrumental transfer

Rats underwent a series of PIT tests to determine the effect of intra-DMS administration of scopolamine and mecamylamine on the outcome-specific influence of reward-paired cues on reward-seeking behavior. The results of PIT testing are presented in Figure 3. Inspection of these data indicate that rats showed a typical outcome-specific bias in lever pressing during CS presentations, choosing to perform whichever action was trained with the same outcome as the current CS over the alternate action (Different). This cue-evoked shift in performance appeared to be augmented when DMS nAChRs were blocked with mecamylamine and attenuated when DMS mAChRs were blocked with scopolamine (Figure 3a). Initial analysis of these data revealed that there was no effect of drug treatment on pre-CS (baseline) response rates (F_2,28 = 0.221, p = 0.80; see Table 1 for full ANOVA table). To focus more directly on the influence of CS presentations on responding, we calculated the change in response rate for each action (e.g., Δ Same), relative to response-specific baseline values (CS – Pre-CS). Analysis of these data (Figure 3b) detected a significant main effect of Drug (F_2,28 = 5.44, p = 0.01) and a significant Drug × Action interaction (F_2,28 = 4.77, p = 0.017). To identify the source of this interaction, separate Drug × Action ANOVAs were run for the 3 combinations of drug treatments. A significant Drug × Action (partial) interaction was detected for ANOVA comparing tests vehicle and mecamylamine (F_1,14 = 4.93, p = 0.043). Pairwise comparisons of this interaction based on Fisher’s PLSD (see Methods) found a significant simple effect of Action for both the vehicle (t₁₄ = 2.22, p = 0.044) and mecamylamine (t₁₄ = 7.83, p = 0.004) tests. There was a marginal effect of Drug for action Same (t₁₄ = −2.07, p = 0.058) but not for action Different (t₁₄ = 0.49, p = 0.633). Together with the significant interaction, these results suggest that intra-DMS mecamylamine enhanced the magnitude of the outcome-specific PIT effect (i.e., the difference between actions Same and Different). A significant Drug × Action (partial) interaction was also detected for the ANOVA comparing tests mecamylamine and scopolamine (F_1,14 = 7.05, p = 0.019). Pairwise comparisons found no effect of Action for the scopolamine test (t₁₄ = −0.94, p = 0.36), demonstrating that the influence of the CS over action selection was abolished by mAChR blockade. Consistent with this, a significant effect of Drug was detected for action Same (t₁₄ = 3.67, p = 0.002) but not Different (t₁₄ = −0.64, p = 0.54). The final Drug × Action ANOVA comparing tests vehicle and scopolamine did not result in a significant interaction (F_1,14 = 2.23, p = 0.16).

Figure 3.

Results of outcome-specific Pavlovian-to-instrumental transfer (PIT) testing. A, Mean rate of lever pressing during pre-CS and CS periods, plotted separately for each action based on its relationship to the CS (Same vs. Difference). B, Difference scores showing CS-induced changes in performance of each Action (CS – Pre-CS). C. Mean rate of food cup entry during pre-CS and CS periods. For all graphs, data are plotted separately for vehicle (black), mecamylamine (red) and scopolamine (blue) tests. Error bars show SEMs (capped bars A–C). Floating lines in B show the standard error of the difference between Δ Same andΔ Different, which reflects the outcome-specific influence of the CS on lever pressing (see text). * indicates p < 0.05, ** p < 0.01 for pairwise comparisons. # indicates significant partial interaction between subset of drug conditions and Action.

Table 1.

Results of 3-way repeated measures ANOVA in Figure 3a.

Factors (levels)	d.f.	F-value	P-value
Drug (3)	2,28	0.899	0.419
Action (2)	1,14	8.274	0.012*
CS period (2)	1,14	0.392	0.541
Drug × Action	2,28	0.274	0.762
Drug × CS period	2,28	5.439	0.010*
Action × CS period	1,14	3.104	0.100
Drug × Action × CS period	2,28	4.766	0.017*

A 3-way repeated-measures ANOVA (Drug × Action × CS period) was performed on the average rate of lever pressing during specific PIT testing. See Figure 3a for means and SEMs and see text for main analyses of cue-related changes in press rate.

^*P < 0.05.

Although our manipulations of DMS acetylcholine transmission produced opposing effects on cue-evoked instrumental reward seeking, they had little impact on cue-evoked anticipatory food cup approach behavior (Figure 3c). Analysis of these data revealed that food cup approach rates were elevated during CS trials, relative to the pre-CS period (main effect of CS period: F_1,14 = 72.94, p < 0.001). Although there was a significant main effect of Drug (F_2,28 = 3.43, p = 0.047), there was no interaction between Drug and CS period (F_2,28 = 1.48, p = 0.99), indicating that drug effects were not specific to responding triggered by the CS. Interestingly, this main effect of Drug appeared to be driven by an increase in food cup approach responses (collapsing across CS period) during the scopolamine test (Fisher’s PLSD, effect of Drug comparing saline and scopolamine, F₁₄ = 7.96, p = 0.014). Such findings strongly suggest that the scopolamine-induced attenuation in PIT performance was not the result of a nonspecific motor impairment.

Discussion

The current study tested the dependence of outcome-specific PIT performance on cholinergic signaling at mAChRs and nAChRs within the DMS. We found that muscarinic acetylcholine receptor blockade disrupted the use of cue-evoked reward expectations when selecting between reward-seeking actions. Blocking nicotinic acetylcholine receptors had the opposite effect, enhancing this aspect of cue-motivated reward seeking. These findings provide evidence that DMS acetylcholine plays an important role in mediating the influence of reward-paired cues on instrumental reward seeking.

While the current study demonstrates that DMS acetylcholine makes important contributions to specific PIT, the complexity of the striatal cholinergic system and its interactions with other neurochemical systems (Calabresi et al., 2000; Goldberg et al., 2012) will make it difficult to determine the specific mechanisms underlying behavioral effects described here. However, several possibilities are worth noting. For instance, it is well established that antagonizing M1 mAChRs decreases the excitability of striatal medium spiny projection neurons (Calabresi et al., 2000). The resulting disruption of DMS output would readily account for the attenuated PIT performance that we observed following intra-DMS injections of scopolamine. Likewise, nAChRs expressed by striatal inhibitory interneurons are well positioned to regulate striatal output (English et al., 2012; Lim et al., 2014). Blocking these nAChRs should weaken inhibitory tone on DMS projection neurons. It is therefore possible that the resulting facilitation of DMS output contributes to the augmentation of PIT performance produced by mecamylamine infusions. Our scopolamine injections may have also attenuated excitatory drive in the DMS via activation of mAChRs on glutamatergic terminals (Calabresi et al., 2000; Goldberg et al., 2012).

The current findings could also be explained by a more complex interaction between striatal acetylcholine and dopamine systems. For instance, a recent study (Collins et al., 2016) on the role of nucleus accumbens core acetylcholine in nonspecific PIT performance found that intra-core injections of scopolamine and mecamylamine had distinct behavioral effects, attenuating and augmenting expression of PIT, respectively, which is strikingly similar to the present findings. Interestingly, this earlier study found that these pharmacological manipulations of cholinergic signaling also altered cue-related dopamine signaling in the nucleus accumbens core, with scopolamine blunting and mecamylamine enhancing dopamine release. Given the well-established role of dopamine in the nonspecific component of PIT (Dickinson et al., 2000; Lex & Hauber, 2008; Wassum et al., 2011; Ostlund & Maidment, 2012; Pecina & Berridge, 2013; Wassum et al., 2013; Ostlund et al., 2014b; Hebart & Glascher, 2015; Aitken et al., 2016; Collins et al., 2016), such findings suggest that acetylcholine activity in the nucleus accumbens core may modulate cue-motived behavior through its known regulation of dopamine release at striatal terminals. This hypothesis is bolstered by slice voltammetry studies showing that nAChRs and mAChRs play opposing roles in regulating terminal dopamine release in both the ventral and dorsal striatum (Sulzer et al., 2016). Although a similar mechanism may underlie the current results, the finding that dopamine-depleting lesions of the DMS produce only modest, nonsignificant effects on specific PIT (Pielock et al., 2011) raises questions about the importance of DMS dopamine in this aspect of cue-motivated behavior.

Although the current findings and previous results (Collins et al. 2016) indicate that striatal nAChRs exert a net suppressive influence over cue-motivated behavior, our previous finding that systemic blockade of nAChRs disrupts specific PIT suggests that nAChRs at extra-striatal sites facilitate expression of this behavioral effect (Ostlund et al., 2014a). Potential targets for future studies include regions rich in nAChRs which have been implicated in specific PIT, such as the orbitofrontal cortex, mediodorsal striatum, basolateral amygdala, and ventral tegmental area (Blundell et al., 2001; Corbit & Balleine, 2005; Corbit et al., 2007; Ostlund & Balleine, 2007; 2008; Shiflett & Balleine, 2010; Prevost et al., 2012; Leung & Balleine, 2015; Malvaez et al., 2015; Parnaudeau et al., 2015).

While reward-predictive cues normally provide an adaptive influence over action selection and initiation, there is great interest in the possibility that aberrant Pavlovian learning contributes to the development of pathological drug seeking, overeating and other disorders of behavioral control (Everitt et al., 2001; Robinson & Berridge, 2008). For individuals attempting to quit using drugs, drug cues can promote intense drug craving and trigger relapse (O'Brien et al., 1992; Epstein et al., 2009; Tiffany & Wray, 2012). Interestingly, it has been shown that rats given repeated exposure to cocaine or amphetamine are more sensitive to the response-invigorating effects of food-paired cues during PIT testing (Wyvell & Berridge, 2001; Saddoris et al., 2011; LeBlanc et al., 2013; Shiflett et al., 2013; LeBlanc et al., 2014; Ostlund et al., 2014b), suggesting that such drugs are capable of producing long-lasting adaptations in the neural circuitry underlying Pavlovian incentive motivation. Most studies on this topic have applied relatively simple PIT tasks that do not assay the influence of outcome expectations on response selection. However, one recent study using a specific PIT task (Shiflett, 2012) found that repeated amphetamine exposure disrupts the outcome specificity of this effect, which is generally in line with a wider body of research showing that chronic drug exposure can impair certain aspects of outcome encoding (Stalnaker et al., 2009). Such findings may be relevant to understanding self-reports of generalized craving for palatable foods and other non-drug rewards by illicit drug users (Picozzi et al., 1972; Gambera & Clarke, 1976; Weiss, 1982; Nolan & Scagnelli, 2007), smokers (Spring et al., 2003; Mahler & de Wit, 2005), alcoholics (Moorhouse et al., 2000), and Parkinson’s disease patients undergoing dopamine agonist treatment (Giovannoni et al., 2000). The current results identify DMS acetylcholine as a neurochemical target for future studies investigating the maladaptive influence of reward-predictive cues on decision making.