The Role of the Nucleus Accumbens in Learned Approach Behavior Diminishes with Training

Abstract

Nucleus accumbens dopamine plays a key role in reward-directed approach. Past findings suggest that dopamine’s role in the expression of learned behavior diminishes with extended training. However, little is known about the central substrates that mediate the shift to dopamine-independent reward approach. In the present study, rats approached and inserted the head into a reward compartment in response to a cue signaling food delivery. On days 4 and 5 of 28-trial-per-day sessions, D1 receptor antagonist R(+)-7-chloro-8-hydroxy-3-methyl- 1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepine hydrochloride (SCH23390) infused to the NAc core reduced the probability and speed of cued approach. The disruptive effect of D1 receptor blockade was specific to the nucleus accumbens core and not seen with drug infusions to nearby dopamine target regions. In rats that received drug infusions after extended training (days 10 or 11), accumbens core D1 receptor blockade produced little effect on the expression of the same behavior. These results could have been due to a continued accumbens mediation of cued approach even after the behavior had become independent of accumbens D1 receptors. However, accumbens core ionotropic glutamate receptor blockade disrupted cued approach during early but not late stages of training, similar to the effects of D1 antagonist infusions. The results suggest that with extended training, a nucleus accumbens D1-dependent behavior becomes less dependent not only on nucleus accumbens D1 transmission but also on excitatory transmission in the nucleus accumbens. These findings fill an important gap in a growing literature on reorganization of striatal function over the course of training.

Graphical Abstract

Past work points to a key role for the NAc in the expression of conditioned reward-directed behavior. Here we found that while NAc core dopamine and glutamate transmission critically mediate cued approach during early stages of training, their roles in behavioral expression diminish with extended training. These findings fill a key gap in the growing literature on reorganization of striatal function over the course of training.

INTRODUCTION

Dopamine (DA) plays a key role in striatal plasticity (Gerfen and Surmeier, 2011; Wickens et al., 2003; Yagishita et al., 2014) and learning (Eyny and Horvitz, 2003; Chang et al., 2017). DA also modulates the expression of previously learned behaviors (Salamone and Correa, 2012; du Hoffmann and Nicola, 2014), perhaps reflecting DA modulation of striatal responsiveness to cortical inputs at synapses that were previously modified during learning (Horvitz, 2009; Cerovic et al., 2013). Of particular relevance to the present investigation, rats trained to press a lever for food upon presentation of a reward-predictive cue show cue-elicited excitation of nucleus accumbens (NAc) neurons, and both the NAc neuronal response and the cued approach latency depend upon NAc DA transmission (du Hoffmann and Nicola, 2014).

Previous work has suggested that the role of D1 transmission in cued approach may be transient. For instance, after early training to respond to a discrete cue signaling reward availability in a food compartment, systemic D1 receptor blockade decreases the expression of cued approach. However, after extended training, the same cued approach is nearly unaffected by D1 receptor blockade (Choi et al., 2005; Bespalov et al., 2007). D2 receptor blockade, at doses that produce a similar amount of locomotor suppression, produces minimal disruption of cued approach likelihood or latency regardless of learning stage (Choi et al., 2005; Choi et al., 2009). It is not known whether this transient role for D1 receptors in the expression of reward-directed behavior reflects a reduction in the behavioral role of D1 transmission within the NAc core. If so, what accounts for the shift from D1-dependent to D1-independent performance after extended training? Does the well-trained behavior become independent of NAc core D1 transmission specifically, or of NAc core activity more generally?

Here, examinations of cued approach during early and late stages of training reveal that D1 transmission within the NAc core plays a critical but transient role in cued approach. If the reduced involvement of NAc core D1 transmission after extended training reflects a reduced involvement of the NAc core in behavioral performance, ionotropic glutamate (GLU) receptor blockade within the region should also disrupt behavior only during early stages of training. Alternatively, if the NAc core plays a continuing role in behavioral expression even after the role of D1 transmission has diminished, cued approach should remain dependent upon GLU transmission even after extended training. The results reported here are consistent with the former hypothesis, i.e., the NAc core plays a diminishing role in the expression of the learned behavior with extended training.

MATERIALS AND METHODS

Subjects

Seventy-eight male Sprague Dawley rats weighing 270–310 g (Charles River Laboratories, Wilmington, MA) at the time of arrival were used in the experiments. Rats were initially housed in pairs within clear Plexiglas cages (20 cm high × 26 cm wide × 46 cm deep) in an animal colony. The colony was maintained at approximately 23°C with a 12 h light/dark cycle (lights on at 9:00 A.M.). Food (Purina Lab Chow) and water were available ad libitum for five to seven days of acclimatization. Rats were handled daily and housed in separate cages from the day of surgery until completion of the experiments. After 3–5 days recovery from surgery, rats were placed on a 22 h food-restriction schedule, and were weighed and handled every three days. Under this regimen, rats maintained at least 85% of their ad libitum weight for the remainder of the study. All experiments were conducted according to the ethical guidelines approved by the City College of New York Institutional Animal Care and Use Committee.

Apparatus

Behavioral training sessions were carried out in test chambers (29 cm high × 29 cm wide × 25 cm deep; Coulbourn Instruments, Allentown, PA) which were individually contained within sound- and light-attenuated isolation cubicles and equipped with fans (120VAC 3”; Radioshack, Fort Worth, TX) positioned at the top right corner of one wall. A house light was centered at the top of one test chamber wall (26 cm above the chamber floor). A speaker located just to the right of the houselight generated a 500 ms 70 dB (2500 Hz) conditioned stimulus (CS) tone. A reward delivery compartment (4.2 cm high × 3.3 cm wide × 3.5 cm deep) was centered at the bottom of the wall, 2 cm above the floor. Liquid reward (0.04 ml of 20% sucrose) was delivered, via tubing, into a small cup attached to the end of a dipper arm (Coulbourn Instruments, Allentown, PA) by a syringe pump (Razel Scientific, Saint Albans, VT). The dipper arm/reward cup lifted upward into the reward compartment to permit access to the liquid. Interruption of a photocell emitter and sensor (H20–94; Coulbourn Instruments, Allentown, PA) located on the sides of the reward compartment detected head entries via computer (Cobalt 4114 LabMax Series, Cobalt, Allentown, PA) and Habitest LabLink interface (Coulbourn Instruments, Allentown, PA). Time of head entries and withdrawals, sucrose delivery, and tone presentations were time-stamped with Coulbourn Instruments Graphic State Notation software with a 50 ms sampling rate.

Surgery

Rats weighed 300–340 g at the time of surgery and were anesthetized with Nembutal (50 mg/kg, i.p.; Henry Schein, Melville, NY); they also received atropine sulfate (0.25 mg/kg, i.m.; Henry Schein, Melville, NY) to reduce bronchial secretions. After being secured in a stereotaxic instrument (Stoelting Co., Wood Dale, IL), they were implanted with 23 gauge bilateral stainless steel guide cannulae (Plastics One, Roanoke, VA) according to standard flat-skull stereotaxic procedures. Stainless steel stylets (Plastics One, Roanoke, VA) were placed inside guide cannulae to prevent blockage.

Stereotaxic coordinates for drug infusions were: NAc core (anterior-posterior [AP] +1.7; medial-lateral [ML] ±1.8; dorsal-ventral [DV] −7.1 mm relative to the skull surface), dorsomedial striatum (DMS) (AP +1.7, ML ±1.8, DV - 5.1 mm), and NAc shell (AP +1.7; ML ± 0.8; DV −7.4 mm). A 6° lateral-to-medial angle was employed in order to avoid passage through lateral ventricles for implantations to the NAc core and DMS. A 12° lateral-to-medial angle was employed to target the NAc shell. Coordinates above correspond to those of the target structures. To reach these structures, internal cannulae extended 2 mm beyond the guide cannulae, as described below.

After surgery, animals received a subcutaneous injection of 5 ml of bacteriostatic (0.9%) saline (Henry Schein, Melville, NY) for hydration and spent the recovery period on a warm surface. Upon gaining consciousness, a single subcutaneous injection of Rimadyl (5 mg/kg; Henry Schein, Melville, NY) was administered for pain relief. Rats were given a minimum of five days for recovery in the home colony before behavioral training began. During the first three days of recovery, animals were given ad libitum food and water, and were then returned to the 22 h food restriction schedule for the remainder of the experiment.

Drugs and microinfusions

D1 antagonist R(+)-7-chloro-b-hydroxy-3-methyl-1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepine hydrochloride (SCH23390; 1 or 2 μg/side) was dissolved in sterile distilled water. AMPA antagonist 6-cyano-7-nitroquinoxaline-2,3-dione (CNQX disodium salt; 1 μg/side) and NMDA antagonist 2-amino-5-phosphonopentanoic acid (AP5; 2 μg/side) were dissolved together in phosphate buffered saline for combined blockade of ionotropic GLU receptors. All drugs were delivered in a volume 0.5 μl/side.

Prior to each bilateral microinfusion, internal cannulae (30 gauge; Plastics One, Roanoke, VA) were inserted into, and extended 2 mm beyond, the guide cannulae. The internal cannulae were connected to a 10 μl microsyringe (Hamilton Co., Reno, NV) with polyethylene tubing (PE-50, Plastics One, Roanoke, VA). Infusions of were driven with a microdrive pump (Razel, Stamford, CT) that delivered fluid at a rate of 0.5 μl/min. Internal cannulae were left in place for an additional 60 s to allow for diffusion. Rats were tested within 5 min after internal cannulae were removed.

Habituation and sucrose acclimation

Prior to training, rats received at least three days of daily handling, and were habituated to 20% liquid sucrose in their home cages during their daily 2 h feeding period. They then received an additional day of sucrose acclimation in which the rat was placed in a test chamber with a bolus of sucrose in the reward cup of the dipper. Once the rat consumed the sucrose, it was removed from the chamber and placed in once again to consume an additional bolus of sucrose. If the rat did not consume both boluses within 45 min the entire procedure was repeated the next day, ensuring that rats were accustomed to sucrose consumption from the reward compartment before cued approach training sessions began.

Behavioral procedure

Rats received daily drug-free 28-trial training sessions for either three (Early Training) or nine (Extended Training) consecutive days. The house light was illuminated from the start to the end of the session. During each trial, the 500 ms tone was presented on a variable-time 70 s inter-trial interval (ITI); with a minimum ITI of 30 s. A dipper containing liquid sucrose was raised 250 ms after the onset of the tone. If the animal failed to enter its head into the reward compartment within 10 s of tone onset, the dipper was lowered, and the next ITI began. If the head entry occurred within 10 sec of cue onset, the dipper remained raised for 4 s beyond the entry time to allow for reward consumption. During the period when the dipper was raised, the reward compartment was illuminated. At the cessation of each session (32 min average duration) animals were returned to their home colony where they received food ad libitum for two hours. After the training phase, animals received test sessions over two consecutive days. Test sessions were preceded by drug or vehicle infusion but were otherwise identical to the training sessions.

Experimental design and statistical analyses

Dependent measures were baseline head entries (measured during the 10 s before the CS), missed trials (trials for which latency to enter the reward compartment upon CS presentation was 10 s or longer), and latency to enter the reward compartment upon CS presentation (with missed trials excluded). In addition, the time of each head entry into, and exit out of, the reward compartment was timestamped, to generate trial-by-trial head entry raster displays for individual subjects and for experimental groups.

Dependent measures collected during the 3 or 9 drug-free training days prior to experiment 1 test sessions were subjected to repeated-measure ANOVAs. In experiment 1 test sessions, dependent measures were examined following NAc infusion of vehicle, 1 or 2 μg/side of SCH23390. In order to avoid repeated-dose effects of the DA antagonist (Fowler, 1990), and to minimize brain tissue damage associated with multiple infusions, each animal was tested under the influence of vehicle and a single SCH dose (1 or 2 μg/side). Different drug doses were therefore examined in different groups of rats, and each dose was compared to vehicle as a within-subject factor with the order of vehicle and SCH infusion counterbalanced. A 3-way ANOVA examined the effects of early vs extended training (Training), vehicle vs drug session (Treatment), and 1 vs 2 μg SCH groups (Dose).

In experiment 2, vehicle or the high dose of SCH was infused in two anatomical sites neighboring the NAc core (NAc shell or DMS). Each animal was tested under the influence of vehicle and SCH in a counterbalanced ordered. Animals were tested only during the early stage of training, i.e., at the stage where SCH effects on cued approach were seen in experiment 1. For each site, drug effects on dependent measures were therefore analyzed with paired t-tests examining performance during SCH compared to vehicle sessions.

Experiment 3 dependent measures were subject to a two-way ANOVA that examined the effects of vehicle vs the AMPA/NMDA antagonist (within subject) and early vs extended training (between subject).

Histology

At the end of the experiment, rats were deeply anesthetized with Nembutal and perfused transcardially with 0.9% isotonic saline followed by 10% formalin. Brains were harvested and stored in a post-fixative solution of formalin for 24 h before being transferred to a 30% sucrose-formalin mixture. They remained in this mixture until they sank and were then sliced on a microtome in 40 μm sections. Sections were mounted on glass slides, stained with cresyl violet and viewed with light microscopy to verify correct placement. Group n’s in the experiments below refer to subjects with cannulae correctly located within target regions (Fig. 1). Rats with off-target cannula placements were excluded from analyses.

Fig. 1. Microinjection sites of all rats included in statistical analyses.

Locations of bilateral microinjection sites for infusions to the NAc core (Early Training SCH 1 μg ⬛, Early Training SCH 2 μg ⬤, Extended Training SCH 1 μg ⬛, Extended Training SCH 2 μg ⬤, Early Training CNQX/AP5 ◆, Extended Training CNQX/AP5 ◆); DMS (SCH 2 μg ✚); and NAc shell (SCH 2 μg * ). Numbers on each coronal section indicate the distance anterior to bregma (Paxinos and Watson 1998).

EXPERIMENT 1. THE ROLE OF NAC CORE D1 TRANSMISSION IN CUED APPROACH DIMINISHES WITH EXTENDED TRAINING

Early in training, systemic D1 receptor blockade increases the latency to approach a food source in response to a brief auditory cue but fails to disrupt the same behavior in well-trained animals (Choi et al., 2005; Bespalov et al., 2007). Because DA activity within the NAc core is implicated in the initiation of learned appetitive behavior (Salamone and Correa, 2012; du Hoffmann and Nicola, 2014), experiment 1 asked whether D1 receptor blocker SCH23390 (add ref) infused to the NAc core would disrupt initiation of cued approach only during early stages of training.

Separate groups of rats received either three (Early) or nine (Extended) days of drug-free cued approach training followed by two consecutive test sessions (see Behavioral Procedure) in which they received an intra-NAc core infusion of D1 antagonist SCH23390 (1 or 2 μg/side) prior to one session, and vehicle prior to the other. The order of D1 antagonist and vehicle infusions were counterbalanced so that half the rats received the vehicle first, and the other half received the drug first. The number of animals in the four experimental groups was Early Training Veh/SCH 1 μg SCH (n = 10); Early Training Veh/SCH 2 μg (n = 10); Extended Training Veh/SCH 1 μg (n = 10); and Extended Training Veh/SCH 2 μg (n = 11).

Results

Over the course of drug-free training, latency decreased in both the Early Training (days 1–3; F_{(2, 38)} = 6.4, p = .004) and Extended Training (days 1–9; F_(8,160) = 29.2, p < .0001) groups. Similarly, the proportion of trials missed decreased during these drug-free sessions in both groups, Early, (F_(2,38) = 13.31, p < .0001); Extended (F_(8,160) = 11.88, p < .0001). The drug-free acquisition of the cued approach for the Early (panel A) and Extended (panel B) Training groups is shown in Fig. 2. The significant increase in speed and probability of cued approach was not a result of an increase in the overall frequency of head entries because baseline head entry frequency was unchanged during days 1–3 (Early Training group F_(2,38) = 0.94, p = .40) and decreased over days 1–9 (Extended Training group; F_(8,160) = 6.89, p < .0001). Rats in both the early- and extended-training groups would be assigned to either low (Veh vs 1 ug SCH) or high dose (Veh vs 2 ug SCH) groups for two subsequent days of testing following intra-NAc core infusions. Animals assigned to the low vs high dose groups did not differ with respect to their final drug-free performance for any of the behavioral measures (latency, missed trials, or baseline approach; p = n.s. for all).

Fig. 2. Drug-free cued approach performance for Early and Extended training groups.

For both panels, the left y-axis and grey line depict mean proportion of CS trials “missed”; the right y-axis and black line depict the latency to enter the compartment upon CS presentation (with missed trials excluded).

The effect of NAc core D1 receptor blockade was examined in animals tested on days 4 and 5 (early training) or days 10 and 11 (extended training) (Fig. 3A). Figure 3B shows the cued approach performance of two representative rats, each receiving an infusion of vehicle and SCH23390 during either early (left panel) or extended (right panel) training. Horizontal lines represent the animal’s head in the reward compartment. As can be seen, animals under the influence of the D1 receptor blocker missed a large proportion of trials during early training (empty rows after the CS in the bottom left panel). The drug produced virtually no missed trials when administered late in training. A similar pattern was seen in group data, as described below.

Fig. 3. Effects of intra-NAc core SCH23390 on cued approach as a function of training.

A) Tests of cued approach during early vs extended training; B) The 28 consecutive trials of each session are plotted on the y-axis (bottom to top). Time relative to CS presentation (0 sec) is shown on the x-axis. Each horizontal line represents the occurrence and duration of a head entry into the food compartment. The left panels show the performance of a rat pretreated with vehicle (top) or SCH (bottom) on days 4 and 5 of training, respectively. The right panels show performance of a rat that received vehicle (top) or SCH (bottom) on days 10 and 11, respectively. C) The mean probability that the head was in the reward compartment for groups of rats under the influence of vehicle or SCH23390 during consecutive 0.1 sec bins. Lower peaks after CS presentation in rats receiving intra-NAc SCH23390 during early training reflect a reduced proportion of trials in which rats responded to the CS. The vehicle controls for low and high dose SCH groups did not differ significantly (p = n.s.) and were pooled for representation of Veh performance above.

Figure 3C illustrates the mean probability of head entry into the reward compartment for groups of rats under the influence of vehicle or SCH23390. Increased latency is shown as a shift to the right, while missed trials reduce the height of the peaks. The probability of a missed trial was greater in the SCH compared to vehicle test sessions (main effect of Treatment, F_{(1, 33)} = 15.97, p = .0003), and these SCH vs Veh differences were larger in the 2 μg compared to the 1 μg SCH groups (Treatment x Dose, F_{(1, 33)} = 11.27, p = .002). The magnitude of the dose-dependent increase in misses differed according to training stage (Treatment × Dose × Training, F_{(1, 33)} = 4.37, p = .04), while latency (with missed trials excluded) showed a marginal Treatment × Dose × Training interaction (F_{(1, 33)} = 3.63, p = .07). Figure 4 shows the effects of SCH on approach latencies and missed trials during early vs extended training.

Fig. 4. Effects of NAc core SCH23390 on cued approach latency and misses.

The panels show the effects of SCH infused during early or extended training on latency (left) and proportion of trials missed (right). Asterisks denote significant differences between SCH dose and within-subject vehicle control scores, *p < .05, **p < .01. Plus sign denotes a significant difference between Early and Extended training groups at the high SCH dose, +p < .05. Vehicle controls for low and high dose SCH groups did not differ significantly, p = n.s., and were pooled for representation of vehicle performance in the graphs.

Unlike cued approach, which was less vulnerable to SCH during extended training, SCH-induced suppression of baseline approach (F_{(1, 33)} = 22.83, p < .0001) did not differ according to stage of training (Treatment x Training, F(1,33) = 0.36; p = 0.55; Treatment x Dose x Training, F_{(1, 33)} = 1.19, p = .28). As can be seen in Fig. 5 (right panel), the high dose of SCH suppressed baseline head entries during both early (t₍₉₎ = 3.10, p = .01) and extended (t₍₁₀₎ = 2.60, p = .03) training. Due to the unusually low levels of baseline approach in the vehicle control condition for the low dose of the drug during extended training (left panel, extended training, vehicle), vehicle data were not pooled across doses as they were in the line graphs of the Fig. 4. Taken together, the data suggest that CS-elicited approach becomes less dependent upon NAc D1 receptors following extended training, while approach behavior per se remains D1 mediated.

Fig. 5. Effects of NAc core SCH23390 on baseline approach.

The high (2 μg) dose of the D1 antagonist (right panel) suppressed baseline approach during early and extended training stages. The low dose (left panel) produced a moderate suppression of baseline approach during early training. Each line represents the performance of an individual rat following vehicle vs drug infusion. The smaller number of visible lines in the low dose extended training condition reflect the fact that a number of these animals showed few or no baseline head entries following either infusion. These low levels of baseline approach may have obscured suppressive effects of the low drug dose on baseline approach. Asterisks denote significant differences from the within-subject vehicle control, *p < .05.

EXPERIMENT 2. D1 MEDIATION OF EARLY-STAGE CUED APPROACH IS ANATOMICALLY SPECIFIC

Experiment 1 results point to a D1 receptor role in cued approach during early but not late training. Experiment 2 asked whether the early-stage D1 mediation of cued approach was anatomically specific. It is possible, for instance, that SCH spread to regions near the NAc core such as the shell or the overlying DMS, and that SCH actions in these other regions disrupted early-stage cued approach. To examine this possibility, the D1 antagonist was infused directly to the medial shell or the DMS (2 mm above the site of experiment 1 NAc core infusions).

Rats received 3 days of drug-free training, and on test days 4 and 5 (early training stage) received pre-session vehicle and high dose (2 μg/side) SCH infusions in a counterbalanced order. Drugs were infused either to the NAc shell (n = 9) or the DMS (n = 10) (Fig. 6A).

Fig. 6. Effects of SCH to the NAc shell (top) or DMS (bottom) on cued approach during early training.

A) Tests of cued approach during early training; B) Neither NAc shell nor DMS SCH23390 affected the probability of a cued head entry, despite reducing head entry probability during the ITI.

Results

Neither NAc shell nor DMS infusions of the D1 antagonist affected the probability or latency of cued approach. Figure 6B shows the probability of inserting the head into the reward compartment before and after CS presentation while under the influence SCH23390 in the NAc shell (top panel) or DMS (bottom panel). Missed trials were unaffected by SCH to the NAc shell (t₍₈₎ = 0.35, p = .74) or DMS (t₍₉₎ = 1.66, p = .13). Approach latency was also unaffected by SCH infusion to the NAc shell (t₍₈₎ = .046, p = .96) or DMS (t₍₉₎ = 1.33, p = .22). In contrast to cued approach, baseline approach during the ITI was suppressed by NAc shell SCH, t₍₈₎ = 2.65, p = .03, and marginally suppressed by SCH infused to the DMS t₍₉₎ = 1.89, p = .09. The results suggest that the disruption of cued approach observed following NAc core SCH infusions during early training was anatomically specific, at least to the extent that it was not observed in DA target sites dorsal or medial to the site of NAc core infusions.

EXPERIMENT 3. LATE STAGE CUED APPROACH: A SUBSTRATE SHIFT

Experiments 1 and 2 point to D1 receptor mediation of early-stage cued approach in the NAc core but not in surrounding DA target regions, and a diminished D1 role with extended training. A number of neurobiological mechanisms might lead to a diminished role of NAc core D1 transmission in behavioral performance. Neither systemically administered D1 nor D2 receptor blockers disrupt cued approach after extended training (Choi et al., 2005), suggesting that the well-learned behavior is not mediated by D1 transmission within other DA target sites.

Either of two general explanations could account for the reduced involvement of NAc core DA in the well-acquired cued approach. First, while cued approach depends upon NAc core D1 transmission during early stages of training, NAc core D1 receptors may play a diminishing role in the well-acquired behavior. Like other parts of the ventral and dorsal striatum, the NAc core receives excitatory GLU inputs which drive the activity of spiny projection neurons, and these excitatory input-output connections are under DA’s modulatory influence (Horvitz, 2002; Lovinger, 2010; Shiflett and Balleine, 2011). It is possible that as DA’s role in cued approach diminishes, excitatory GLU transmission through the NAc core continues to mediate the behavior (depicted in Fig. 7, top panels). If so, then antagonism of NAc core ionotropic GLU receptors, which prevents firing in spiny projection neurons (Hu and White, 1996; Sandstrom and Rebec, 2003), should disrupt cued approach both before and after extended training.

Fig. 7. Hypothetical mechanisms mediating well-acquired cued approach.

The figure depicts two possible mechanisms to account for the diminished role of NAc core D1 receptors in cued approach during extended training: 1) GLU inputs to the NAc core may gain the ability to promote behavioral expression even when D1 transmission is blocked; 2) The NAc core itself may play a diminished role in behavioral expression.

A second general explanation is that the NAc core itself is no longer part of the neural circuitry that critically mediates the well-acquired cued approach (Fig. 7, bottom panels). According to some models, DA actions in the striatum are necessary for behavioral acquisition and performance during early stages of learning, but eventually cortico-cortical connections come to mediate behavioral performance (Carelli et al., 1997; Ashby et al., 1998; Ashby et al., 2010; Crossley et al., 2016). If the well-acquired behavior were to shift to the cortex, and/or other anatomical substrates, then disruption of NAc core ionotropic GLU activity should disrupt the learned behavior during early but not extended training.

Co-antagonism of AMPA and NMDA receptors greatly reduces or prevents the activation of NAc output neurons (Hu and White, 1996). In experiment 3, rats received 3 (n = 9) or 9 (n = 9) drug-free sessions as in experiment 1 and were tested under the influence of vehicle or combined AMPA/NMDA antagonists CNQX/AP5 in counterbalanced order prior to test sessions on day 4 and 5 (early training) or 10 and 11 (extended training) (Fig. 8A).

Fig. 8. Effects of intra-NAc core ionotropic GLU receptor blockade on cued approach probability as a function of training.

A) Tests of cued approach during early vs extended training; B) During early training, AMPA/NMDA receptor blockade reduced head in probability (lower peak) and increased head entry latency (rightward shift).

Results

The key question of interest was whether or not blockade of NAc core GLU receptors would disrupt the learned behavior during early and not late stages of training, or whether it would disrupt behavioral expression during both stages. The effect of intra-NAc core AMPA/NMDA blockade on ‘missed trials’ depended upon stage of training, Treatment × Training, F_(1,14) = 5.53, p = .03. CNQX/AP5 increased the proportion of missed trials during early training (t₍₈₎ = 3.54, p = .008), but not in well-trained rats (t₍₈₎ = 1.65, p = .14) (Fig. 8B and 9). Similarly, the effect of AMPA/NMDA blockade on latency depended on training stage, Treatment × Training, F_{(1, 14)} = 7.36, p = .02. Blockade of NAc GLU receptors increased latency to respond to the cue during early (t₍₈₎ = 4.60, p = .002) but not extended (t₍₈₎ = 0.98, p = .36) training. GLU receptor blockade did not significantly affect baseline head entries during either early (t₍₈₎ = 1.29, p = .23) or extended (t₍₈₎ = 0.01, p = .99) training (Fig. 10). NAc GLU receptor blockade therefore did not reduce head entries per se, but specifically reduced head entries elicited by the reward-predicting cue during early training.

Fig. 9. Effects of ionotropic GLU receptor blockade on latency and trials missed.

Blockade of NAc core AMPA/NMDA receptors increased latency (left) and missed trials (right) during early training only. The smaller number of visible lines in the missed trials/extended training condition reflect the fact that a number of rats showed overlapping data points. Asterisk denotes a significant difference from within-subjects vehicle control, *p < .05.

Fig. 10. Effects of ionotropic GLU receptor blockade on baseline approach.

Blockade of NAc core AMPA/NMDA receptors did not significantly affect ITI head entries during either training stage.

DISCUSSION

The NAc plays a critical role in the acquisition (Chang et al., 2012; Dalley et al., 2005) and expression (Parkinson et al., 2000; Parkinson et al., 2002; McGinty et al., 2013; Hamel et al., 2017) of conditioned reward approach. It is known, for instance, that the magnitude of cue-elicited NAc neuronal activation predicts latency to approach a reward-associated lever (McGinty et al., 2013). Further, intra-NAc DA receptor blockade disrupts both cue-evoked neuronal excitation and approach latency (du Hoffmann and Nicola, 2014). Here we show for the first time that this NAc DA role (and in fact overall NAc role) in reward approach is temporary, I.e., it is restricted to early stages of learning.

The results showed that during early learning NAc core D1 receptor blockade disrupted approach to a reward compartment normally elicited by a cue signaling food delivery. However, the results further showed that this disruption was greatly attenuated in animals that had received extended training, suggesting that while NAc core DA plays a critical role in cued approach, the role diminishes with training. After extended training, this reward-directed behavior occurs with normal probability and speed even when NAc D1 transmission is blocked.

Previous work showed that systemic D1 receptor blockade disrupted the speed and probability of cued approach during early but not later stages of training (Choi et al., 2005; Bespalov et al., 2007) and that systemic D2 receptor blockade had no effect on either of these measures regardless of training stage (Horvitz and Eyny, 2000; Choi et al., 2005; Choi et al., 2009). The present results suggest that early-stage cued approach does not depend upon D1 transmission generally throughout the brain, but specifically within the NAc core. D1 receptor blockade in the NAc shell or DMS failed to disrupt cued approach expression.

While the disruptive effect of NAc core D1 receptor blockade on cued approach latency and probability diminished with extended training, the intra-NAc core D1 antagonist suppressed spontaneous approaches (during ITIs) both early and late in training. One might describe suppression of baseline approach as reflecting a reduced ability of contextual cues in the environment to elicit approach, or a more general motor suppression that reduces approach behavior during the ITI. In light of this suppression, the fact that cued approach is intact in well-trained animals under the influence of the drug shows that the drug did not merely produce a general motor impairment which would have been expected to be seen later in training as well. Instead the results suggest that extended training leads to a specific reduction in the role of D1 transmission in mediating approach responses to reward-paired cues.

One explanation for the diminishing effects of SCH on cued approach is that the well-acquired conditioned behavior no longer depends upon NAc D1 receptors but is still mediated by NAc core excitatory transmission (depicted in Fig. 8, top panels). However, experiment 3 showed that, like the D1 receptor antagonist, blockade of ionotropic GLU receptors in the NAc core also disrupted cued approach during early but not extended training. This suggests either that the behavioral substrate shifts from the NAc core to other brain areas (Fig. 8, bottom panels), or comes to include other brain regions in addition to the NAc core.

The reduced NAc core involvement in the speed and probability of performing the well-learned cued approach is of particular interest in light of recent findings that extended training leads to an uncoupling between NAc core DA concentration and behavioral performance (Collins et al., 2016). During early training, rats executing a two-lever sequence reinforced by sucrose reward showed elevations in NAc core DA concentration that ramped up within trials, eventually peaking before the first lever press of the sequence. These within-trial DA elevations were inversely correlated with the latency to complete the sequence for the same trial. However, after extended training (beyond asymptotic performance), within-trial DA concentrations declined and were no longer correlated with the speed of behavioral performance. In another study, in rats nosepoking for cocaine infusions, phasic NAc DA signals tied to the behavioral response were seen early in training and were attenuated with extended training (Willuhn et al., 2014). Further, the decreases in DA concentration with extended training were strongly tied to the emergence of compulsive drug taking. While these data suggest that NAc DA signaling tied to learned behavioral performance diminishes with extended training, the present results importantly demonstrate that the critical role of not only NAc DA, but also NAc excitatory transmission, in performance of learned behavior diminishes with training.

Earlier work suggests that the involvement of dorsal striatal regions also changes in well-trained animals. For instance, DMS neuronal activations seen mostly at the decision point of a t-maze diminish after extended training (Thorn et al., 2010; Smith and Graybiel, 2014). Activations in the dorsolateral striatum (DLS) shift over the course of task performance as well. They are initially evoked as animals move throughout the t-maze, but are eventually seen primarily at the start and end of the maze (Barnes et al., 2005). Rats lever-pressing on variable interval schedules shift from a DMS-mediated goal-directed mode early in training to an eventual DLS-mediated stimulus-response mode (Yin et al., 2004).

Despite the currently observed reduction in the role of the NAc core in conditioned approach, it is possible that some reward-directed behaviors may not become independent of the NAc core, at least within the relatively short time frame observed here. For instance, it is known that NAc DA depletion reduces rates of responding on fixed ratio 16 or fixed ratio 64 schedules even in rats that received many weeks of operant training prior to NAc DA loss (Aberman and Salamone, 1999). This presumably reflects the fact that high-effort behaviors are particularly vulnerable to compromise of NAc DA transmission (Salamone et al., 2016).

It also remains to be determined whether sign-tracking behavior becomes NAc and/or NAc-DA independent. While the current study was not designed to dissociate sign- from goal-tracking components of conditioned approach, approach to the reward delivery compartment is typically an example of goal tracking. Other work, separating sign- from goal-tracking, shows that while DA receptor blockade disrupts the expression of both (Flagel et al., 2011), NAc and NAc DA play particularly important roles in sign-tracking (Flagel, et al., 2011; Saunders and Robinson, 2012; but see Chang and Holland, 2013). Some evidence is consistent with the possibility that sign-tracking eventually becomes NAc independent. For instance, rats receiving excitotoxic lesions of the NAc prior to training are slow to acquire an autoshaped lever press (Chang et al., 2012). However, with a sufficient number of training sessions, the same animals come to show levels of sign-tracking comparable to those seen in non-lesioned subjects. The fact that animals were eventually able to express normal levels of the behavior suggests that sign-tracking may eventually become NAc-independent. Further, extended training has been shown to both reduce CS-evoked NAc DA release, and reduce disruptions in sign-tracking normally produced by systemic DA receptor blockade (Clark et al., 2013). While these data converge to suggest that sign-tracking may become DA-independent with extended training, NAc core DA antagonist infusions during either early or later stages of training were found to produce similar sign-tracking disruptions (Fraser and Janak, 2017). Therefore, one cannot rule out the possibility that the NAc core continues to mediate sign tracking even after extended training.

Another point not addressed in the current work relates to NAc involvement in CS+ approach (examined here) vs approach suppression following presentation of a CS-. In animals pretrained on a sign-tracking task involving an autoshaped CS+ and CS- lever, the primary effect of NAc core AMPA receptor blockade on response expression was to increase responding on the lever associated with non-reward (the CS-) (Di Ciano et al., 2001). While we observed that the role of NAc core ionotropic GLU in conditioned responses to a CS+ diminishes with extended training, it remains to be determined whether extended training also reduces the role of NAc ionotropic GLU transmission in conditioned behavioral inhibition.

Some empirical and theoretical work has suggested that well-acquired behavior eventually becomes independent of the striatum altogether (Carelli et al., 1997; Ashby et al., 2007; Ashby et al., 2010). In support of the latter view, extended training greatly reduces the number of DLS neurons showing activations time-locked to either an operant lever press (Carelli et al., 1997) or operant head movement (Tang et al., 2007). Ashby (Ashby et al., 2010) describes a theoretical model in which DA-modulated basal ganglia activity and plasticity during early training promotes task-relevant plasticity in the cortex, eventually leading to the establishment of basal ganglia-independent behavioral performance. From this view, extended training permits cortical sensory inputs to directly access task-relevant cortical motor outputs via cortico-cortical pathways. As a result, well-trained behaviors become independent of DA and basal ganglia mediation.

Precisely how cue-driven NAc core output neurons (du Hoffmann and Nicola, 2014) drive downstream circuitry mediating approach is unclear. Major recipients of NAc core output include the ventral pallidum and substantia nigra pars reticulata (SNr) (Mogenson et al., 1983; Zahm and Heimer, 1993; Deniau et al., 1994). Pauses in SNr activity are associated with both disinhibition of brainstem motor regions (Kim et al., 2017) and periods of forward locomotion (Freeze et al., 2013) and head movement (Schmidt et al., 2013), i.e., behaviors seen as rats approach the reward compartment. It is therefore possible that approach behavior occurs when reward-paired cues activate NAc core GABAergic neurons, which inhibit SNr activity and disinhibit brainstem motor regions to permit approach. While the details of the circuitry driving cued approach remain to be elucidated, the present results reveal that with extended training the behavior becomes less dependent upon the NAc core. This involves not only a reduced dependence on NAc core D1 transmission but on NAc core excitatory transmission as well. The precise neuroanatomical and/or other neurobiological changes leading to a reduced NAc role in cued reward approach is a key question for future work.

ABBREVIATIONS

AP

anterior-posterior

AP5

2-amino-5-phosphonopentanoic acid

CNQX

6-cyano-7-nitroquinoxaline-2,3-dione

CS

conditioned stimulus

DA

dopamine

DMS

dorsomedial striatum

DV

dorsal-ventral

GLU

glutamate

ITI

inter-trial interval

ML

medial-lateral

NAc

nucleus accumbens

SCH or SCH23390

R(+)-7-chloro-8-hydroxy-3-methyl-1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepine hydrochloride

SNr

substantia nigra pars reticulata